Notes: Check-in for Hyrax-Valkyrization Community Effort March 7-18, 2022



HomeDeveloper ResourcesCheck-in Notes 
 Table of Contents



We'll record discussions and updates here in case there's anyone who misses the call

Week 1


Updates Mar 7

Issue #5461 (Anna) - batch edit not working with valkyrie

  • just that batch edit doesn't work
  • got nurax-pg up and running

Issue #5462 (Eliot) - editing parent work error

  • got nurax-pg up and running
  • got error following steps, but it is different
  • working on figuring that one out 

Issue #5480 (Trey) - file upload multiple problems

  • issue was filesets don't have a thumbnail
  • there are multiple errors
  • FIXED: valkyrie ingest job was failing because test and app worked a little different.  Can't copy carrier wave stream directly into a file. 
  • FIXED: characterization wasn't working because not able to set file set id 
  • FIXED: missing title 
  • derivatives don't run
  • rubocop doesn't like long method

Issue #5449 (Lynette) - nested collection form exceptions

  • working on nested collection form issues
  • FIXED: method missing for find
  • working on #nestable? method missing

Issue #6490 (Shaun) - UI consistency

  • Created a PR for a changed label.
  • Would recommend taking out icons from action menu

Daniel

  • planning to work with dassie with valkyrie setup
  • will look at setting up nurax-pg

Chris

  • looking to pick up something tomorrow

Updates Mar 8

Anna (#5461) - batch editing

  • pairing with Eliot
  • batch edit will need to be converted to a change set
  • perhaps it would be better to tear it out
  • There's another project for batch editing - bulkrax. Maybe putting bulkrax into hyrax is a potential?
  • Bulkrax may be more for bulk ingest.
  • Julie may know if there's something on the roadmap to do something for batch edit
  • Batch edit can be turned off. Maybe we can just make it so if Valkyrie's turned on, batch edit is left flipped off? Toggle based off Hyrax.config.disable_wings.


Chris (#5463) - missing _pcdm_partials

  • got docker setup working
  • taking a look at searches that return pcdm collections
  • looks like it has something to do with routes
  • also may need to override a blacklight helper for path resolution
  • look at it more tomorrow
  • would it work to add partials instead of overriding by rendering the _collection partial from _pcdm_collection partial.  Probably would see similar issues.


Daniel (#4788) - fileset id not indexed

  • thumbnail ids are being set on the model, but not indexed
  • first step is to index the fileset id
  • next step is to get access to the files
  • worked on getting nurax-pg docker environment be easier


Eliot (5462) - error editing with parent work

  • working on parent work editing
  • somewhat blocked by nurax-pg setup
  • can't replicate in Hyrax
  • registering postgres adapter


Lynette (5449) - collection nesting

  • lots of moving parts
  • exploring current process
  • looking to see if can address the bigger issue of not running the collection indexing unless membership has changed


Shaun (5490) - UI consistency

  • working on ui consistency tickets
  • 2 PRs passing - removal of icons, renaming
  • gonna pick up table sort and column sorting next
  • going to have to sort by solr on server side


Trey (5504) - file derivatives

  • looked some at derivatives issue
  • need parallel of current derivatives job that is valkyrie specific
  • doesn't store derivatives in Fedora currently
  • probably look at tomorrow



Updates Mar 9

Anna (#5461) - batch editing

  • Found a bug that's the cause of the ticket about correctly pulling the permissions that a work is allowed to have - being able to set an embargo/lease. See https://github.com/samvera/hyrax/pull/5514
  • Looking at #5457 - works deposited via proxy getting the right visibility.


Chris (#5463) - missing _pcdm_partials when searching with collections

  • Merged PR resolving the issue of _pcdm collections not being in search results. Ended up cloning a feature test and did some work to make it run against the Valkyrie solr core. Useful patterns here: https://github.com/samvera/hyrax/pull/5515/files
  • Won't be back on the sprint until Friday.


Daniel (#4788) - fileset id not indexed

  • Looking at thumbnail/representative ID indexing. Tricky because derivatives aren't being made. Mostly making sure how the current system works - it's a little convoluted in places.
  • Spent some time trying to solve weird AccessControls::Embargo reloading thing. It seems like ActiveFedora::Relation keeps a cached reference to a class which becomes a different class on reload. Tried to hack it to reload the class, but it got unhappy about wings models.


Eliot (5462) - error editing with parent work

  • Just started looking at #5462 again. Trying to find where in nurax-pg the problem lies - tried to spin up a fresh rails app the same way nurax-pg was and getting the same error. Still tracking down.


Lynette (5157) - collection metadata should be same as in AF

  • (5449) - collection nesting - not worked on today to finish up 5157
  • Update batch delete to use transactions - originally used ActiveFedora to delete.


Shaun (5490) - UI consistency

  • Working on inventorying UI for consistent language/works
  • 3 PRs in. Ping slack or is the github ping better?
    • Slack ping is better.
  • Might need recommendations on a good issue to pick up next.


Trey (5504) - file derivatives

  • Comments in the ticket.

Tao

  • Working on testing.
  • Found many issues with collections. Collection creation/discovery of collections. Those issues are blocking his testing.
  • Lynette is working on nested collections - hopefully there will be work on that soon.
  • Lynette will deploy a new version of nurax-pg with the fixed collection pages.
  • Will Lynette announce new PRs?
    • Lynette pushes with a diff you can browse.

Can someone merge their institution's PRs?

  • Yes

Has Julie approved all the icon 

Shaun needs a new issue

  • Can work on feature tests - can pair with Chris today or Friday.
  • If wants help with the solr backend stuff, can reach out to Lynette for pairing.

Updates Mar 10

Anna (#5457) - proxy deposit visibility always private

  • off this afternoon
  • continuing to work on Issue #5457
  • looked at transactions related to this
  • continuing with this tomorrow

Chris

  • off today
  • Issue #5463 closed by PR #5515 - missing _pcdm_partials when searching with collections

Daniel (#4788) - fileset id not indexed

  • thumbnails are now indexing
  • question about how thumbnails are indexed in hyrax as hasRelated_image_ssim; not intuitive, but sticking with what they have always been
  • in dassie, trying to show the image in the UV, but seems like the manifest isn't there
  • one other thing want to do is the changes in nurax-pg to make docker easier to work with

Eliot (5462) - error editing with parent work OR #5461

Lynette (5157) - collection metadata should be same as in AF

  • PR #5157 - Collection basic metadata should be opt-in by the application
  • PR #5511make change set forms allow multiple values for title
  • PR #5518 - adds deprecation notices for transactions/steps that are no longer supported
  • Trying to get back to collection nesting

Shaun (5490) - UI consistency

  • PR #5508 - changing labels caused a bunch of feature tests to fail requiring tests to be tweaked; switching to ids to avoid in the future

Tao - testing with spreadsheet

  • #5517 Search result page of collections later pages create error

Trey (5504) - file derivatives

  • local meeting this afternoon
  • continuing to make progress on derivatives

Updates Mar 11

Chris

  • Starting Invalid authenticity token issue

Anna

  • Working on when you try to make a work as a proxy depositor you can't set it to public visibility.
    • It also doesn't go through as a proxy depositor.
    • Figured out where it's going wrong - working on a fix.
    • Problem was a race condition on the listener

Trey

  • Built out structure for derivatives job

Eliot

  • continuing work on parent editing issue
  • Blocked on a test that fails in CI (which uses engine cart)
  • May be helpful to reference the PR from earlier this week that added a feature test for searching.
  • Lynette can pair.

Lynette

  • Trying to separate out some work that allows for empty values and doesn't actually save them in the data.
  • Working on test failures
  • Then plan to go back to the nesting

Daniel

  • Thumbnail work - put up a branch and working on some failing tests
  • nurax-pg - PR with docker compose fixes ready for review
    • Uses hyrax main branch (doesn't try to mount a local development hyrax)

Tao

  • Used a new admin user to do testing on collection type
    • Still get an error when editing a collection type

Shaun

  • Working on PR#5508 - seems ready to go at this point, pending CI
  • Had to guard against nil date updated values. Use date uploaded as a backup value. May want to refactor at some point.
  • Maybe add a solr document method like document.latest_date?

Agenda:

  • Offers to pair.
  • Idea: rspec --only-failures: can we put the result in the artifacts of circleCI? And then recombine the 10 files to let us download it and use it locally.
  • Output of valkyrie index: sometimes the key is a string and sometimes a symbol. Can / should we use one or the other?
    • In the solr document they're all strings.
    • It would be easier to test if it were consistent. The base indexer uses strings.
    • Make a refactor ticket. We'll need to update a bunch of tests to do this.
    • In blacklight id is a symbol and everything else is a string.
    • Can/should we use with_indifferent_access?
  • Listeners
    • I don't think the listeners are used in active fedora. They often have code that kicks out of the method if there's an AF resource.
    • (In this case it doesn't)
  • Post in slack if you have a PR that needs to be review or re-reviewed!


Week 2


Updates Mar 14

Chris

  • I'm here this week part-time
  • Looking for a new ticket

Eliot

  • Continuing work from last week on parent editing issue
  • bug appears to be in PcdmMemberPresenterFactory - it's not fully tested and there's a subtlety with solr
  • Have a fix, refactoring it

Shaun

  • PR #5527 about date column consistency ready for review
    • The date modified column will be blank until the resource is updated
    • There's a draft PR to fall back on date uploaded (#5526)
    • One proposal is to create a solr field for the latest date
    • Another option is to back the logic up into the presenter
    • Lynette will look and recommend.
  • Planning to pick up one of the issues to add valkyrie feature tests
    • Offer to pair

Anna

  • Working on 5457 still.
  • Noticing that we pass resources directly rather than IDs. It would be good to inventory and move to passing IDs and loading objects, especially in listeners and jobs.

Lynette

  • Shepherding PRs
  • About to look at issues again to see what our biggest blockers are
  • We're doing good work!
  • Need review of #5472 (seeds)
  • Will go back to nested collections

Updates Mar 15

Eliot

  • Responding to feedback on #5516.
  • Discovered a new issue - can't remove children through the form. Writing a ticket and then will probably take it.

Chris

  • Planning to review branding work.
  • Will try to get to issue where members don't show up in search results on a collection tomorrow or later today.

Shaun

  • Question about reverting text strings to IDs
    • Provided an example regarding philosophy.
    • Two PRs where Justin did that. Anna just accepted it.
    • There's one more open PR - a few were changed, PR could still be reviewed.
  • Do tests need to be changed back...?
    • Will do some follow up here.
  • No blocking PRs for this group.
    • That open PR could be commented on.
  • Draft PR in for testing valkyrie collection creation. PR 5541.
    • Testing creation of collections with traditional collections and repeating all the steps for the valkyrie collections.
    • That right?
      • Lynette will take a look.
    • Lynette was going to look at search builders for the tables - Lynette might take a look at it to see if she can create something to pass sort to in order to move that issue forward.

Anna

  • Finishing touches on 5457. Adding specs/double checking logic. Should be ready soon.
  • Lynette will look at PR descriptions

Lynette

  • Collection branding is done! Tests are running now - works in UI.
  • Spent some time going through test worksheet and marking cleared things for retesting.

Trey

  • Manifests don't generate.
  • PR for derivative creation is up.
  • Getting UV working would be good.
  • Will also make a ticket for extracting text.

Julie

  • Checking in to see if there are things for her to look at from this morning.
  • Will try a few things that aren't part of main yet, but could be merged in. 
    • (Lynette) One for sure is Geonames where it gets whole state/country - make sure Julie's okay with that.
    • Changes to UI stuff, but that's all merged.
  • (Anna) Maybe we should release this rollback of the somethingsomethingActor.
    • Think it's the cause of a problem with Duke.
    • Julie's out Thursday/Friday this week. Is a release something we can manage by end of tomorrow, or maybe release 3.3.1 next week?
    • Lynette will look at that PR and see if it can just get in and get a release out tomorrow.
    • Does Julie have to be around for the release?
  • What about ActiveFedora reversion?
    • Happened
  • 3.3.1 release should happen after these 3 PRs are merged/done. If we want to do it tomorrow, Julie is around except for an hour in the afternoon.
  • Emory may join for maintenance next week. Any valkyrie issues expected to remain?
    • Yeah probably.
    • Lynette will be minimalist for the next couple of weeks.

Updates Mar 16

PRs merged: 28

Eliot 

  • Valkyrie Parent Resources to add/remove children. Adding a new transaction to make that happen. Just about done.

Shaun

  • Working on issue Valkyrie Collection Creation Testing #4590. Collection feature test is having some problems.  Not sure where UI is.  3 failing tests.  Can't find title and description field for one scenario.
  • Pulled out repeated test login into separate tests.  

Anna

  • About to look at the status of my open PR #5537. Concerns about backwards compatibility. We are changing it from invoking a background job that calls a service, to directly calling the service first and then invokes a background job. Lynnette has no problem with the transaction actor. Changing the way the job works is maybe a little harder. Lynette suggests perhaps deprecating the listener, but needs to think about it. Anna says that we will need to continue to support that listener which could be a nightmare. It's possible that there hasn't been a release since that listener, but Anna contends that it was introduced at least a year ago. If the listener is a concern we may have to revert that PR. If the Job is a concern, we could deprecate the Job and the Service and we can introduce new ones. Lynette wants to look at it again.
  • Question about use of FindBy vs FindByAlternateId...
    • Issue #4085
    • Lynnette says the way Wings is written, it uses FindByAlternateId when FindBy is called. 
    • Lynnette wonders if there will be a problem when data gets migrated as the NOID used may not be its actual id.

Thanya (not present)

  • Looked at the Default Search ticket. Will write up some notes on what she learned and will un-assign herself from the ticket.

Lynette

  • Wrapping up issue #5449 
  • Prioritize PR reviews and will take a look at the Default Search ticket.

Chris 

  • Working on issue that if a collection has works within it, they are not showing up on the count on the dashboard, and not showing up on the collection show. 
  • Working on a feature test for it.

Updates Mar 17

Eliot

  • Couple PRs in review column. Lynette reviewed PR about adding stuff to a PR, made a comment. Split off into separate ticket? Open to whatever.
    • Fine to do a separate ticket for that. We can merge what's in there now.
  • Another PR for review which enables search for child functionality in a Valkyrie edit page.
  • Pushing up a PR to fix 5452 to fix citation button.

Anna

  • Chat about deprecation concerns for work doing for 5457. Spent time working on deprecation stuff. Really close on all of the deprecations - for this PR just created need to fix a couple of tests. Want to address comments from previous PR.
  • Will make a couple tickets.
  • Started looking at the seed PR and then looked at deprecation stuff instead. Will try and get back to it.

Shaun

  • No updates, doing local work.

Lynette

  • Put in tests for nested collection. Just tested with main and something's not quite right. Getting an error again. Will take another look at it again.

Trey

  • Can try and pick up one of the tickets that got created.
  • Look at if manifest generation is broken for regular hyrax resources.
  • Message from Julie 3/16 at 14:27 eastern on #hyrax-valkyrie indicates that image display in viewers are broken


Updates Mar 18

See Retros Dev Congress March 7-18

33 PRs!

Chris

  • Working on feature tests for valkyrie
  • If you enable an adapter other than Wings, hide the Wings constant altogether
    • "tests with the most LDP requests" now has zero tests listed
  • Working through load and authorize before action – collection class is only provided at load time so this makes it hard in the test.
    • Maybe specify a method instead, which can then check the config.
  • At the end of this we should be able to do work by cloning and updating a feature test, then making it pass

Eliot

  • Shepherding some PRs. One outstanding is one that updates hyrax on nurax-pg. How does this get deployed?
    • Lynette manually deploys this
    • Hopefully this will allow us to close more tickets that have been fixed by other work

Shaun

  • Have been working on 4590
    • failures encountered:
      • There's no description in the form for valkyrie collections
      • there's an undefined method banner_info
    • These are both known issues and both are under current development.

Tao

  • Back today

Anna

  • working on proxy deposit.  First PR is green and ready to merge.  Second PR is still in progress.
  • Working on adding another PR that would break out the processing of files into a background job.

Lynette

  • Lots of code review
  • worked on debugging universal viewer on nurax-dev
    • fits wasn't being called correctly > characterization not happening > images don't display in viewer
    • wasn't a code issue

Julie

  • 3.3.1 bugfix release? Do we want to do this on Monday?
    • Let's answer this after Anna's PRs go in
    • UV not an issue
    • still holding off on GeoNames – we can talk to Daniel on Monday
      • this will require 3.4
    • We need a 3.4 anyway because of deprecation with proxy depositor code – needs a release note.
      • Lynette and Julie will talk more
    • If we want both releases we need to do it via backporting

Trey

  • Hi friends!