Notes: Check-in for Hyrax-Valkyrization Community Effort March 7-18, 2022
Home | Developer Resources | Check-in Notes |
We'll record discussions and updates here in case there's anyone who misses the call
Week 1
Updates Mar 7
Issue #5461 (Anna) - batch edit not working with valkyrie
- just that batch edit doesn't work
- got nurax-pg up and running
Issue #5462 (Eliot) - editing parent work error
- got nurax-pg up and running
- got error following steps, but it is different
- working on figuring that one out
Issue #5480 (Trey) - file upload multiple problems
- issue was filesets don't have a thumbnail
- there are multiple errors
- FIXED: valkyrie ingest job was failing because test and app worked a little different. Can't copy carrier wave stream directly into a file.
- FIXED: characterization wasn't working because not able to set file set id
- FIXED: missing title
- derivatives don't run
- rubocop doesn't like long method
Issue #5449 (Lynette) - nested collection form exceptions
- working on nested collection form issues
- FIXED: method missing for find
- working on #nestable? method missing
Issue #6490 (Shaun) - UI consistency
- Created a PR for a changed label.
- Would recommend taking out icons from action menu
Daniel
- planning to work with dassie with valkyrie setup
- will look at setting up nurax-pg
Chris
- looking to pick up something tomorrow
Updates Mar 8
Anna (#5461) - batch editing
- pairing with Eliot
- batch edit will need to be converted to a change set
- perhaps it would be better to tear it out
- There's another project for batch editing - bulkrax. Maybe putting bulkrax into hyrax is a potential?
- Bulkrax may be more for bulk ingest.
- Julie may know if there's something on the roadmap to do something for batch edit
- Batch edit can be turned off. Maybe we can just make it so if Valkyrie's turned on, batch edit is left flipped off? Toggle based off Hyrax.config.disable_wings.
Chris (#5463) - missing _pcdm_partials
- got docker setup working
- taking a look at searches that return pcdm collections
- looks like it has something to do with routes
- also may need to override a blacklight helper for path resolution
- look at it more tomorrow
- would it work to add partials instead of overriding by rendering the _collection partial from _pcdm_collection partial. Probably would see similar issues.
Daniel (#4788) - fileset id not indexed
- thumbnail ids are being set on the model, but not indexed
- first step is to index the fileset id
- next step is to get access to the files
- worked on getting nurax-pg docker environment be easier
Eliot (5462) - error editing with parent work
- working on parent work editing
- somewhat blocked by nurax-pg setup
- can't replicate in Hyrax
- registering postgres adapter
Lynette (5449) - collection nesting
- lots of moving parts
- exploring current process
- looking to see if can address the bigger issue of not running the collection indexing unless membership has changed
Shaun (5490) - UI consistency
- working on ui consistency tickets
- 2 PRs passing - removal of icons, renaming
- gonna pick up table sort and column sorting next
- going to have to sort by solr on server side
Trey (5504) - file derivatives
- looked some at derivatives issue
- need parallel of current derivatives job that is valkyrie specific
- doesn't store derivatives in Fedora currently
- probably look at tomorrow
Updates Mar 9
Anna (#5461) - batch editing
- Found a bug that's the cause of the ticket about correctly pulling the permissions that a work is allowed to have - being able to set an embargo/lease. See https://github.com/samvera/hyrax/pull/5514
- Looking at #5457 - works deposited via proxy getting the right visibility.
Chris (#5463) - missing _pcdm_partials when searching with collections
- Merged PR resolving the issue of _pcdm collections not being in search results. Ended up cloning a feature test and did some work to make it run against the Valkyrie solr core. Useful patterns here: https://github.com/samvera/hyrax/pull/5515/files
- Won't be back on the sprint until Friday.
Daniel (#4788) - fileset id not indexed
- Looking at thumbnail/representative ID indexing. Tricky because derivatives aren't being made. Mostly making sure how the current system works - it's a little convoluted in places.
- Spent some time trying to solve weird AccessControls::Embargo reloading thing. It seems like ActiveFedora::Relation keeps a cached reference to a class which becomes a different class on reload. Tried to hack it to reload the class, but it got unhappy about wings models.
Eliot (5462) - error editing with parent work
- Just started looking at #5462 again. Trying to find where in nurax-pg the problem lies - tried to spin up a fresh rails app the same way nurax-pg was and getting the same error. Still tracking down.
Lynette (5157) - collection metadata should be same as in AF
- (5449) - collection nesting - not worked on today to finish up 5157
- Update batch delete to use transactions - originally used ActiveFedora to delete.
Shaun (5490) - UI consistency
- Working on inventorying UI for consistent language/works
- 3 PRs in. Ping slack or is the github ping better?
- Slack ping is better.
- Might need recommendations on a good issue to pick up next.
Trey (5504) - file derivatives
- Comments in the ticket.
Tao
- Working on testing.
- Found many issues with collections. Collection creation/discovery of collections. Those issues are blocking his testing.
- Lynette is working on nested collections - hopefully there will be work on that soon.
- Lynette will deploy a new version of nurax-pg with the fixed collection pages.
- Will Lynette announce new PRs?
- Lynette pushes with a diff you can browse.
Can someone merge their institution's PRs?
- Yes
Has Julie approved all the icon
Shaun needs a new issue
- Can work on feature tests - can pair with Chris today or Friday.
- If wants help with the solr backend stuff, can reach out to Lynette for pairing.
Updates Mar 10
Anna (#5457) - proxy deposit visibility always private
- off this afternoon
- continuing to work on Issue #5457
- looked at transactions related to this
- continuing with this tomorrow
Chris
- off today
- Issue #5463 closed by PR #5515 - missing _pcdm_partials when searching with collections
Daniel (#4788) - fileset id not indexed
- thumbnails are now indexing
- question about how thumbnails are indexed in hyrax as hasRelated_image_ssim; not intuitive, but sticking with what they have always been
- in dassie, trying to show the image in the UV, but seems like the manifest isn't there
- one other thing want to do is the changes in nurax-pg to make docker easier to work with
Eliot (5462) - error editing with parent work OR #5461
Lynette (5157) - collection metadata should be same as in AF
- PR #5157 - Collection basic metadata should be opt-in by the application
- PR #5511 - make change set forms allow multiple values for title
- PR #5518 - adds deprecation notices for transactions/steps that are no longer supported
- Trying to get back to collection nesting
Shaun (5490) - UI consistency
- PR #5508 - changing labels caused a bunch of feature tests to fail requiring tests to be tweaked; switching to ids to avoid in the future
Tao - testing with spreadsheet
- #5517 Search result page of collections later pages create error
Trey (5504) - file derivatives
- local meeting this afternoon
- continuing to make progress on derivatives
Updates Mar 11
Chris
- Starting Invalid authenticity token issue
Anna
- Working on when you try to make a work as a proxy depositor you can't set it to public visibility.
- It also doesn't go through as a proxy depositor.
- Figured out where it's going wrong - working on a fix.
- Problem was a race condition on the listener
Trey
- Built out structure for derivatives job
Eliot
- continuing work on parent editing issue
- Blocked on a test that fails in CI (which uses engine cart)
- May be helpful to reference the PR from earlier this week that added a feature test for searching.
- Lynette can pair.
Lynette
- Trying to separate out some work that allows for empty values and doesn't actually save them in the data.
- Working on test failures
- Then plan to go back to the nesting
Daniel
- Thumbnail work - put up a branch and working on some failing tests
- nurax-pg - PR with docker compose fixes ready for review
- Uses hyrax main branch (doesn't try to mount a local development hyrax)
Tao
- Used a new admin user to do testing on collection type
- Still get an error when editing a collection type
Shaun
- Working on PR#5508 - seems ready to go at this point, pending CI
- Had to guard against nil date updated values. Use date uploaded as a backup value. May want to refactor at some point.
- Maybe add a solr document method like
document.latest_date
?
Agenda:
- Offers to pair.
- Idea:
rspec --only-failures
: can we put the result in the artifacts of circleCI? And then recombine the 10 files to let us download it and use it locally.- https://relishapp.com/rspec/rspec-core/docs/command-line/only-failures
- Check whether the file exists on CI
- This could really help in a scenario where there are 200 failures
- Make a discussion or design ticket?
- Output of valkyrie index: sometimes the key is a string and sometimes a symbol. Can / should we use one or the other?
- In the solr document they're all strings.
- It would be easier to test if it were consistent. The base indexer uses strings.
- Make a refactor ticket. We'll need to update a bunch of tests to do this.
- In blacklight
id
is a symbol and everything else is a string. - Can/should we use
with_indifferent_access
?
- Listeners
- I don't think the listeners are used in active fedora. They often have code that kicks out of the method if there's an AF resource.
- (In this case it doesn't)
- Post in slack if you have a PR that needs to be review or re-reviewed!
Week 2
Updates Mar 14
Chris
- I'm here this week part-time
- Looking for a new ticket
Eliot
- Continuing work from last week on parent editing issue
- bug appears to be in PcdmMemberPresenterFactory - it's not fully tested and there's a subtlety with solr
- Have a fix, refactoring it
Shaun
- PR #5527 about date column consistency ready for review
- The date modified column will be blank until the resource is updated
- There's a draft PR to fall back on date uploaded (#5526)
- One proposal is to create a solr field for the latest date
- Another option is to back the logic up into the presenter
- Lynette will look and recommend.
- Planning to pick up one of the issues to add valkyrie feature tests
- Offer to pair
Anna
- Working on 5457 still.
- Noticing that we pass resources directly rather than IDs. It would be good to inventory and move to passing IDs and loading objects, especially in listeners and jobs.
Lynette
- Shepherding PRs
- About to look at issues again to see what our biggest blockers are
- We're doing good work!
- Need review of #5472 (seeds)
- Will go back to nested collections
Updates Mar 15
Eliot
- Responding to feedback on #5516.
- Discovered a new issue - can't remove children through the form. Writing a ticket and then will probably take it.
Chris
- Planning to review branding work.
- Will try to get to issue where members don't show up in search results on a collection tomorrow or later today.
Shaun
- Question about reverting text strings to IDs
- Provided an example regarding philosophy.
- Two PRs where Justin did that. Anna just accepted it.
- There's one more open PR - a few were changed, PR could still be reviewed.
- Do tests need to be changed back...?
- Will do some follow up here.
- No blocking PRs for this group.
- That open PR could be commented on.
- Draft PR in for testing valkyrie collection creation. PR 5541.
- Testing creation of collections with traditional collections and repeating all the steps for the valkyrie collections.
- That right?
- Lynette will take a look.
- Lynette was going to look at search builders for the tables - Lynette might take a look at it to see if she can create something to pass sort to in order to move that issue forward.
Anna
- Finishing touches on 5457. Adding specs/double checking logic. Should be ready soon.
- Lynette will look at PR descriptions
Lynette
- Collection branding is done! Tests are running now - works in UI.
- Spent some time going through test worksheet and marking cleared things for retesting.
Trey
- Manifests don't generate.
- PR for derivative creation is up.
- Getting UV working would be good.
- Will also make a ticket for extracting text.
Julie
- Checking in to see if there are things for her to look at from this morning.
- Will try a few things that aren't part of main yet, but could be merged in.
- (Lynette) One for sure is Geonames where it gets whole state/country - make sure Julie's okay with that.
- Changes to UI stuff, but that's all merged.
- (Anna) Maybe we should release this rollback of the somethingsomethingActor.
- Think it's the cause of a problem with Duke.
- Julie's out Thursday/Friday this week. Is a release something we can manage by end of tomorrow, or maybe release 3.3.1 next week?
- Lynette will look at that PR and see if it can just get in and get a release out tomorrow.
- Does Julie have to be around for the release?
- What about ActiveFedora reversion?
- Happened
- 3.3.1 release should happen after these 3 PRs are merged/done. If we want to do it tomorrow, Julie is around except for an hour in the afternoon.
- Emory may join for maintenance next week. Any valkyrie issues expected to remain?
- Yeah probably.
- Lynette will be minimalist for the next couple of weeks.
Updates Mar 16
PRs merged: 28
Eliot
- Valkyrie Parent Resources to add/remove children. Adding a new transaction to make that happen. Just about done.
Shaun
- Working on issue Valkyrie Collection Creation Testing #4590. Collection feature test is having some problems. Not sure where UI is. 3 failing tests. Can't find title and description field for one scenario.
- Pulled out repeated test login into separate tests.
Anna
- About to look at the status of my open PR #5537. Concerns about backwards compatibility. We are changing it from invoking a background job that calls a service, to directly calling the service first and then invokes a background job. Lynnette has no problem with the transaction actor. Changing the way the job works is maybe a little harder. Lynette suggests perhaps deprecating the listener, but needs to think about it. Anna says that we will need to continue to support that listener which could be a nightmare. It's possible that there hasn't been a release since that listener, but Anna contends that it was introduced at least a year ago. If the listener is a concern we may have to revert that PR. If the Job is a concern, we could deprecate the Job and the Service and we can introduce new ones. Lynette wants to look at it again.
- Question about use of FindBy vs FindByAlternateId...
- Issue #4085
- Lynnette says the way Wings is written, it uses FindByAlternateId when FindBy is called.
- Lynnette wonders if there will be a problem when data gets migrated as the NOID used may not be its actual id.
Thanya (not present)
- Looked at the Default Search ticket. Will write up some notes on what she learned and will un-assign herself from the ticket.
Lynette
- Wrapping up issue #5449
- Prioritize PR reviews and will take a look at the Default Search ticket.
Chris
- Working on issue that if a collection has works within it, they are not showing up on the count on the dashboard, and not showing up on the collection show.
- Working on a feature test for it.
Updates Mar 17
Eliot
- Couple PRs in review column. Lynette reviewed PR about adding stuff to a PR, made a comment. Split off into separate ticket? Open to whatever.
- Fine to do a separate ticket for that. We can merge what's in there now.
- Another PR for review which enables search for child functionality in a Valkyrie edit page.
- Pushing up a PR to fix 5452 to fix citation button.
Anna
- Chat about deprecation concerns for work doing for 5457. Spent time working on deprecation stuff. Really close on all of the deprecations - for this PR just created need to fix a couple of tests. Want to address comments from previous PR.
- Will make a couple tickets.
- Started looking at the seed PR and then looked at deprecation stuff instead. Will try and get back to it.
Shaun
- No updates, doing local work.
Lynette
- Put in tests for nested collection. Just tested with main and something's not quite right. Getting an error again. Will take another look at it again.
Trey
- Can try and pick up one of the tickets that got created.
- Look at if manifest generation is broken for regular hyrax resources.
- Message from Julie 3/16 at 14:27 eastern on #hyrax-valkyrie indicates that image display in viewers are broken
Updates Mar 18
See Retros Dev Congress March 7-18
33 PRs!
Chris
- Working on feature tests for valkyrie
- If you enable an adapter other than Wings, hide the Wings constant altogether
- "tests with the most LDP requests" now has zero tests listed
- Working through load and authorize before action – collection class is only provided at load time so this makes it hard in the test.
- Maybe specify a method instead, which can then check the config.
- At the end of this we should be able to do work by cloning and updating a feature test, then making it pass
Eliot
- Shepherding some PRs. One outstanding is one that updates hyrax on nurax-pg. How does this get deployed?
- Lynette manually deploys this
- Hopefully this will allow us to close more tickets that have been fixed by other work
Shaun
- Have been working on 4590
- failures encountered:
- There's no description in the form for valkyrie collections
- there's an undefined method banner_info
- These are both known issues and both are under current development.
- failures encountered:
Tao
- Back today
Anna
- working on proxy deposit. First PR is green and ready to merge. Second PR is still in progress.
- Working on adding another PR that would break out the processing of files into a background job.
Lynette
- Lots of code review
- worked on debugging universal viewer on nurax-dev
- fits wasn't being called correctly > characterization not happening > images don't display in viewer
- wasn't a code issue
Julie
- 3.3.1 bugfix release? Do we want to do this on Monday?
- Let's answer this after Anna's PRs go in
- UV not an issue
- still holding off on GeoNames – we can talk to Daniel on Monday
- this will require 3.4
- We need a 3.4 anyway because of deprecation with proxy depositor code – needs a release note.
- Lynette and Julie will talk more
- If we want both releases we need to do it via backporting
Trey
- Hi friends!