Notes: Check-in for Hyrax-Valkyrization Community Effort September 6th - 16th, 2022

Owned by Heather Greer Klein
Last updated: Sept 16, 2022 by Juliet Hardesty
7 min read

 

We'll record discussions and updates here in case there's anyone who misses the call

Week 1

2022-09-06

2022-09-07

  • Fritz #5795 Working on delete fileset fix. Problem is actually during the show after the deletion

    • Further discussion shows the id is retained in hasRelatedImage_ssim and hasRelatedMediaFragment_ssim. These should be cleaned up when deleting the fileset

    • The representative presenter’s download_url method should also be made more resilient to missing records.

  • Jose #5492 Subcollection create error. Mystery of how count works in AF. Might be related with nested indexing?

  • Chris #5596 Nested collection indexing not running valk resources. Some tests passing, needs further work.

  • Jon has hyrax up in docker on Mac

  • Rebekah also has hyrax up, will review nurax-pg issues

  • Gabriela looked through nurax-pg tickets, closed ~8 with Julie

2022-09-08

  • Gabriela still working on getting koppie up and running locally (Windows issues, we think)

  • Jose paired with Chris on #5492 subcollection creation

  • Rebekah - koppie up and running locally (Windows is working!); reviewed issues on nurax-pg

  • Annamarie - testing in Firefox for nurax-pg spreadsheet; will review failed tests when nurax-pg is updated

  • Daniel - will update nurax-pg from main; working on #5794; looking at Surfliner for this issue; helping Gabriela with Windows issues

  • Fritz - has ticket fixed and writing tests now (#5795); reviewing examples of factories to get the test set up for deleting a fileset; Daniel recommend posting PR, Fritz planning to post tomorrow morning

  • Julie - reviewing embargo use cases locally from feature test spreadsheet for Tamsin’s work on #5572; need help testing Release and Visibility tab use cases related to embargo; also think lease use cases could be tested; need to work through nurax-pg issue list on Samvera-labs

2022-09-09

  • Gabriela has Koppie working on Windows!! There are details for this that would be good to add to our instructions.

    • When running docker compose -f docker-compose-koppie.yml build the container showed as failing with koppie-app-1 exited with code 1 saying exec /app/samvera/hyrax-entrypoint.sh: no such file or directory. The problem on this Windows 10 Lenovo laptop was due to the difference in EOL characters on Windows and Unix/Linux. Windows uses CRLF as its line endings and Unix/Linux uses LF. Windows Git is set up to auto convert EOL characters to CRLF, so to clone the repo I needed to use the autoclf attribute to disable it:

      git clone https://github.com/samvera/hyrax --config core.autocrlf=false

      After that, building koppie and getting it up and running worked (make sure you have at enough memory, 8 GB).

  • Daniel - file downloads controllers behaviors, trying to integrate them and getting very entangled so trying to do a better separation to make it a specific Valkyrie code path, still in progress

  • Rebekah - embargo testing on VR tab through VR1.12; 2 issues, managers of admin set can’t see embargoed work, adding participant to admin set clears dates in Release tab on admin set

  • Fritz - working with testing process; thought had unlinking of file set on delete working and did a lot of debugging but testing code runs delete and does save but trying to find object again shows that only some of the info is saved and not all info might be updated from the delete and save action; Daniel can help look at this

  • Jose - working with Chris on sub collections; it’s working!! created new adaptor just for Valk resource and now looking at tests

  • Chris - fixed some other things along the way with sub collections work; looking at ripping out nested indexing entirely in favor of Solr graph query parser that seems to do traversal like how we have indexed before; will likely have a PR to review soon; limitation is only available for standalone Solr or Solr Cloud running with single shard (so multi-shard would not be able to use this); checking with tamsin johnson and Rob Kaufman; doesn’t seem to be too recent of a change, it is in Solr 6 docs

  • Annamarie - still having problem with Image issue in Firefox; will have more time on Monday; this Image work might be an old work and will try again creating a new work

Week 2

2022-09-12

  • Fritz & Daniel - #5795 - Working on making representative presenter creation more resilient and also fixing spec for removing fileset

  • Daniel - Continuing on #5794

  • Gabriela - Checking for some partial failures in issues

  • Annamarie - Identifying some UI issues (likely resulting from Blacklight 7 update?)

  • Jose - Working on #5594, Seeing an error related to deprecation warnings?

  • Chris - PR is up for using solr for nested collection membership. PRs #5851 #5852. Now working on ownership issue #5456

  • Julie - Whittling down nurax-pg repo issue list. Identified a few UI issues

2022-09-13

  • Daniel and Fritz working on PR for issue #5795 with expanded specs and cleanup after feedback from tamsin

  • Daniel working on downloading of file #5794

  • Gabriela - went through Fail/Partial Fail tests (on nurax-pg); looking at Ready for Review issues, not sure if they are ready to QA, will wait for those to move to QA column

  • Chris - nested re-indexing getting ready for 4.0, tamsin started backport of that for 3.x but will need to be updated, partial reviews have happened but no full review yet, Daniel says good to get this into 4.0 since it is blocking things from moving forward; PR for transferring ownership of works and needs re-review; working on batch edit

  • Fritz - PR for fileset delete issue; picked up new issue (embargo deactivate), know the problem but don’t know how to solve it yet, leftover AF issue

  • Annamarie - checking in this afternoon, nurax-pg updated and going through QA list - need to check issue board; QA list is meaning spreadsheet; looking at logo file on nurax-dev and might have 4.0 issue if multiple logo files in place, can’t remove

  • Jose - need to talk to Chris about Rails 6.1 monkeypatch change, might be causing issues getting to PR on current work, #5594

2022-09-14

  • Daniel - fixing up process to attach files to filesets and create derivatives; nurax-pg was used to generate koppie and file storage is not as expected so that needs some fixing up; working on the Valkyrie ingest job, not calling up FITS successfully right now because FITS is not running; Chris - Avalon uses mediainfo and ffprobe, FITS wraps ffprobe

  • Fritz - working on update embargo (deactivate embargo); have it working and tests submitted, waiting for that to run

  • Gabriela - QA test on transfer ownership, works on koppie

  • Chris - batch edit is close, tests are passing for main case but delete batch of collections case not passing yet; permission attributes posted to controller weren’t getting handled at all, couldn’t add users with new permissions; tamsin helped figure all of this out; making PR for just that fix (in transactions)

  • Jose - PR submitted this morning but failing tests; having hard time getting rspecs to run, says Gemfile.lock has to be removed, did this on .dassie; Daniel - can’t run rspec on koppie yet so .dassie is the place, have to go to hyrax-engine directory and run rspec from there; if Gemfile.lock is a problem then it is a file that is located in that directory (not .dassie); bundle install should help to update so rspec (maybe after removing Gemfile.lock from hyrax-engine - directory one level up from .dassie)

  • PR from Fritz likely to be approved and merged soon for delete action on file list!

2022-09-15

  • Fritz - 2 PRs verified and ready for merging! They are approved so Fritz will merge; working on 5575, work view page - items action menu - can’t edit; new valkyrie world has different structure and not clear on how to fix this up, need to talk with Daniel

  • Jose - PR ready for review; no tests, tamsin didn’t think they were necessary; working on 5592, statistics link on dashboard; Chris did some work on work-relation and there is PR that Jose is reviewing to narrow in on the errors occurring for statistics

  • Daniel - local koppie can do valkyrie ingest job, which means UV came up!; still not showing anything though (UV is blank); working off of PR from May, changing how derivatives are generated or persisted, actual derivative generation might not be done, Jeremy F is also looking at this now; working in branch to capture some of this older work and add in all the new stuff with koppie; changing up where files are stored with koppie, PR for this ready to review; don’t think this requires rebuild to try, just restart; not possible to have tests for this since it is about file storage; still dealing with weird error that was blocking May PR as well; works - item download not working was original issue and will try to get back to that

  • Julie - couple new issues to QA

  • Annamarie - can QA issue

  • Chris had a couple of PRs at the end of the day yesterday, Daniel might look at them; might be another graphDB-related PR to review for backporting

2022-09-16

  • Fritz - revisited embargo deactivate issue; putting PR in Ready to Review soon; will add notes to other issue getting started and put back in Ready (no commits)

  • Daniel - took old PR for derivative storage, fixed it some more and then UV viewer tries to load things! still not displaying anything which might be access problems from other issue with filesets and looking at Chris' permissions PR to see if that fixes it; there is a branch that downloads file but fileset page not working; might ask tamsin to take a look at large PR; might try to make separate PR for file downloads

  • Gabriela - reviewed QA issue for Rights Statement; will test on nurax-pg if there is a deploy

  • Jose - couldn’t figure out issue that he discussed with tamsin, need to talk to Chris, about statistics from Dashboard not working (5592); picked up issue about Rights statement not showing up and fixed that; picked up other issue and working on debugging now, collection metadata showing

  • Feedback on Community Effort

    • pairing sessions were really useful to look into places in code that haven’t seen a lot of attention

    • koppie in Docker was really well set up

    • folks were welcoming for questions

    • good first community effort experience

    • wanted to be more engaged with testing

    • maybe use Github action to enable more people to deploy to nurax-pg; make deploy to nurax-pg part of having check-in meetings

    • Chris' PR for graph is still outstanding and will resolve a lot of open issues with nested collections

    • learned about running koppie and dassie internal test apps at same time

    • Windows CRLF, learned about that for running docker

    • learned more about valkyrie resources

    • need more devs that have valkyrie xp

    • video or talk to show valkyrie resources, how they work - code work and PRs, doesn’t have to do code work but explain things

      • Trey has talks about Valkyrie the gem; Hyrax uses valkyrie somewhat differently

      • There’s documentation out there from Lynette but it might be out of date now; Hyrax wiki on Github has info (Valkyrie Usage Guide); also Valkyrie Google Docs

      • want to know things like where config files are located, documentation might be too scattered right now

      • Maybe work towards presentation/video to explain Valkyrie for SVC

      • also something that explains how to create a new connector