Valkyrized Hyrax Community Effort - January 2024 Update

Owned by Kevin Kochanski
Last updated: Jan 26, 2024 by Heather Greer Klein
3 min read

Work is reported organized by the three milestones identified at the beginning of the 2023 Valkyrie Sprints:

  • Milestone 1 is to create a fully functioning version of Hyrax that is backed by Postgres. 

  • Milestone 2 is to pave a well-established workflow for live-migrating an existing Fedora 4 backed Hyrax repository to Valkyrie Postgres backed Hyrax repository. 

  • Milestone 3 is to have the Fedora 6 Valkyrie adapter fully working in Hyrax. 

Going into 2024

Milestone 1
Following the 2023 Valkyrie sprints, Milestone 1 is complete. This means the work has been done that allows Hyrax to connect to something other than Fedora 4; in this case, Postgres.

  • We have a migration path from Fedora 4 to Postgres for anything that doesn’t have files, which has already been implemented with GBH.

  • Princeton added file versioning support. Fixity checking is still missing but expected in a 5.x release. The migration of file sets and files needs to be implemented but is part of the sprint the board authorized to take place in early 2024.

  • We still need to make sure that file sets work with Valkyrie and fix the specs up on the new adapters (SoftServ) and get the specs green for koppie. Developers from Emory University have been taking on this work, and thanks to their efforts only two specs remain unresolved. Once that is done there will be a fully working adapter for Fcrepo 4 to Postgres (Freyja) and an untested but likely fairly close to working adapter for Fcrepo 4 to Fcrepo 6 (Frigg).

This work puts the model in place, but next is migrating an existing application to Postgres (Milestone 2), followed by adaptation to connect to Fedora 6. This work is not in the scope of the 2023 Valkyrie Sprints.

Status as of January 2024

Milestone 2
Supported by the Samvera Board, SoftServ has undertaken work to migrate repo.samvera.org to Postgres through Valkyrie. This is Milestone 2, which is an actual migration. While Milestone 1 does the work and proof-of-concept, Milestone 2 puts it into practice. Currently, much of the code for the valkyrization of the demo instance is in place, but the migration to Postgres has not begun.

The suggested path for undertaking a Hyrax Valkyrie migration at this time is:

  1. Upgrade Hyrax, Rails, Blacklight etc.

  2. Create a new set of models that mirror your existing models

    1. If you have ETDs, Books, etc for work types, you want to create a new one for each so that you have an object in ActiveFedora and in Valkyrie

    2. The new models will read existing data from ActiveFedora store

    3. Valkyrie is now the resource version, pointing to Fedora under the hood

    4. Now everytime you save a record, it moves from Fedora 4 to postgres (or Fedora 6)

Milestone 3
Concurrently, the Hyrax-Fedora Working Group is finalizing their initial milestone of planning the work for the Valkyrie Fedora 6 connector and updating group deliverables to move from Planning to Development. As of now, the Sirenia dev app (Valkyrie-only Hyrax with Fedora 6 metadata and storage adapters) is functional and in user testing.

Next work, as of January 2024

Milestone 2
The Postgres migration of repo.samvera.org should move forward very soon, as the code work for Valkyrization wraps. This work is the precursor to migrating a second community instance, PALNI and PALCI’s Hyku for Consortia. Their support and early adoption should set a good precedent for the community and refine this process for followers.

Additionally GBH is working with SoftServ to migrate their Hyrax application to Postgres. At this writing, the migration is almost complete. This involves migration of metadata, not files, so it does not complete Milestone 2, but the work should be acknowledged as a significant contributor to it.

As a benchmark for performance, SoftServ’s experience with GBH was migrating 1.5 Million records on an app that was live and functioning, which took about 30 days in the background with about 2% failure rate.

Milestone 3
We’re waiting for a Fedora 6.5 release, which enables a pair tree identifier configuration option. Work is required to actually do a Fcrepo 4 to Fcrepo 6 migration either with an internal dev team or through consultants. This will likely kick out any remaining bugs and really prove the process to the community.