2023-10-24 Hyrax Fedora 6 Working Group


Oct 24, 2023(Hybrid meeting at Samvera Connect in Philadelphia)


  • @Jon Dunn

  • @Randall Floyd

  • @Daniel Pierce

  • @Juliet Hardesty

  • @Collin Brittle - notetaker

  • Apologies



 Discussion topics

















Background: use cases doc: https://docs.google.com/document/d/15CPJxVIhriCcnzn2boomfWAJvZ3mwN7KDiPHLvvlS2w/edit?usp=sharing

Daniel presented update on Hyrax-Valkyrie-Fedora 6 work as part of this morning’s Hyrax update.

Need to consider a new chair/convener for the WG

  • Contact Arran Griffith or Dan Field if interested

Two options for metadata storage

Binary storage, first proposed by Oxford, versus text storage

Binary storage is similar to what Valkyrie does

Tamsin mentioned working on producing a graph from Valkyrie, but that needs predicates, text.

Binary JSON would be saved on disk as a file, and returned as a file.

Not as well preserved if a binary blob. Better preservation as triples, as it is human readable.

A JSON (text) file would be better

Postgres adapter is adaptable to use other SQL storage, but is designed to work with Postgres first.

Plan is to implement JSON as a secondary format once the initial work is done.

Need to loop in Metadata IG?

Potential for storing some metadata as binary if there is less interaction with it.

Sirenia exists as a proof of concept, is ready to play with. https://github.com/samvera/hyrax/blob/main/docker-compose-sirenia.yml

May become part of Dev Congress

Would be nice to start doing performance testing.

A nurax-style environment would be helpful - in the works

Issues w/ Hyrax + Fedora 4 start around 4K AF objects in a collection, or 100 filesets in an object (which is around 40K+ objects in Fedora). Can force the issue by limiting the amount of memory that Fedora has.

There is a test script that can be shared.

Can use Bulkrax to script up performance testing

Bulkrax is mostly ready for use with Valkyrie

The representation of fiesets as being part of the work node exacerbates the performance issue

Group should revisit the charter and discuss if changes are necessary given the progress in implementation.

Is migration in-scope-able? SoftServ has a metadata migration adapter (No files, yet).

Largest questions are around representation of binaries in Fedora

Another issue is around pairtree id problem.

Another future topic is Archival Groups

How or if they are created may become a configuration option in the adapter.

Performance differences between atomic and archival versioning? On Dan’s TODO list.

Now working against bare metal, will be working with TACC.

Fedora UI, analytics, user admin are also going to get an overhaul.

How is provenance tracked? It’s not, but that use case has come up for Fedora before.

Hyrax interacts with Fedora as one, anonymous user

Fedora provides a log of actions that could be used as an audit log.

To track provenance effectively, Hyrax would need to change.



 Action items