Notes from the Developer Congress

Owned by Jessica Hilt
Last updated: Jan 27, 2020 by Jeremy Friesen

Day 1

We got Zoom set-up AND with a polycom! 

Opening - Tom

Intros

Project board - https://github.com/orgs/samvera/projects/5

Why people are here: 

Tom - Accomplish moving forward Valkyrie native model questions

Aaron - Looking for how we could use Valkyrie and contribute 

Mike - Looking to be productive

Bess - Starting a new project that may use Valkyrie

Max - Putting some faces with names. How Active Fedora/Valkyrie works 

Thor - Infrastructure. Get a feel for the people and what's being worked on. 

Nikitas - Use Valkyrie and interested to see how other people are using it. 

Julie - Get back up to speed. Work types and models on the metadata side. Roadmap council hat (FEDORA! LOL) how to better resource this. 

Jeremy - Placing a bet where we are moving our repository. Come and say what do we need to do and what does a Valkyrie system look like. Working on the Samvera Permissions group. 

Longshou - Participated in Wings work - see what I can learn and contribute. 

Brandon - Work with Hyrax tech primarily. Keep up to date and figure out the migration path. Hyrax into Kubernettes. 

Randall - A lot of vul-de-sac applications and want to move to a large Hyrax install. Get some traction with wings. Blocker for wings with extracted text that doesn't copy to the repository. 

Rob - 8 Fedora instances - can't be clustered. Scaling issues that we want to solve. New projects on the horizon that are Valkyrie or bust. Dynamic metadata. Permissions work. Optimizing ingest. 

Jessica - Here to help. 

Kate - Wants to know what the community needs are, shore up documentation. 

Chris - Interested In non-local, non-avalon, permissions discussions, rails 6, blacklight 7. Transition from dry-transactions. But open. 

Brian - Randall's team. Half time on the hyrax maintenance working group. Learning whatever I can. 

Lynette - Hyrax/Valkyrie - Files and file sets. Still work to be done in Active Fedora and needs help with that. Pretty much done but needs help. Put a bunch of links in Slack. 

Rachel - Trying to get oriented to the community. 

PR Checklist: https://samvera.github.io/pr-checklist.html

Plenary - Overview 

Active Fedora is an under-maintained abstraction in Hyrax. 

Hyrax to Valkyrie - Last January effort - Penn State - Polar Vortex 

Build an adaptor for Valkyrie that would persist data for Active Fedora

Rather than rippin gup the world and replace the code base. Pick off little bits to use Valkyrie. And then Active Fedora is gone. And then you can switch backends! Valkyrie adaptor is called WINGS. 

There are pieces that are in progress. Metadata object layer, the ability to create a Valkrie resource and have it behave, exists. 

Since Oct: 

How do I get those models in place 

Edges of indexing, protopattern but not a complete implementation

Forms, Hyrax has a prior form pattern in place. 

Can I put Valkyrie native model behind controllers and have them interact. 

Look at outstanding pull requests. 

Putting ideas on the project board. https://github.com/orgs/samvera/projects/5#card-32013076

https://github.com/orgs/samvera/projects/5

Report Back for Day 1

  • Rails 6 preparation - preparing for work in Blacklight
  • Indexers - Reviewing architecture of Indexers. Indexing partial amounts of the metadata (keyword and subject IDX). Next, need to dive into how the rules are expressed in the schema.
  • Documenting Dive into Valkyrie - Current working document https://docs.google.com/document/d/1VeLPU3gIrKt6BkSqGBz3ZFlMPCwWtCuG_ZRj95N5DPY/edit# (got to the validator part of the tutorial)
  • Tidying up some stale hyrax PRs to get those in place
  • Spinning up a Hyrax application with Postgresql to see what’s broken
  • Refactoring jobs - worked through jobs that handled a repository object, created several new issues on the SOLAR Vortex board (@rdfloyd can you provide the grounding link). Working on characterizing jobs, there is an existing blocker. This also blocks on the job authorization.
  • File related issues #4055 works on round-trips on files. Needs to be applied to the other files. Adding file to fileset would sometimes lose the context of the file (due to not rewinding) (“Be kind, rewind!“)
  • Working on making progress on Valkyrie native forms. Pushed this morning’s PR through. Worked on the view, found that routes were breaking, needed to do some customization for ActiveFedora style routes and new Valkyrie style routes. We’ll now need to do the work on form objects.

Report Back for Day 2

  • Rails 6 preparation - There are a few Spotlight test failures, there are some incremental PRs submitted. Work is done on Samvera core components
  • Indexers - Spending time on related issues around generating a new Valkyrie resource; Working on properly setting classes for the object. Back to working on the indexer. Wings had no concept of a Valkyrie resource existing before an ActiveFedora resource existed. This is close to full and proper index of Valkyrie native objects.
  • Documenting Dive into Valkyrie - Created a long list of what was found, triaged what to do, and worked to not clobber each others work. Hope to wrap up tomorrow.
  • Working on upgrading Hyku to Hyrax 3 - So many dots…bundler is stuck on many dependencies; Oh boy!
  • HOLDING - Spinning up a Hyrax application with Postgresql to see what’s broken; Indexing and the Form objects don’t yet work, may be things around permissions
  • Working on jobs - Pushed a branch up for characterization jobs; separating AF and Valkyrie characterization, cribbed from Visibility Copy. Broke off work samvera/hyrax#4207, with full rational. Working through Create Derivatives job. See erayle's writeup on patterns.
  • Minting a new ActiveFedora as part of the plenary
  • Pushing FileSets to generalize the types of associations that are processed for Valkyrie - Progress continues, we can get the custom work done; There are some helper methods for original file and extracted text. Will need to write some tests concerning this work. There is some refactoring to consider. There are some custom queries aligning with Valkyrie language. PR samvera/hyrax#4209
  • Valkyrie Native Forms - Conversations about strategy for handling routes on new objects; There are WIPs in the making
  • Lunchtime discussion/report back around permissions - Tabled for the day
  • Looking at switching DryTransactions to DryMonads - Tabled for the day
  • Back porting fixes for v13.1.2 of ActiveFedora to v11.x and v12.x

Report Back for Day 3

  • Worked on Spotlight compatability for Rails 6. Coordinating with Stanford to get this resolved upstream.
  • Went through Hyrax README and installed without issue.
  • Hyrax #4207 - removing Collection size feature, expecting it to go green soon
  • Hyrax #4213 - addresses characterization job; extracts Job code into 2 services one for ActiveFedora and Valkyrie FileSets
  • Working on derivatives Job, there are current performance issues around casting to LDP resource
  • Released v12.2.1, v12.0.3, and v11.5.5 of ActiveFedora
  • Close to a PR for File actor behavior and it custom file type; There remains a question about how we store files in a File resource. Pattern where you can store one resource within another OR setup to have each resource stored as a separate line in the database. This is only a question for other metadata storage adapters.
  • Working with Indexing, understanding the indexers and how they interact with Valkyrie. Working on tests surrounding loading the indexer for a Valkyrie resource. Dug into why indexers aren't connecting which is due to pub/sub interaction. Need to hook up the proper listeners. All to say, Valkyrized indexing is close.
  • Addressed Valkyrie model routing with a merged PR.
  • Changed configuration of Virus Scanner, now a general setting, instead of a per work setting. We have a backwards compatable virus scanner setting.
  • PR #4181 requires a review (ChildTypes service)
  • Working on tweaks for Wing's ActiveFedora converter; Takes Valkyrie resources to ActiveFedora so Wings can save to Fedora. These are driven by the indexing work.
  • Did an extended "How do indexers and pub/sub interfaces work"; Hopefully we'll move from shared knowledge into documentation.