2017-11-20 Avalon Hyrax Pals

Owned by rsteans
May 09, 2018

Attendees:

  • Steven Van Tuyl (Oregon State)

  • Mike Giarlo (Stanford)

  • Maria Whitaker (Indiana)

  • David Schober (Northwestern)

  • Jon Cameron (Indiana)

  • Chris Colvard (Indiana)

  • Ryan Steans (Northwestern/Avalon)


Summary:  

At Samvera Connect 2017, Steve, Mike, Chris and Ryan met to discuss community issues within Hyrax and Avalon.  We agreed to touch base following Samvera Connect. In the week following, The Avalon management team expressed deep interest in  touching base with Mike and Steve to discuss the current state of Hyrax for the purpose of roadmap and scheduling Avalon 7, which will be built as a feature clone of Avalon 6 (with more bells and whistles, one assumes) on Hyrax.

Agenda:

  • Quick introductions/ reminders of who’s who (Ryan)

  • Discussion of Avalon 7 (David and Jon)

    • Purpose

    • Projected start date

    • Ties to IMLS grant

  • Discussion of Hyrax

    • Purpose

    • Current development track

    • Development issues, needs, etc...

    • Ties to larger Samvera community

  • Co-scheduling for development



Action Items: (all for Ryan)

  1. Set up a monthly Avalon/ Hyrax Pals Meetings

  2. Set up a bi-weekly Hyrax meeting

  3. Internal Avalon “what does this mean?” meeting


Major Issue Uncovered:

How to move forward with Hyrax 2 or 3 as we look at Avalon 7 with a similar delivery date for final product release.  Does Avalon develop against 3 and slow down or develop against 2 with a known immediate migration problem?

Notes:

Maria and Jon:  After 6.3, move on to 7.   Looking to make Avalon a component into Hyrax.  Look to make Avalon a part of Hyrax.

David:  we started the strategy about 6 months ago.  Did a proof-of concept with DCE. Last year, Avalon team participating in Hyrax sprints.

Jon and Chris:  scope out timeline part to rebase on top of Hyrax.  Figure out take on timelines of Hyrax and other communities around Hyrax.

Maria:  Also the fact that one of the decisions we have to make is whether we base work on Hyrax 2 or somehow base it on hyrax 3, which is under development.  Important because of work with WGBH that will add features to Avalon that are needed, so a combo of work that WGBH will be doing that. As we were trying to figure out how to combine them all.

David:  can you give us an idea of the direction so we know when people should be developing on Valkyrie?  

SVT:  Hyrax driven Valkyrie - run a second round of work on Valkyrizing Hyrax in Mid-December/ Janauary.  Hope is that will get us 2/3rds of the way to a functional Hyrax on Valkyrie. Look for rest of work to happen in 2018 sometime.

MG:  We’ve got 150ish tests to get passing.  That will be some level of integration done, and identified 4-5 other buckets of work with features working.  Focus of December/ January sprints. Sprints to fire up UI and see what test suite didn’t catch. So, possible by End of 2nd phase, you could justify working against Valkyrie 2 branch, but don’t want to give too much hope. Not sure who is allocated on what percentages.  Want same people but we may not get them. New people will slow veolcity.

MG:  Do we base on Hyrax 2 and structure work so we don’t feel performance pain or ride the wave as early adopters on Hyrax 3 and use that as a reason to invest in that work further.  OSU went to production recently. Steve?

SVT:  We launched on Monday an IR on Hyrax 2 and we’ve been developing against master branch for 7-8 months.  Wasn’t without problems, but for the most part by paying attention to what was going on, that experience wasn’t as jarring as I thought it would be.

Some of the ways we’re talking about 2.x to 3.0 is going to make it even less problematic.  We have a sense for the roadmap and learned a lot over the year and know how to add features without blowing up the product.  

No sense for - at a high level- what the Avalon timeline is?

David:  would like to have components done in 6 months, but if we decided to work against Valkyrizied Hyrax, we could push out.  A year out, makes some sense.

Maria:  Some features going into both 2 and 3?  

MG:  That’s part of the overhead.  Collection Extensions getting pulled in to 2.x will go into 3.0 when it launches.  

D:  How long will that go on?

SVT:  Try to overcome churn of last year or so.  During that time period, try to get things on roadmap to incorporate in - but about a year.

M:  Suppose we decide to build on Hyrax 2?  How big of a migration would that be?

MG:  in terms of code migration, very different.  Any customizations or overrides - will need to be redone.  Migration of data - more of a question. Can make it so people don’t have to do a change of data.  If the community values not doing a data migration from 2-3, that’s an understandable goal and migration.  

D:  Code changes substantial, Avalon will likely be impacted.

Chris:  depends on what we do.  If our change is minimal as plug-ins, so work will need to happen, but don’t perceive it to be too huge.  Maybe a month, but not 3-4 months. In terms of data, a key thing is that Avalon 7 as a component - we don’t care about the data.  For Avalon the product, we’re coming from a Fedora 4 thing that’s not Hyrax or PCDM. We’ll need an adapter, anyways.

D:  Hopefully we don’t do a double jump.

Chris:  For institutions with Avalon in production switching to 7 before we switch to 3.

D:  Our timelines of a year synch up, but we launch on Hyrax 2 while Hyrax 3 launches at same time as Avalon 7.  

Chris:  make jump when we feel good

Maria:  Question for Mike and Steve - Jon Dunn attended partners meeting saying that it is not clear that community will go with Valkyrie.  It is a goal of many, but not entirely clear.

SVT:  Talking about Valkyrie on Hyrax, Valkyrie will be part of Hyrax so you can swap out the back-end.  Will be in 3.0. Question of whether they will swap out back end or swap out for Fedora 4 and use that.  

D:  Hyrax is going to be based on Valkyrie

MG:  yes. Everyone running Hyrax will et to choose how they persist their own metadata and bitstreams.  No opinions on what does the Valkyrie mean for Fedora 4, no opinion. Folks working on Hyrax will continue to work on gems that are for Hyrax.  Subsidizing most work on Samvera work. If community seems to be moving to solution bundles, only some will develop on Valkyrie. People developing on Fedora 4 will be developing and maintaining all on their own.  

D:  Think about work as 2 phase - work that doesn’t touch underpinnings and phase 2 where we can review what’s in 3.0.

Maria:  we can think of that as a good approach, but see how that works with WGBH.  Don’t know if it will be a lot of work, may be deep into Hyrax code.

Chris:  data migration is a much uglier issue than worktype changes, etc…Workflows won’t change between Hyrax 2 and 3

SVT:  what is WGBH looking for?

M:  moving collections to avalon for mgmt, but have needs Avalon doesn’t meet.  Developing what they need. Want PB core as their metadata model. So, one of the big things.  

Chris:  WGBH will be working on Avalon 7, contracting AV preserve to do preservation work.  Early December, figuring out that work.

D:  Really basing it on Hyrax because Avalon 7 doesn’t exist and won’t for a year.

M:  will have to build things Avalon 7 needs.

C:  Question - how quickly do we think we’ll adopt Hyrax 3 vs Hyrax 2.  Building components for future.

SVT:  asked who was planning a major migration at Partner’s Meeting.  5-6 large institutions. Sense a Valkyrized version of Hyrax what they’re looking for to try to swap out back-end to deal with performance issues.  

Chris:  to meet WGBH’s needs, write plug-ins to Hyrax 2 and plan for Hyrax 3.  

D:  Internal convo to deal with 3 lines of tech development

MG:  No one making active jump to 3 work until the spring.  If past experience is right, see a fair number of early adopters at beta, don’t be surprised if 3-5 do that.  But heard a lot of people want more stable releases with longterm support. Those not experiencing performance issues will probably hang on.

D:  Do we think WGBH content will fall into large scale content is going to make them one of those problem children?

M:  not just media, but metadata is also an issue.  Want metadata included. Problem is not size of files, its size of collections.  

MG:  Will sideload content in for large collections.  Will redirect to the content.

MG:  Sounds like an Avalon internal team meeting, but a hopeful note - if any team is going to pull this off, team at NU and IU can handle the challenges.  We’ve got investments in Hyrax and Valkyrie.

Jon:  Plug-in architecture - in a very basic sense how does that look, how will that evolve?

MG:  My sense is that in the world we’re in with Rails, the concept of a plug-in is fuzzy, ill-defined.  A plug-ins groups spun up, maybe put out a white paper, but no follow up. May be some useful stuff to pin down a plug in.

Maria:  we talked about taking the work of that team and picking it up again.  Overall impression that it’s not complete.

Chris:  basically wrote guidelines for if you’re writing a plug-in.  But not in a way for how to do it exactly, closest thing to architecture, not really hooks in Hyrax for plug-ins, would be nice for that to exist.  Lot of discussion about what that would be. Will be overrides and patches to make things work.

MG:  sounds right.

SVT:  a couple other efforts coming along this year thinking in a plug-in way.  Notre Dame’s work toward groups, roles, permissions. Making workflows more of a plug-in thing.  Driving at: while that WG has completed its work, there are a bunch of people looking to implement a plugin framework on Hyrax, might want to get those folks to chat a bit.  

Chris:  what I’m hearing about Donut? And API’s, potential future direction

David:  just digging into architecture, will need an ingestion API for bulk updates and uploads.  Current model for JSON API on SOLR data return. How much we massage that a big question. If we can get to that data, we can build a reactive UI either on Hyrax or other technology.

Presentation tied into SOLR somehow, may build into Hyrax with Blacklight

Don’t know how applicable it will be for Hyrax core initially.  

MG:  So, this notion of separating UI from API has been on wishlist a long time ago.  In terms of getting that work done, Hyrax 3 is so different from 2, I don’t know how much appetite community will also want to redo API work.  One of these things where you take temp of community to see what they want.

Use Rails to build Hyrax and tooling.  Routes thru Rails, you get REST API’s for free.  No one has poked at current UI with a JSON client to see how well it works.  Poke to see how far current code gets us.


D:  Maybe Michael Klein.  

MG:  Having that analysis in hand will give us idea of that Delta.

C:  get an idea how it is to port from 2-3, 3-4.  

D:  Look to see what we want to stabilize on.