Core Gems Committers Notes

Owned by Bill Branan
Jun 16, 2017
8 min read

Notetaker: Rob Sanderson

Download Markdown file

Agenda

  • Committers call become something different // Participation

  • Update HoP

  • Relationship to Fedora4

  • Release Management Process

  • Longer term vision

  • Communication

  • Developer face to face time

  • Managing divergence/convergence of solutions

Participation (30 minutes)

Hydra Commiters Call

Justin feels somewhat left out by himself as the committers call has low participation. It's "a Lonely place" with just a few issues along the lines of requesting checks of pull requests.

Suggestion to make it broader -- a participant call. Need a time that works for more people.

Justin: Proposal to change the name to Hydra call. Committers sounds exclusionary. Not a committer shouldn't be on the call, and not the case.
Mark: Hydra-Tech call?
Justin: Not-the-hydra-managers call?
Jeremy: Take it to the call to discuss for people who is not here?
Karen: We have quorum here and can decide.

RESOLUTION: Hydra tech call

Justin: The topics of the call could be broader, such as having an RDF section for issues.
Esme: Or any other WG or IG.
Tom: We need to time box sections.

(Agreement)

Jeremy: 11:30 eastern the best time? (lots of ungood)
Giarlo: Conflicts with other calls
All: General dislike for Monday.
Justin: Friday?
Adam: Wednesday?
Mark: What about the Stanford line? Technology shouldn't be a blocker.
Suggestion to use Google Hangouts, but doesn't support more than 10 easily, with some trick to get up to 15.

PROPOSAL: 11:30 Eastern, Wednesdays

Mark: Enough Hydra developers in Europe -- they should find their own time, and some US people be on it. Get cross pollination through thosee individuals. One call doesn't have to work for everyone.
Esme: Committers is not what we want. Let's not use the term any more. Hydra-Tech.

ACTION: Atz/Az to find out if call time is okay for the line

Adam: Do we need reminders?
All: Yes. :)
Karen: Can do agenda with the reminder.

ACTION: Karen and Adam to sort it out :)

Developer Congress

Justin: Shoud we have Developer Congress meetings?
All: Yes!
Mark: one community wide per year, and then regional / topical smaller ones. 12-20 a couple times a year.
Tom: Yes. Can be hard to get to lots of meetings though.
Mark: Funding is depending on the institution
Carolyn: Penn State was budgeting for quarterly meetings
Jeremy: Helpful to have a house coordinator, eg air-bnb. Facillitation for where to stay.
Mark: There's about 4 trips per year. Dev congress, connect, plus 2 that any individual would go to, but more ad hoc and not everyone.

All: LDCX discussion -- another trip to take into account.
PROPOSAL: LDCX as 2 days of unconference, 3 of hydra dev work.

ACTION: Rob and TomC to work out a plan for LDCX

Jeremy: People charged with bringing an idea to the congress worked well elsewhere.

PROPOSAL: San Diego in the winter. Who is following up on it?
ACTION: Rick and Mark to work with steering to make sure it happens

Justin: Need people not in the room to come and engage. Need a day to solve problems and pair program.
All: (Agreement)
Esme: Maybe pre-conf at code4lib on a certain topic. eg RDF. class-room style rather than heads down pairing.
Justin: I was talking more about one on one etc.

Tom: Workshops at non hydra events vs. hydra dev congress. Let's focus on congress. No projectors, hack-fest? Need a bit more structure to do what Justin is saying.
Jeremy: Code retreat model is good -- solves a team and how we develop together problem.
Big mixer of developers and work through idioms.
Joe: At the front end of the week.

Carolyn: New people should bring the ideas.
Joe: Ambassadors for the ideas
Esme: Training events to get new people interested to feel comfortable to come at all important. Then we need to get them to be productive.

ACTION: Jeremy, Justin, Aron Coburn to work on a plan.

Training Engendering Participation

Mark: training session this afternoon, please come to that.
David: Deeper level of participation. Need a place to get deeper into the stack and talk about it. Stack is pretty deep. Pair programming good, but when do we have the hard conversations.
Jeremy: Can have them, but at a hard point in understanding how we're forming solutions. Dev congress would not solve institution problem, it's solving the programmers working together well problem. Build a similar mindset.
David: Committers call is lonely in different ways. Everyone friendly and nice, but could have more team work. Hard to bond with people.
Esme: Week long hack fest with a sprint and tickets, 5-6 people. Very deep collaboration.
Jeremy: Need to decide when to go fast, when to go slower but further.
Justin: New people, been to camp, hard to do advanced training as it's broad. Which part to look at is different for everyone. Need to have somewhere to do hand holding.
Mark: Advanced hydra camp was well received. Just settled what was in 1, would advanced be good? e.g. rdf modeling in hydra
All: (Agreement)
Jeremy: Some people want to receive, some will want to help present.
Mark: Would that help if we planned one of them per year? Need to figure out how to do it in practice.

Hierarchy of Promises

All: We can't agree on the whole hierarchy, but a few people up to auditing it? Fixing the language, etc. Agreement that we have the authority to update it. Good for developer event -- walk through and fix it with the right people first, and then use as introduction for newcomers.

Rick: Dan BH to be one of the people
David: Needs to be updated for F4
Justin: Rewrite for Hydra8?
Tom: The right time to do this -- eg what's the commitment to LDP
Jeremy?: Need an audit of it. Plus someone to steer the conversation.
Joe: Characterization of practice, and intent. Should be deliberate about intent, and audit what the code actually does -- doesn't need institutional agreement.
Justin: HoP is above the code level.

Chris: What is the document used for?

(flurry of discussion)
The Name doesn't help. Not complete list.
Who is the audience? Developers?

Giarlo: Should be an agenda item for future call.

ACTION: Joe to organize working group to look at the document. e.g. Dan FB and Tom.

Longer Term Vision

Justin: Getting Active Fedora 8 ready -- Hydra on Fedora4. Trying not to change things we dont have to, other than making things rdf-y.
Tom: Is that process about done? Changing as little as possible, works on F4... done?
Adam: 90% done?
Jeremy: Vision for the year.
Justin: Will have beta hydra for fedora4 when it releases. Early spring.

(Discussion as to actual date)

Rick: Where do we want to go? This is just what we're doing now.
Tom: Community relationship to LDP, F4 is just one implementation of it.
Jeremy: Also useful for things like Riak -- longer term decomposition problem. Need to discuss how to go about doing that.
Esme: Longer term vision -- less of a special Fedora front end, more of a decoupled but cohesive platform.
Justin: Specifically for ActiveFedora.
Tom: Modeling of concerns in AF is the problem.
Esme: Medium term is F4. Longer term is generic stack.
Justin: Universal / common / extensible model construct for multi-file works. Everyone needs it. Needs to be generic.
Jeremy: How do we ask for the details of the model.
Justin: Extracted Works from Curate. Then people doing gap analysis, eg to put it in Sufia.
Mark: Solution bundle X, Y, Z... what no one gets is the object models are incompatible. Can't use Avalon with Sufia.
Esme?: Make a model that could be shared, how to interact with it in a reliable way.
Justin: Needs to be in HoP.
Esme: Agree on DC for basics, agree on other ontologies for extra.

ACTION: Joe to do user stories, Matt to gap analysis. Jeremy, Tom for model. Pursue multi-file agreement, how going to implement. Works Working Group.

Relationship to Fedora4 / Short Term goals

Esme: Versus LDP? Or feature sets, eg for projection? What goes in Hydra what goes in Fedora.

(Discussion)

Issues re LDP vs Fedora:
* Versioning
* Transactions
* Fixity checking

Hypothetical ActiveMarmotta could handle the divergence between LDP and Fedora requirements. Bulk of the operations are just LDP.

Core stuff that's LDP should be deployable spearately from Fedora specific stuff: Fedora / LDP would be decoupled.
Marmotta is the main implementation of LDP, several others.
Would be essentially breaking ActiveFedora into multiple components and making it easier to do things together -- eg more flexibility and more convergence.

Someone familiar with LDP and Fedora should inventory the interactions. (Chris Beer suggested)
Then loop Chris into architecture discussions. After that have separate task to get the F4 only features implemented.

Interactions on resources just LDP. Transactions etc
Modeling of Fedora interactions into LDP -- just Rob and Chris at the moment, any other people interested? Tom interested in LDP modeling.
Jeremy: Hangouts with recording? A persistent water cooler?
Esme: Emails to the hydra-tech list. Checkins on the call.


Release Management

Good practices for releasing and managing gems, w.r.t. dependencies.
E.g. where to peg the version to dependencies and giving guidance.

Products that are diverging, eg sufia / curate. Effort to converge is how to decompose the stack so we're not stuck when developing. Effort to avoid making divergent but arbitrary decisions.

Rick:

E.g. Works -- we feel like we need to converge. THen allow adoption, and stop unnecessary divergence. Publish an API for shapes of works. Common data model -- Build in flexibility/extensibility, but have shared framework underneath, can use it in different places. Can allow local divergence, but promote shared base.

Mark: Good for greenfield, but hard for legacy. More transparency would help between models.

Jeremy: Want to expose an API. The structure can be interacted with via an API in Sufia, or other models. What are the predicates, what do they mean ... rather than mandating predicates.
Need various options, have a show and tell, and look at what is abstractable. Then making the clear contract of what to expose.

Justin: UCSD has components ... should everyone do it or not?
Jeremy: Comes down to discussion. Probably face to face. Small enough group to really engage.
Hacker house option -- deep dive into this.
Esme: Timelines are short, esp with travel.
Justin: not going to get decisions very quickly.
Giarlo: Not going to solve divergence, but need to acknowledge it. And decide what to work on together. About communication.
Esme: Good to get to pain points and then come together. Not sustainable or maintainable.
Rick: Divergence can be healthy.
Mark: F3/F4 repos are going to diverge.

ACTION: (several people) Looking at works concept and trying to get people to work through it. What works doing, how interacting with them, etc. How are we going to do that? back to tech call, and where/when face to face?
Institutions: UCSD, Oregon, DCE, Penn State, Avalon, Stanford, Notre Dame.

Carolyn: Compare use cases, worth-while. Report the findings, and then look at how to proceed.
Justin: Look at model, not the UI for worthwhile.

Discussion of Divergence:

* Lots of divergence in many areas.
* Lots of people want to keep XML.
* Convergence on transformation to RDF.
* RDF WG to help institutions with modeling.

Refactor OM so teach transformation once, and then do mapping.
Push OM closer to ActiveModel? Some fundamental flaws.
Other stuff out there that's better than OM.
OM good at reading XML structures.

Mark: Look a year out, what can we do with LDP and F4, having OM hanging around might hit the point where we look back at how to support XML.

Release Management

* Use semantic versioning.
* Everything should run through CI and pass.
* CLA on file before merging a PR
* Email out the release notes
* Update history.txt

Small group to review documented process. Similar incantations in different repos.
Script to push things up to all repos
Not quite contributing, more releasing.

Hydra gem which is a manifest of supported state of the art stack.
Use hydra gem 7.1 to make sure have all baseline stack. 6 month basis.
No code, just the set of other gems.
Couldn't upgrade stack -- will document the dependencies this way.

ACTION: Jeremy submit a PR for releasing.md and propogate.
~~ACTION: Mark to send wiki URL to Rob~~
https://wiki.duraspace.org/display/hydra/Release+Manager+Responsibilities

Giarlo: some rake differences in releasing.
Do rake -t.

Chris: testing for hydra gem. How is it determined when to release a new version?
Workshop/Hydra camp / dive into hydra to check release
that's not in the docs, should be, and should be super stable.

ACTION: ??? to write Releasing the hydra gem docs. How to decide to do it, and verify that it's good?

Announce release candidates. From one week to next (or two weeks or ...)
Need more people to have time assigned to review.
If no one says anything, then it's good.

As use it for training, majority of new adopters start with it. Everyone should be using it unless some big reason to diverge.
Would be good to have everyone used it for production.
Long term vision.
Would need faster release cycle.

If have sufia / worthwhile etc in gem file, then can't use hydra gem file.
Need stable versions.

__End of Discussion__