Long Term Vision and Architectural Oversight

Owned by Bill Branan
Jun 16, 2017

HydraConnect Day 2

UnConference

10/2/14

2:35-3:15pm

Facilitator: Tom Cramer (Stanford)

Notetaker: Jeremy York (Michigan)

What is our long-term vision/Architectural Oversight


Focus: What is the current state of Hydra? Where are we going? What is the architectural agenda of Hydra and how is it set?


Issues

  1. Participation/Communication

    1. As the community has grown, the model of accepting input without too much planning has not scaled

      1. Ex: 15-20 active developers at a time in the past. That has grown quite a bit. Too ad hoc for planning around changes in the core, the direction components will take.

      2. Never more than 5 contributors in the last month.

      3. The concern 6 months ago - people on core gems, but not others.

      4. There is development where someone has a critical need, but development will lag in other areas. Maybe not that many people in there because someone has a critical need, but everyone else does not have the compulsion to do that.

      5. For core gems, one person is doing most of the maintenance.

      6. Development often happens in reaction to projects. More planning may not be the issue, but getting people engaged and communicating between the developer community and those who are engaged in broader planning. Enhancing communication may be enough to give less of an ad hoc feeling to development.


Actions


    1. Intentionally integrate the conversation between developers and planners

      1. Bring the issue of communication between developers and planners up in the core committers meeting tomorrow (10/3/14)

      2. Encourage managers to attend develop tracks - this is often where innovation happens.

    2. Focus on strategies to bring more people into development activities. There are many key pieces but few people working on them.  Identify and create points of Hydra solution convergence:

      1. Introduce a development tithe where when working on a given component, a certain amount of additional effort is contributed to benefit the larger community.

        1. If this is done, development will need to have more oversight and priorities for the year; a published roadmap.

      2. To avoid divergence of solutions and/or create opportunities for convergence and sharing of common code track common feature needs in a central place; also track points of disagreement and why to identify what features should be served by common solution(s)

        1. This could be a hierarchical index/list of desired feature list and process for handling feature requests so that developers entering can know which projects are in need of attention/development.

      2. Encourage newcomers to work on areas they identify as needing work (don’t be shy about whether others will agree or not)

      3. One way of contributing is to offer helpful suggestions to others who are doing development (contact Michael Giarlo, who receives notifications for pull requests).

    2. Focus attention on having the right people (skills, interests, etc.) working on the right problems (organizing development interests and talents so they have the greatest impact).

  2. Connection/Integration

    1. Newcomers to Hydra are often not aware of development projects that live outside of Hydra or Hydra Labs (e.g., projects at DCE). They are hard for newcomers to discover.

      1. Are there things we all use? Is there a Hydra “core”?

      2. Core gems may go silent for periods of time. How do newcomers know that projects they might contribute to will not lead to dead ends (e.g., others are not developing on them)?

    2. Divergence happens from large projects like Sufia, Curate, etc. Code is branched to develop custom functionality. This can be healthy, but what process is there to bring development back together to reduce redundancy and provide more external clarity about which projects will have the greatest impact to contribute to?

      1. How many common components are there?

      2. Is it feasible to develop practices to develop code in smaller modular components that can be assembled into larger blocks, to make the work of integration after branching easier/more efficient?


Actions


 

  1. Pay renewed attention to an up-to-date “marketplace” of Hydra gems and development projects.

  2. Discuss common practices for developing on core and experimental or supplemental components that emphasize modularity of components and reuse.

  3. Face-to-face meetings drive alignment. Leverage in-person meetings as opportunities to bring community development together and re-focus efforts.

  4. Increase frequency of developer face-to-face meetings (2-3 times per year?). Encourage meetings of developers at other conferences.

  5. Increase number of developers on committer calls.

    1. Low attendance might be symptom of other things, e.g., that there is not a lot happening on core; unfortunate scheduling.

3. Architecture and Planning

 

    1. At what level should development occur, e.g.,

      1. Fat heads - development of Hydra heads with lots of functionality

      2. Pin heads - development of small, modularized heads

      3. Pushing functionality further into the “body” (i.e., Fedora)

        1. How much functionality in Hydra is there because of something that was lacking in Fedora 3. What needs to be pushed lower in the stack?

        2. Solr synchronization in Fedora could be handled in Fedora (?) (comment from Michael Giarlo)

        3. Authorization and authentication could be handled in Fedora

      4. For discussion: if someone didn’t want to use Fedora 4, which of the services could they abstract well enough so they were handled by Fedora or something else. Otherwise we are really a front end for Fedora.

    2. How Hydra code manages message passing (I’m not sure I got this well - JJY)

      1. If things happen asynchronously, is development on Rails going to be blocked, waiting until Solr is updated?