Samvera Tech Call 2024-10-30
- Chris Colvard
- Daniel Pierce
Meeting Logistics:
Time: 9:00am PDT / Noon EDT
Zoom Meeting URL: https://princeton.zoom.us/j/7739591625
(link will launch Zoom client – if you do not have Zoom, expand the instructions below)
Agenda (meeting notes below)
<add your agenda item here (your name)>
Moderator: @dmoles
Notetaker: @Daniel Pierce
Attendees:
@Rebekah Kati
Margaret Kibi
@Chris Colvard
@Bradley Watson
Meeting Process
Notes
Margaret: Bulkrax work, noticed there isn’t any locking in transactions for creating valkyrie objects. Added it, checking if it’s needed upstream.
Collections and members of collections were being created in parallel.
David: What is the right way to do this? How should it work in hyrax?
Overrode create method of work factory
How many people using bulkrax with valkyrie? Hyku at least.
Customized bulkrax a bit in the way it retries failed jobs; might have caused this to surface.
Daniel: Related work: IU sequential jobs feature that delays enqueuing until current job is finished.
Put a lock around just the relationship assignment? Should relationship json be a first class valkyrie resource?
Is this a bulkrax specific problem, or does it affect any use of hyrax transactions.
Only noticed during bulkrax use, but also with large file that take longer to process.
Lock timeouts: hyrax uses a large timeout for locks, 2 minutes
Made retry shorter, a couple seconds.
Could pass in optional lock length if needed in certain places.
Still using redlock, interested in moving to something without redis dependency
Need rails 7 to move to alts
Moving to GoodJob form Sidekiq also would remove redis
Chris: reading about rails 8, features also targeted at avoiding redis dependency