2023-05-24 Meeting notes

Nurax terraform overview

Thanks to Michael Klein (Northwestern University Libraries) for taking us through a tour of the Nurax terraform repository in samvera-labs GitHub - samvera-labs/nurax-terraform: Terraform manifests to get nurax, nurax-pg, and all dependencies running in AWS !

In March 2023, Michael, Daniel and others ported Terraform code wholesale from NU to a new repository:

• Infrastructure-as-code repository (community resource) designed for reusability

• Deploys three (3) environments: stable, dev and pg-backed

• One (1) ECS task with multiple definitions running inside of it (Fedora, Solr, etc.)

• No solr cluster, no Zookeeper

• Data mounted as Elastic File System (EFS) volumes

• Who has the ability to deploy the Nurax terraform code? Individual access granted to users out of IAM with permissions for read/write.

• Uses a custom Fedora 4 build container (same one in use at NU), which contains a bugfix to Modeshape that allows for multipart S3 uploads

  • Rob K. and Michael K. talked about swapping Fedora 4 to fix issues with S3 uploads and newer Postgres versions (they updated dependencies). Open question about pushing a patch for folks still using Fedora 4 as a stopgap.

  • Rob K. talked about working with Yale to improve issues with solr booting up ECS container tasks. Moved data to EFS, now see consistent boots. Backup directories are EFS volumes as well.

AWS reference architecture next steps: Potentially deployment configurations for using Elastic Search, making it compatible with Blacklight.

Kubernetes/Helm charts

The Hyrax chart (many are using it now):

• Comet uses the Hyrax chart

• Oregon digital also uses it

• Hyku uses it (Hyku charts were repurposed into the Hyrax charts)

• WGBH uses a fork because of a MySQL dependency

High-level elements discussion:

• Discussion around the desire for the reference architecture to include high-level elements along with a description of their roles, upsides, and downsides. (Anecdote told about a new system getting all the way to staging using GoodJob instead of Redis but derivative creation broke because of a hard dependency on Redis that was not easy to discover)

Open discussion

• Should the solr recommendation be clustered or single?

• Solr Cloud is a prerequisite for Hyku. Hyku needs an API for creating solr collections in order to add `solr` cores. Reference architecture would have to include Solr Cloud to be useful to Hyku implementors

• ECS mandates are out there in our community

• `EC2` possibility (though not sure if there any people still doing this)

• Brief discussion about serving static assets with Cloudfront and S3 as an alternative to nginx or puma

Want to gather use cases to see what approaches are taken in the community. This can be a goal for the group. How do we want to gather this information?

A point was raised that October will come sooner than we would probably like to admit, so we can’t wait on this task for too long because surveys take a lot of time (based on past experiences of the group members).

Other items