Upgrading Avalon 7.7 to Avalon 7.8

Owned by Chris Colvard
Last updated: Aug 20, 2024
3 min read

Requirements

Ruby 3.2.x and Node.js 20.x are recommended.

Config changes

There are new options for how to handle fedora and solr connection errors that happen when servicing requests (app_controller) and background jobs (app_job). The defaults are to raise errors in background jobs so sidekiq can reenqueue them and only log in controllers allowing the possibility of better error messages for users instead of a generic 500. This is configurable in settings.yml.

app_controller: solr_and_fedora: raise_on_connection_error: false app_job: solr_and_fedora: raise_on_connection_error: true

Derivative download is a new feature in Avalon 7.8 and is enabled by default for admins and collection managers.

derivative: # Choose whether collection managers and admins can download high quality derivatives allow_download: true

A maximum upload size for web uploaded files has been disabled by default. It can be enabled and set (in bytes) by uncommenting the line in settings.yml.

# Maximum size for uploaded files in bytes (default is disabled) #max_upload_size: 2147483648 # Use :none or comment out to disable limit

Troubleshooting

Before starting the upgrade there are some steps to keep a close eye on when running.

SupplementalFile Migration

Transcript search, a new feature in Avalon 7.8, is a powered by indexed SupplementalFiles and relating them back to their owning parent objects. This migration populates a new `parent_id` column on the SupplementalFile model.

Some objects may not migrate properly and errors encountered will be written to log/parent_id_backfill_errors.log. Besides checking the logs, you can check the number of indexed SupplementalFiles against the total number of SupplementalFiles in the rails console:

If the migration reports that the migration was successful but the counts do not match, it could be that parent objects have been deleted orphaning SupplementalFiles, a known problem.

MediaObject Section List Migration

Avalon 7.8 moves data about section order from a separate fedora object to a property on the MediaObject model to be more performant when objects have many sections. The MediaObject model is already coded to lazily migrate existing objects whenever they are saved and this migration will finish updating objects that haven’t already been migrated.

Before running the migration, you can dump the section lists stored in ordered_master_file_ids by running:

Some objects may not migrate properly and the script will log any errors that happens. This could be due to networking issues or objects in an atypical state. Trying running the migration again and reach out to the Avalon team if you have problems resolving any objects that fail to migrate.

After the migration, run the following line to dump again and validate that the section lists, now stored in section_ids matches the one dumped prior to the migration:

Note that while the migration is running there will be duplicate values in the format facet in search results. The indexed value of the format changed in 7.8 so both will appear until all items have been migrated.

Upgrade Steps

For Manual Installations

  1. Install ruby 3.2.x (if needed)

  2. Update gems, and migrate DB

  3. Install node.js 20.x (if needed)

  4. Update JS dependencies

  5. Update configuration, see Config changes above

  6. Restart passenger (if using)

  7. Run rake migrations

For Docker Installations

Before upgrading an AWS docker install based on avalon-terraform or the aws_min branch of avalon-docker make a backup of your supplemental files prior to destroying the avalon container!

Then load them into an S3 bucket:

See for more discussion on this. If you run into any problems, please reach out on slack or the email list.

  1. Stop Avalon

  2. Update new code from avalon-docker

  3. Pull new images

  4. Check config changes then restart the Avalon containers

  5. Run rake migrations (inside avalon container)

You can also build your own image, Docker Buildkit is recommended