...

SupplementalFile Migration

Transcript search, a new feature in Avalon 7.8, is a powered by indexed SupplementalFiles and relating them back to their owning parent objects. This migration populates a new `parent_id` column on the SupplementalFile model.

Some objects may not migrate properly and errors encountered will be written to log/parent_id_backfill_errors.log. Besides checking the logs, you can check the number of indexed SupplementalFiles against the total number of SupplementalFiles in the rails console:

SupplementalFile.count
ActiveFedora::SolrService.instance.conn.get("select", params: { q: "has_model_ssim:SupplementalFile", rows: 0 })["response"]["numFound"]

If the migration reports that the migration was successful but the counts do not match, it could be that parent objects have been deleted orphaning SupplementalFiles, a known problem.

MediaObject Section List Migration (optional)

Avalon 7.8 moves data about section order from a separate fedora object to a property on the MediaObject model to be more performant when objects have many sections. The MediaObject model is already coded to lazily migrate existing objects whenever they are saved and this migration will finish updating objects that haven’t already been migrated.

Before running the migration, you can dump the section lists stored in ordered_master_file_ids by running:

bundle exec rails r "ActiveFedora::SolrService.instance.conn.get('select', params: { rows: 1_000_000, q: 'has_model_ssim:MediaObject', fl: [:id, :section_id_ssim], sort: 'id asc' })['response']['docs'].each {|doc| puts \"#{doc['id']}: section_ids: #{doc['section_id_ssim'] || []}\" }" > section_list_dump.pre_migration.txt

Some objects may not migrate properly and the script will log any errors that happens. This could be due to networking issues or objects in an atypical state. Trying running the migration again and reach out to the Avalon team if you have problems resolving any objects that fail to migrate.

After the migration, run the following line to dump again and validate that the section lists, now stored in section_ids matches the one dumped prior to the migration:

bundle exec rails r "ActiveFedora::SolrService.instance.conn.get('select', params: { rows: 1_000_000, q: 'has_model_ssim:MediaObject', fl: [:id, :section_list_ssim], sort: 'id asc' })['response']['docs'].each {|doc| puts \"#{doc['id']}: section_ids: #{JSON.parse(doc['section_list_ssim'].first || '[]').to_s}\" }" > section_list_dump.post_migration.txt
diff -awu section_list_dump.pre_migration.txt section_list_dump.post_migration.txt

Upgrade Steps

For Manual Installations

...