Upgrading Avalon 7.8 to Avalon 8.0
Requirements
Ruby 3.3+ and Node.js 20 are recommended.
Redis 6+ is required.
Config changes
Solr 6 configuration files removed
Ensure solr has been upgraded to solr 9 (see instructions included in Avalon 7.6 → 7.7 upgrade notes)dropbox.sharepoint (options) - settings to enable sharepoint integration in dropbox modal
See Sharepoint Dropbox for details about setting it up.ffprobe (optional) - path to ffprobe executable on Avalon server (defaults to
ffprobe
)
Troubleshooting
Restarting export or migrations in case of errors
If you run into issues when running the Fedora 4 export, it can be resumed by re-running the export with a couple different flags: repositoryRoot
and resourcesFile
instead of resource
. You will have to find the remaining_*.log
file that matches that last failing run of the exporter.
jdk-21/bin/java -jar fcrepo-import-export-1.2.0.jar -b --dir fcrepo4.7.5_export --user fedoraAdmin:fedoraAdmin --mode export --repositoryRoot http://fedora:8080/fedora/rest --resourcesFile remaining_TIMESTAMP.log --binaries --membership --auditLog > importexport_`date +%Y%m%dT%H%M%S`.log 2>&1
The same approach can be used for the migrations by passing in --resource-info-file
and the matching remaining_*.log
file.
Checking that export and migration are complete
If the export ran to completion (log file should include (Exporter) Export complete
) and there isn’t a remaining_*.log
file then the export should be complete.
Upgrade Steps
For Manual Installations
Pre-upgrade steps
Ensure section_list migration has been run (See Upgrading Avalon 7.7 to Avalon 7.8 | MediaObject Section List Migration)
Export Fedora 4 repository and migrate it to Fedora 6 OCFL
cd /tmp mkdir fc4_to_fc6 cd fc4_to_fc6 # Setup Java 21+ (if needed) curl -O -L https://download.java.net/java/GA/jdk21/fd2272bbf8e04c3dbaee13770090416c/35/GPL/openjdk-21_linux-x64_bin.tar.gz tar xvzf openjdk-21_linux-x64_bin.tar.gz # Download specific version of import/export tool curl -O -L https://github.com/fcrepo-exts/fcrepo-import-export/releases/download/fcrepo-import-export-1.2.0/fcrepo-import-export-1.2.0.jar # Export from fedora 4 jdk-21/bin/java -jar fcrepo-import-export-1.2.0.jar -b --dir fcrepo4.7.5_export --user fedoraAdmin:fedoraAdmin --mode export --resource http://fedora:8080/fedora/rest --binaries --membership --auditLog > importexport_`date +%Y%m%dT%H%M%S`.log 2>&1 # Download specific build of upgrade-utils curl -O -L https://github.com/avalonmediasystem/fcrepo-upgrade-utils/releases/download/6.3.0-AVALON/fcrepo-upgrade-utils-6.3.0-AVALON.jar # Run migration Fedora 4 -> Fedora 5 (inside container again) jdk-21/bin/java -jar fcrepo-upgrade-utils-6.3.0-AVALON.jar --input-dir fcrepo4.7.5_export --output-dir fcrepo5_export --source-version 4.7.5 --target-version 5+ > upgrade_5_`date +%Y%m%dT%H%M%S`.log 2>&1 # Run migration Fedora 5 -> Fedora 6 jdk-21/bin/java --add-opens java.base/java.util.concurrent=ALL-UNNAMED -jar fcrepo-upgrade-utils-6.3.0-AVALON.jar --input-dir fcrepo5_export --output-dir fcrepo6_export --source-version 5+ --target-version 6+ --base-uri http://fedora:8080/fedora/rest > upgrade_6_`date +%Y%m%dT%H%M%S`.log 2>&1
Setup Fedora 6 (See Deployment - Fedora 6.x Documentation - LYRASIS Wiki )
Copy migrated Fedora 6 OCFL directory to
fcrepo.home
and restart Fedora
Upgrade steps
Install ruby 3.3.x (if needed)
Update gems, and migrate DB
bundle install --with production --without development test RAILS_ENV=production bundle exec rake db:migrate
Install node.js 20.x (if needed)
Update JS dependencies
Update configuration, see Config changes above
Generate credentials (if needed)
Restart passenger (if using)
For Docker Installations
Before upgrading an AWS docker install based on avalon-terraform
or the aws_min
branch of avalon-docker
make a backup of your supplemental files prior to destroying the avalon container!
Then load them into an S3 bucket:
See Investigate Supplemental File Storage · Issue #77 · avalonmediasystem/avalon-docker for more discussion on this. If you run into any problems, please reach out on slack or the email list.
Pre-upgrade steps
Ensure section_list migration has been run (See Upgrading Avalon 7.7 to Avalon 7.8 | MediaObject Section List Migration)
Export Fedora 4 repository and migrate it to Fedora 6 OCFL (inside avalon container)
Copy migrated data out into new
fedora_data
directory (outside avalon container)
Upgrade steps
Stop Avalon
Update new code from avalon-docker
Pull new images
Setup fedora 6 cache database
Check config changes then restart the Avalon containers
You can also build your own image, Docker Buildkit is recommended
For Terraform installations
These instructions are a work in progress.
Backup all data including the EC2 filesystem and s3 buckets (if possible) prior to applying the new terraform as it will destroy and rebuild most of the infrastructure including the EC2 instance.
Before upgrading an AWS docker install based on avalon-terraform
or the aws_min
branch of avalon-docker
make a backup of your supplemental files prior to destroying the avalon container!
Then load them into an S3 bucket:
See Investigate Supplemental File Storage · Issue #77 · avalonmediasystem/avalon-docker for more discussion on this. If you run into any problems, please reach out on slack or the email list.
Pre-upgrade steps
Ensure section_list migration has been run (See Upgrading Avalon 7.7 to Avalon 7.8 | MediaObject Section List Migration)
Export Fedora 4 repository and migrate it to Fedora 6 OCFL (inside avalon container)
(optional) Tar up export and migrations for backup (still in avalon container):
Update avalon-terraform on your local machine to the latest in main and run the following to create the s3 bucket for the Fedora 6 OCFL data store:
Outside of the avalon container copy migrated Fedora data to the new s3 bucket:
(optional) Put export/migration backups into s3:
Upgrade steps
Set
avalon_docker_code_branch
andavalon_branch
interraform.tfvars
if necessaryPerform the upgrade:
This might error on destroying fedora 4 s3 buckets if they aren't empty. You can safely ignore these errors and decide later if you want to backup and retain these buckets or delete them.
You may also see an error when attempting to create an iam user which already exists. If this happens rerunning the terraform apply should resolve the issue and successfully run to completion