Performance testing is a key part of delivering a Minimum Viable Product (MVP) as we build support for Fedora 6 as a back-end option. As part of the Hyrax Fedora 6 Working Group’s scope, we want to include performance benchmarking for key resource-intensive activities performed by Hyrax users. To reflect the current state of Hyrax adoption, the following combinations will be tested with representative performance tasks.
Hyrax, ActiveFedora, and Fedora 4 (for baseline comparisons)
Hyrax, Valkyrie, and Fedora 6
Hyrax, Valkyrie and Postgres
Representative Performance-Intensive Tasks
Run fixity checks on 5,000+ works
...
Metrics: time to complete
...
The following activities have caused known performance issues in Hyrax with ActiveFedora and Fedora 4.x. These issues are compounded when executed in a repository with a significant amount of content already ingested, but given the challenges of generating a representative content sample in a test environment, and trying to approximate a baseline operating environment, these minimum statistics test cases are suggested as minimum thresholds only. The activities noted below Fedora 6 test cases will also assume a basic Hyrax installation using a default Hyrax work type/model .
...
Bulk-ingest a work with over 100 FileSets using Bulkrax and a default Hyrax model (e.g. Generic Work)
Metrics: time to complete
Dependencies: Bulkrax configuration supporting a default Hyrax work type/model
...
Bulk-ingest one or more new works using Bulkrax into a collection with over 5,000 works
Metrics: time to complete
Dependencies: 5,000 works must be pre-populated
...
Add a FileSet to a work with over 100 Filesets using the Hyrax user interface
Metrics: time to complete
Dependencies: work with FileSets pre-populated
...
Edit metadata for a work with over 100 FileSets
Metrics: time to complete
Dependencies: work with FileSets pre-populated
...
Change visibility for a work with over 100 FileSets
Metrics: time to complete
Dependencies: work with FileSets pre-populated
...
Add a work to a Collection with over 5,000 works using the Hyrax user interface
Metrics: time to complete
...
Edit metadata for a Collection with over 5,000 works
Metrics: time to complete
Dependencies: 5,000 works must be pre-populated and assigned to a Collection
...
Delete a FileSet from a work with over 100 FileSets using the Hyrax user interface
Metrics: time to complete
Dependencies: work with FileSets pre-populated
...
Delete a work with over 100 FileSets using the Hyrax user interface
Metrics: time to complete
Dependencies: work with FileSets pre-populated
...
View a Work Page with over 100 FileSets
Metrics: page load time
Dependencies: work with FileSets pre-populated
...
Navigate between page 1 and 2 on the Items list for a Work with over 100 FileSets
Metrics: page load time
Dependencies: work with FileSets pre-populated
...
View a Collection Page with over 5,000 works
Metrics: page load time
Dependencies: 5,000 works must be pre-populated
...
View a FileSet page attached to a work with over 100 FileSets attached
Metrics: page load time
...
Use the File Manager to re-sort FileSets for a work with over 100 FileSets attached
Metrics: time to save changes
...
Generate a IIIF manifest for a work with over 100 FileSets attached
Metrics: time to complete
Dependencies: image files compatible for IIIF rendering
on a Nurax environment.
Additional References:
System Stack Error when deleting FileSets [is this still applicable?]
Applying visibility change to a work with many FileSets in the UI fails/times out
Performance Tests (draft)
Worksheet (please add more test cases, examples and/or local benchmarks)