Waveform Backfill Script
- Ying Feng
Owned by Ying Feng
The is under avalon/script/waveform_backfill.rb.
- Upon initial run, it scans all master file solr indices to retrieve the IDs of those that don't have waveform stored yet, and save them into a tmp file '/srv/avalon/scriptdata/waveform_backfill.txt'
- It then schedules Waveform Job for each such master file
- Upon subsequence runs, the script will reuse the saved file to avoid massive read from solr.
To run the script, make sure directory srv/avalon/scriptdata/ exists on the server.
Usage: bundle exec rails runner script/waveform_backfill.rb [start_index] [ row_max]
The script takes 2 optional parameters:
- start_index: the index of the row within waveform_backfill.txt, default 1. If the script failed and needs a restart, one can check the log to see which row/masterfile ID it failed at, and restart from there.
- row_max: maximum number of rows to retrieve, default 1000000. This is only useful for the initial run, especially for testing, one can set a small number to test.