Waveform Backfill Script

The is under avalon/script/waveform_backfill.rb.

• Upon initial run, it scans all master file solr indices to retrieve the IDs of those that don't have waveform stored yet, and save them into a tmp file '/srv/avalon/scriptdata/waveform_backfill.txt'
• It then schedules Waveform Job for each such master file
  
• Upon subsequence runs, the script will reuse the saved file to avoid massive read from solr.

To run the script, make sure directory srv/avalon/scriptdata/ exists on the server.

Usage: bundle exec rails runner script/waveform_backfill.rb [start_index] [ row_max]

The script takes 2 optional parameters:

1. start_index: the index of the row within waveform_backfill.txt, default 1. If the script failed and needs a restart, one can check the log to see which row/masterfile ID it failed at, and restart from there.
   
2. row_max: maximum number of rows to retrieve, default 1000000. This is only useful for the initial run, especially for testing, one can set a small number to test.