Uploading Content using Batch Ingest

Introduction

While content can be uploaded to Avalon either individually or several at a time using an Avalon dropbox, the batch ingest feature creates many items with associated metadata at one time outside of the Avalon user interface. This user guide will explain how to form an Ingest Package and upload it into Avalon using Batch Ingest.

For help with resolving issues related to a failed Batch Ingest, see Troubleshooting a Batch Ingest.

Manifest Template

Table of Contents:

Definitions

  • Batch Ingest: the process of creating multiple media items by uploading an Ingest Package to an Avalon dropbox
  • Ingest Package: the combination of content and metadata, consisting of one Manifest File and one or more Content Files
  • Manifest File: a spreadsheet containing the metadata for the items to be created as well as the names of the Content Files that make up each item
  • Content File: a single media file that is part of an item; one or more Content Files can make up an item in the Avalon Media System
  • Item: a single media object accessible through one Avalon page; consists of one or more media files and metadata describing the media file(s)
  • Collection: a grouping of items in Avalon for administrative and discovery purposes. Items belong to one and only one collection

Preliminaries

When a new collection is created, Avalon creates a sub-directory with the name of that collection (substituting underscores for any blanks) within the Avalon dropbox directory. A Batch Ingest is initiated by uploading an Ingest Package to the collection sub-directory in the dropbox. To connect to the Avalon dropbox, see Uploading Content to an Avalon Dropbox.

The Manifest File is a spreadsheet (xls, xlsx, csv, or ods) containing the metadata for the items to be created, as well as the file paths to the content files that make up each item; download manifest_example.xlsx for a blank manifest template. The Manifest File lists many metadata fields, but only a few of these fields are required: "Title," "Date Issued," and "File." However, "Bibliographic ID" may be used in place of "Title" and "Date Issues." For a description of the other fields, as well instructions for adding structure or captions, see Batch Ingest Package Format

Creating a Basic Manifest File

  1. Open the Manifest File that will contain the metadata for the Content Files.
  2. Column A Row 1 contains a reference name for the batch; this field is only for reference and should be renamed to something convenient or descriptive.
  3. Column B Row 1 contains the email address or username of the submitter, to be used for notifications and exceptions. The submitter's username or email address must already be assigned as manager, editor, or depositor for the collection to which the batch is submitted.
  4. Each of the columns on Row 2 specifies the potential metadata fields for each Item. If a field for an item is multi-valued (e.g. has more than one topical subjects or languages, or the Item is made up of more than one Content File), the column can be duplicated with the same header.
  5. Row 3 and and all subsequent rows each represent a single Item to be created in Avalon. The minimum required fields for each item include:
    1. Title:
      1. Not repeatable: each item may only have one title.
      2. The title field represents the name of a single item, which may consist of one or more Content Files.
      3. If title is not available or missing, create a title that describes something about the content of the item.  This is necessary for identifying items in search results.
    2. Date Issued:
      1. Not repeatable: each item may only have one date issued.
      2. Date should be the main publication date associated with the item.
      3. Simple format should follow "yyyy-mm-dd", see EDTF specifications for other options, including a date range or an unknown date.
    3. File:
      1. The name of a Content File, which consists of two parts:
        1. The path to the file, relative to the Manifest File
        2. The name of the file itself, including the file extension (e.g. video1.mp4, songB.aac)
        3. Both path name and file name are case-sensitive; incorrect cases will result in a processing error.
        4. Example: A Manifest File and a folder titled "video_content" (with .mp4 Content Files) are uploaded to the dropbox collection "MyVideos". The Manifest File and the folder are now located in the same directory; therefore, each file name would follow the format of video_content/videoname.mp4
      2. Repeatable: if an item consists of several Content Files, the column "File" should be duplicated for each individual Content File constituting the item.
        1. Each "File" column should be immediately followed by a "Label" column.
        2. "Label" represents the name or short description of each Content File (e.g. Part 1, Introduction, essay topic, musical movement, etc.)

Example Manifest File showing reference name, username, Title, and Date fields (please note there are some intentional omissions in the Title and Date fields in this example):

Continued (file name and path name, file labels):

Uploading an Ingest Package to the Dropbox

A Batch Ingest will begin automatically once the Manifest File and the Content Files are uploaded to the appropriate collection in the Avalon dropbox. If you are not yet connected to the Avalon dropbox, follow the instructions at Uploading Content to an Avalon Dropbox. Once you are connected, uploading the files is a simple drag-and-drop process. The image below shows the files to be transferred in the left pane, and the files after they have been transferred in the right pane.

If the spreadsheet passes validation, an email notification will be sent to the user listed on the Manifest File stating that the spreadsheet has been accepted:

After all of the objects listed on the Manifest File have been successfully uploaded, an email notification will be sent to the user regarding the completion of the Batch Ingest:

This concludes a successful Batch Ingest process; all the items should be uploaded into the Avalon Media System and either ready for further editing or available for viewing. However, if there are any errors with the spreadsheet or with any of the media objects on the spreadsheet, the user will receive an error message. To resolve any issues with a Batch Ingest, see Troubleshooting a Batch Ingest.