Mission and data annotations

Annotations are user-entered records that add structured human intelligence to robot telemetry or mission data. The template or structure of these records are defined using tags within Formant or in Google Sheets . Annotations are associated with time and can be over a range of time but requires at least the start time as part of its definition. Annotations can be streamed or exported to Google Sheets for further analysis. For example, to analyze mission performance, create mission annotations (either manually or programmatically) and then stream it to Google Sheets and run analysis on it.

Prior to creating annotations, it has to be defined.

Defining annotations in Formant using tags

To create an annotation template in Formant, navigate to Settings>>Annotations and click "Add Annotation" in the top right.

Give the template a descriptive name which outlines the context in which it will be used, and then filter down which devices should have access to this annotation template. Then click Next at the bottom right to navigate to the Settings tab.

In the Settings tab, click on "Add Tag" to create a new field in the annotation. These defined fields will be used to describe the situation they label. Give a name for the field, and enter the distinct values this field can have. In the example below, the field in the annotation is Severity, and the discrete options we give the user to select from are "Minor", "Major", "Critical", and "Blocker". You must add at least one value for a user to select from, as these fields are not free form. After typing in each value, press enter to finish inputting the value. You can also make this field required by toggling the "Required" slider. Note, annotations will, by default, include a notes section for users to add description or additional information that may not be captured by the fields you enable.

Defining annotations in Google Sheets

In order to create an annotation template in Google Sheets, the data must be organized in a way that Formant understands. There is a one-to-one mapping between a Google Sheet tab and a row in the annotation template. If multiple rows are required in the annotation template, then separate Google Sheets/tabs must be used to define each annotation row. The Google Sheet must be organized as follows:

• The top-left cell (A1) is a column heading and uniquely identifies a row in the annotation template (and name)
• The rows below A1 contain the values available for selection for that row
• Additional columns (to the right) can be used to constrain the values within the first column. In addition to mapping the values in the same Sheet to indicate filtration, these columns must be defined in additional Sheet or tab for it to appear as option within the annotation template.

For example, consider the definition in this spreadsheet. This spreadsheet has three tabs - Jobs, Activities and Subactivities. The first tab Jobs has column named Job Name (A1) which will be the annotation row name. The rows A2 and A3 will be the values for that annotation row.

Even though B1 (Location) is defined in the same spreadsheet, since it is not defined in any other spreadsheet, it will not be displayed as an annotation row and hence cannot be used to constraint the values of Job Name.

The Activities tab lists the set of activities. The first column A1 -- Activity Name -- defines the annotation name and the rows below it define the There are no constraints defined for Activities.

The Subactivities tab defines the Subactivities -- each Subactivity is constrained by an Activity Name. In the annotation template, only Subactivities that are part of the selected Activity Name will be displayed in the dropdown list.

For example, if Mobilization is selected as the Activity Name , then Machine at location and _Equipment at location _are displayed as Subactivity options and the other Subactivity items are not displayed.

📘

Info

Changes made to the list in Google Sheet will reflect in the Formant interface within 2 minutes. So, if a new row is added to annotation, it will automatically show up in the annotation feed within 2 minutes.

Steps to define annotation template in Google Sheets

• From the burger menu, select Settings ⇒ Annotation ⇒ Add Annotation
• Choose a name for your annotation definition. This will be displayed on the device channels and event pages when adding new annotations.
• Choose devices - You can limit the devices on which a specific annotation definition applies.
• Add Sheets - This is the section to add sheet URLs. Each Sheet URL added is a new row within the annotation.
• Click SAVE to save the definition and exit to the Annotation Settings page.

Create annotations

Annotations can be created from Device Channels or Modules. From the device channel or modules more menu select Annotate.

If the option to Annotate is unavailable on the menu, this is because there are no Annotations defined for the current device.
If more than one Annotation is available for the current device, the user is prompted with a choice of available Annotations. After the desired Annotation is selected, or if there is only one Annotation, the form for submitting annotations is displayed.

The form for submitting annotations contains the following input fields:
Start time and duration - Describe the time period being annotated. The start time is determined by the current time selected by the scrubber in the Device View. The duration is input in minutes. Duration is optional and can be updated later if it is not known when the Annotation is initially submitted.

Drop-down menus for each configured item in the Annotation - Each sheet in the Annotation is represented by a drop-down select menu. The options listed in each menu will be constrained by the selections made in other sheets.

Note (optional) - Any additional notes about the annotation

Tagged users (optional) - Users who are linked to this specific annotation. This might not be the user who is creating the annotation.

After all form fields are filled, the annotation is submitted by selecting Save.

If any required fields are not filled, the fields will be highlighted red and the annotation will not save. The user should input all of the required fields and select Save again.

Annotations are visible in Device View in the scrubber area at the bottom.

Clicking an Annotation causes it to be selected and details visible in a header. The header includes an Edit button which is used to change annotated data after it has already been created. The header also includes a Delete button which deletes the annotation record.

Publish Annotation

Annotation can be published "real-time" to a Google Sheet. When an annotation is created it is appended to the sheet. When the annotation is updated, the sheet is also updated. The update is achieved by storing locally with the annotation event the spreadsheet location where the annotation was appended.

It is an optional tab under Add New Annotation and designates to which sheet the annotations that use this template should be published.

There are some limitations to publishing to Google Sheets:

• The user should not add or delete any rows that have been added by Formant. This may cause the wrong rows to be updated when the annotation is updated.
• Changing the annotation template will change the spreadsheet column used for each field in the template because Formant does not read the spreadsheet header row to determine what the columns are.
• Only exports to the 1st tab in the spreadsheet
• Does not delete the exported annotation if it is deleted in Formant Export Annotation

Annotations can be also exported from the events page by using the Export button at the bottom of the page. The Export button respects filters on the events page. Hence, to download only the annotation, filter down the event type to Annotation and then export to Google Spreadsheet.