Telemetry Streams

A telemetry stream is a named sequence of datapoints.

A datapoint is a typed, timestamped data payload. A datapoint can be optionally tagged with several key-value pairs.

A telemetry stream may contain text data, numerical data, perception data, navigation data, log files, or any other of Formant’s supported stream types. It is the core building block of the Formant data platform.

The full list of stream types is as follows:

Type	Description	Asset?
Text	A string	No
Numeric	An integer or float value	No
Numeric set	A set of labeled integer or float values	No
Bitset	A set of labeled bits	No
Location	Latitude and longitude	No
Health	Sent by the Formant agent to the reserved stream `$.agent.health`. Values are `unknown`, `operational`, `offline`, or `error`	No
Battery	Percentage, voltage, current, and charge	No
File	A raw file of any type	Yes
Image	A compressed image file bytestring (jpeg or png)	Yes
Video	A video file (mp4)	Yes
Point cloud	A raw point cloud	Yes
Localization	A unified localization datapoint with odometry, map, an array of point clouds, a path, and a goal	Yes
Json	A JSON-encoded string	*
Transform tree	A raw transform tree	Yes

* JSON data are datapoints if they are 1000 characters or shorter, and assets if they are longer than 1000 characters.

See the Formant agent's datapoint ingestion schema for a more detailed reference: https://github.com/FormantIO/formant/blob/826322aca53a6cc112346d171729f07e17e62e58/protos/model/v1/datapoint.proto#L16

Ingestion from a device

The process by which telemetry data goes from a device to the cloud is called ingestion.

The primary method of ingesting telemetry data is via the Formant agent. The Formant agent runs on each device in your organization and is responsible for delivering telemetry data to the cloud.

The agent has built-in mechanisms to add guarantees to the ingestion process. The most important ones to understand are buffering and throttling.

Mechanism #1: Buffering

The Formant agent maintains four ring buffers that persist to disk.

Each buffer stores in-flight datapoints that have been received by the Formant agent, but have yet to be delivered to the cloud. Datapoints from default, hardware, ROS, and API sources all flow through these buffers. The four buffers are:

Datapoint buffer
Asset buffer
Datapoint on-demand buffer
Asset on-demand buffer

We'll go through what each of those mean, but first, let's define what datapoints are defined as assets, and what on-demand streams are.

Assets vs Datapoints

Some data is too large to send around raw in batch ingestion requests. For these, we upload the data to storage and only send around the URL. These datapoints are considered assets.

The asset buffers store the following stream types:

File
Image
Video
Point cloud
Localization
Json
Transform tree

By contrast, the datapoint buffers store the remaining types:

Text
Numeric
Numeric set
Bitset
Location
Health
Battery

Refer to the table toward the beginning of this document for a reference with descriptions.

On-demand streams

Streams can be configured to be either "live" (default) or "on-demand."

Live streams will upload data when they can. If the device has an internet connection, and the Formant agent has datapoints for a live stream, the device will be attempting to upload these datapoints.

On-demand streams upload data from a device to the cloud on request. This request is usually done manually by a user of the Formant web application, but can also be done using an API call.

Buffer types

Let's define those four types of buffers now that we have full context:

Datapoint buffer : Non-asset datapoints for live streams
Asset buffer : Asset datapoints for live streams
Datapoint on-demand buffer : Non-asset datapoints for on-demand streams
Asset on-demand buffer : Asset datapoints for on-demand streams

Each buffer's size on disk is individually configurable. When a buffer overflows, data is overwritten by newer datapoints in a FIFO process. Datapoints inside a buffer are also uploaded to the cloud FIFO.

Mechanism #2: Throttling

The Formant agent throttles incoming datapoints at a configurable stream rate of up to 20Hz, with 5Hz being the default stream rate. What does this mean?

When the agent receives a new datapoint, it checks if at least 1/(stream rate) seconds have elapsed since the previous datapoint was successfully received on this stream. If not enough time has passed, the datapoint is throttled, which means dropped.

So, if datapoints are being sent to the agent at a rate of 10Hz, and the stream rate is 5Hz (the default), every other datapoint will be dropped.

Why throttle?

Throttling is an easy knob in Formant to control bandwidth allocation and reduce the number of datapoints being uploaded, without having to change anything about how your robot-side application works.