The Formant agent can encode incoming image streams to H264. This guide will walk you through the default settings of the Formant agent encoder, and describe the configuration options available.

GStreamer encoder settings

Formant uses GStreamer to encode images to a video stream. Common configuration options, such as bitrate and desired quality, can be configured in the Formant web application when configuring an image stream, as seen in Add a video stream and Add a teleoperation image stream.

Advanced configuration options and custom encoder settings can be set using environment variables, as seen in Environment variables for the agent.

Default behavior: lowest latency

The Formant agent prioritizes low-latency video encoding by default. This uses the fastest encoding setting available, so that frames spend the least amount of time being processed by the encoder and are passed to the front end as quickly as possible. In this configuration, the encoder drops packets when they arrive out of order, and uses the adaptive bitrate setting to maintain frame rate.

📘

Telemetry vs. real-time streams

Telemetry streams are ingested to the Formant cloud before being displayed in the Formant web application. A delay of up to several seconds is normal for telemetry streams.

Real-time streams are transmitted directly from your device to the Formant web application via a peer-to-peer WebRTC connection. These streams have extremely low latency, but are not ingested to the Formant cloud and cannot be reviewed later.

It is possible to configure a video stream to send real-time data directly to a teleoperation UI, while also ingesting that video stream to the Formant cloud as telemetry data for review.

For more information, see Telemetry vs. realtime data streams.

Increasing video quality

If the default encoder settings do not provide video quality that meets the needs of your application, you can increase the video quality via the following settings.

📘

Note that increasing video quality may cause your data consumption and throughput to increase considerably.

Increase bitrate

Increasing the bitrate will give the encoder greater bit depth to encode video and will result in higher quality.

Reduce encoding speed

By default, Formant uses the "ultrafast" encoding speed in GStreamer. This is the fastest setting, but comes at the cost of video quality. Slower encoding speeds will output higher quality video at the expense of increased CPU and GPU usage.

Try setting the encoding speed to “medium” and adjust from there to balance video quality, throughput, and computation resources.

📘

Encoder speed is a global setting and cannot be set for individual streams.

Increase encoder buffer size

By increasing the size of the video buffering verifier (VBV), you will allow the encoder to have larger spikes in bandwidth when needed. This will have the largest impact if you have a mixture of static scenes and dynamic scenes.

Disable adaptive bitrate

The Formant agent has an adaptive bitrate setting which, if enabled, allows the encoder to degrade the quality of video to maintain frame rate.

By disabling adaptive bitrate, you can force the encoder to maintain the designated bitrate. However, if network conditions worsen, you may see choppiness, glitchy video, and/or a drop in frame rate.

Manual encoder configuration

You can configure many environment variables to customize the behavior of the Formant encoder. For more information, see Environment variables for the agent .

See also

• Environment variables for the agent
• Add a video stream
• Add a teleoperation image stream
• Agent SDK: Streaming image and video data
• The Formant agent

👋

If you notice an issue with this page or need help, please reach out to us! Use the 'Did this page help you?' buttons below, or get in contact with our Customer Success team via the Intercom messenger in the bottom-right corner of this page, or at [email protected].