--- updatedAt: 2025-11-11T01:48:13.000Z agentTools: projectIndex: https://docs.formant.io/llms.txt --- # Add a teleoperation image stream This guide will teach you how to add image and video streams to your teleoperation interface in Formant. You can add image or video streams via ROS, a direct hardware connection, or the Formant Agent SDK. # Step 1: Navigate to device teleoperation settings 1. In Formant, in the upper-left corner, open the menu and click **Settings**. 2. Click **Devices**, and then click on the device you want to configure. 3. Click on the **Teleoperation** tab to open the teleoperation configuration page. # Step 2: Configure an image stream ## ROS image stream Formant supports two ROS data types: `sensor_msgs/Image` and `sensor_msgs/CompressedImage`. To save bandwidth,`Image` topics are encoded to `H.264` video. `CompressedImage` topics can be transmitted either frame-by-frame as images, or encoded to `H.264`. 1. In the teleoperation configuration page, scroll down to **Image** and click the **plus (+) icon**. 2. Formant will populate the Image configuration window with ROS topics found on your device which have a data type of `sensor_msgs/Image` or `sensor_msgs/CompressedImage`: Click one of these topics to add it as a video stream. You can click the **pencil icon** to configure your video stream further. If you don't see the stream you want: 3. Click **Add ROS topic**. 4. Configure the ROS topic: | Property | Usage | | :---------------------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- | | Topic name | Enter the name of the image ROS topic you want to add to your teleoperation view. | | Topic type | Provide the appropriate image topic type: `sensor_msgs/Image` or `sensor_msgs/CompressedImage`. | | ROS version | The ROS version of the node publishing the topic. | | Enable encoding (`CompressedImage` topics only) | If set to **YES**, Formant will encode the incoming image stream to `H.264`. `Image` streams are encoded to `H.264` by default. | | Video quality (encoded streams only) | Choose the resolution of the encoded video stream. Higher video quality will use more data. | | Bitrate | Desired bitrate of your video stream. If left blank, the agent will automatically determine the ideal bitrate for the selected video quality. | | Disable adaptive quality | If disabled, Formant will keep the video quality consistent throughout poor network conditions. | | Clock Overlay | If set to **YES**, Formant will overlay a clock in the lower-right corner of your video stream. | 5. Click **Done**, and proceed to Step 3. ## Camera stream The Formant agent can transmit an image stream directly from a camera. Formant supports V4L2 devices \[USB and Camera Serial Interface (CSI)], and IP cameras. ### USB and CSI 1. In the teleoperation configuration page, scroll down to **Image** and click the **plus (+) icon**. 2. Click **Add from hardware**. This will open the camera configuration window: 3. Configure your camera stream: | Property | Usage | | :-------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- | | Hardware type | Set this to **usb**. | | Hardware path to USB camera | Enter the hardware path to your camera. This field should auto-populate with detected USB devices. | | Video quality | Choose the resolution of the encoded video stream. Higher video quality will use more data. | | Bitrate | Desired bitrate of your video stream. If left blank, the agent will automatically determine the ideal bitrate for the selected video quality. | | Disable adaptive quality | If disabled, Formant will keep the video quality consistent throughout poor network conditions. | | Clock Overlay | If set to **YES**, Formant will overlay a clock in the lower-right corner of your video stream. | When using a USB or CSI camera, Format will attach to the camera stream with the closest dimensions to the preferred video quality specified above. The frame rate is controlled by the environment variable `FORMANT_VIDEO_FRAMERATE`, which has a default value of `24`. You can see the native video formats of your camera with the command `v4l2-ctl`. Here's an example: ``` $ v4l2-ctl --list-formats-ext ioctl: VIDIOC_ENUM_FMT Index : 0 Type : Video Capture Pixel Format: 'YUYV' Name : YUYV 4:2:2 Size: Discrete 640x480 Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.042s (24.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.133s (7.500 fps) Interval: Discrete 0.200s (5.000 fps) Index : 1 Type : Video Capture Pixel Format: 'MJPG' (compressed) Name : Motion-JPEG Size: Discrete 640x480 Interval: Discrete 0.017s (60.000 fps) Interval: Discrete 0.033s (30.000 fps) Interval: Discrete 0.042s (24.000 fps) Interval: Discrete 0.050s (20.000 fps) Interval: Discrete 0.067s (15.000 fps) Interval: Discrete 0.100s (10.000 fps) Interval: Discrete 0.133s (7.500 fps) Interval: Discrete 0.200s (5.000 fps) ``` Formant will avoid attaching to the `MJPG` stream unless you set environment variable `FORMANT_PREFER_MJPG` to `true`. 4. Click **Done**, and proceed to Step 3. ### IP cameras Formant supports IP cameras via the Real-Time Streaming Protocol (RTSP), Real-Time Messaging Protocol (RTMP), and UDP, as well as the ONVIF protocol. 1. In the teleoperation configuration page, scroll down to **Image** and click the **plus (+) icon**. 2. Click **Add from hardware**. This will open the camera configuration window: 3. Configure your camera stream:
Property Usage
Hardware type Set this to **ip**.
RTSP URI of IP camera Enter the URI of your camera in the following format:\ **RTSP:**\ `rtsp://:/H264`\ **RTMP:**\ `rtmp://:/steam/live`
Encoding required Set this to **YES** if your camera outputs a format other than `H.264`. Formant requires incoming `H.264` video to be encoded with the `baseline` or `baseline constrained` profile. If Formant detects a different profile, or a non-supported set of encoding parameters, it will try to re-encode the video. If this fails, you must enable this property. Proper encoding of your video prior to ingestion by the Formant agent (allowing you to set this property to **NO**) will save computation resources.
Supports ONVIF Set this to **YES** if your camera supports the ONVIF protocol. Formant will be able to connect to your camera automatically. If you specify a fully qualified address in **RTSP URI of IP camera**, this property will be ignored, as its functionality will already be fulfilled.
Username Enter the username to access your camera.
Password Enter the password to access your camera.
Bitrate Desired bitrate of your video stream. If left blank, the agent will automatically determine the ideal bitrate for the selected video quality. * \*Note:\*\* If your IP camera is encoded with the `baseline constrained` profile, Formant does not encode the video and passes it directly from your camera to the UI. In this case, the Bitrate property in Formant will be disabled, and you will not be able to modify your camera's bitrate in Formant.
Disable adaptive quality If disabled, Formant will keep the video quality consistent throughout poor network conditions.
Clock Overlay If set to **YES**, Formant will overlay a clock in the lower-right corner of your video stream.
4. Click **Done**, and proceed to Step 3. ## API stream You can use the [Formant Agent SDK](https://docs.formant.io/reference/agent-sdk-installation-and-overview) to capture images and video on your device and ingest it with the Formant agent. You can post images and videos, and control their encoding, using the `post_image()` function. For more information and examples on ingesting images and video using the Agent SDK, see [Streaming image and video data](https://docs.formant.io/reference/sending-image-datapoints). For a complete reference of the Agent SDK, see [Agent SDK Reference](https://docs.formant.io/reference/complete-reference). Once you have written an application which captures images or video on your device: 1. In the teleoperation configuration page, scroll down to **Image** and click the **plus (+) icon**. 2. Click **Add from API**. This will open the API stream configuration window: 3. Configure your API stream: | Property | Usage | | :----------------------------------- | :-------------------------------------------------------------------------------------------------------------------------------------------- | | Image stream name | Enter the name of the stream to which you are posting image or video data (`stream` parameter in `post_image()` function). | | Enable encoding | If set to **YES**, Formant will encode the incoming image stream to `H.264`. | | Video quality (encoded streams only) | Choose the resolution of the encoded video stream. Higher video quality will use more data. | | Bitrate | Desired bitrate of your video stream. If left blank, the agent will automatically determine the ideal bitrate for the selected video quality. | | Disable adaptive quality | If disabled, Formant will keep the video quality consistent throughout poor network conditions. | | Clock Overlay | If set to **YES**, Formant will overlay a clock in the lower-right corner of your video stream. | 4. Click **Done**, and proceed to Step 3. # Step 3 (optional): Further camera configuration via environment variables In addition to the configuration in Step 2, you can use Formant environment variables to control a wide range of camera settings. For a full list of environment variables available in Formant, see [Environment variables for the agent](https://docs.formant.io/docs/environment-variables-for-the-agent#media-environment-variables). Once you've configured your environment variables, proceed to Step 4. # Step 4: Add an image stream to your teleoperation view 1. Scroll to the top of the teleoperation configuration window. Video streams which have been added under the **Image** section can be added to the **Primary View** or **Sidebar Views** by clicking the dropdown arrow: 2. Click **Save** to confirm your teleoperation settings. # Step 5: Verify your work in teleoperation mode 1. In your device view, in the upper-right corner, click the **three dots >> Teleop**, or enter **Shift + T**. 2. Verify that your video stream is working properly: # See also * [Telemetry: Add a video stream](https://docs.formant.io/docs/getting-started-add-a-video-stream) * [Difference in video quality between telemetry and teleoperation](https://docs.formant.io/docs/faq-difference-in-video-quality-between-telemetry-and-teleoperation) * [Video performing poorly](https://docs.formant.io/docs/troubleshooting-video-performing-poorly) * [RTSP camera not working](https://docs.formant.io/docs/troubleshooting-rtsp-camera-not-working) ### If you notice an issue with this page or need help, please reach out to us! Use the 'Did this page help you?' buttons below, or get in contact with our Customer Success team via the Intercom messenger in the bottom-right corner of this page, or at [support@formant.io](mailto:support@formant.io).