APIs & SDKs

Streaming image and video data

The post_image method can be used to send image or video data to the Formant Agent.

Posting a single image datapoint to telemetry

import cv2
from formant.sdk.agent.v1 import Client as FormantClient

if __name__ == "__main__":
    fclient = FormantClient()

    image = cv2.imread("/path/to/example.jpg")
    encoded = cv2.imencode(".jpg", image)[1].tostring()

    fclient.post_image("example.image", encoded)

By default, post_image accepts JPEGs. To send a PNG image, add the keyword argument content_type="image/png", e.g. fclient.post_image("example.image", encoded, content_type="image/png")

formant/send_image_datapoint.py at master · FormantIO/formant
github.comgithub.com
formant/send_image_datapoint.py at master · FormantIO/formant

Streaming video to telemetry or teleop

The Formant Agent can automatically encode high-frequency image streams to H264 video data. It can also accept a raw H264 stream directly.

🚧

Consider using a hardware video stream

Hardware streams can automate the process of reading and streaming USB and IP cameras.

See Add a video stream.

Streaming from a USB camera

import sys
import cv2
from formant.sdk.agent.v1 import Client as FormantClient

if __name__ == "__main__":
    fclient = FormantClient(ignore_throttled=True, ignore_unavailable=True)

    cap = cv2.VideoCapture(0)  # usb cam may be on a different index: try 1, 2 ...
    if cap is None:
        sys.exit()

    while True:
        _, image = cap.read()
        encoded = cv2.imencode(".jpg", image)[1].tostring()
        fclient.post_image("usb_cam", encoded)

When encoding is turned on in device telemetry or teleop configuration for the stream "usb_cam", the JPEG stream sent with post_image in this script will be automatically encoded into an H264 stream by the Formant Agent and streamed as telemetry and/or teleop data.

If you have direct access to a raw H264 stream and wish to ingest it, use content_type="video/h264".

formant/stream_usb_cam.py at master · FormantIO/formant
github.comgithub.com
formant/stream_usb_cam.py at master · FormantIO/formant

Configuration

The stream that is sent on can be configured in both telemetry and teleop configuration for a device.

Telemetry image API stream configuration

No configuration is required for images. Consider using video to preserve bandwidth unless the images are meant to be single frame shots.

Telemetry video API stream configuration

  1. Navigate to Settings->Devices
  2. Locate the name of the stream
  3. Make sure "enable encoding" is toggled on in the stream's configuration.
  4. Save

  1. Navigate to Settings->Streams
  2. Find the stream in the list of streams for your organization.
  3. Make sure the "type" dropdown is set to "video".
  4. Save

Teleop video API stream configuration

  1. Navigate to Settings->Devices
  2. Add a new stream under the "Image" section
  3. Enter the name of the stream
  4. Make sure "enable encoding" is toggled on
  5. Save

👋

If you notice an issue with this page or need help, please reach out to us! Use the 'Did this page help you?' buttons below, or get in contact with our Customer Success team via the Intercom messenger in the bottom-right corner of this page, or at [email protected].