The post_image method can be used to send image or video data to the Formant Agent.

Posting a single image datapoint to telemetry

import cv2
from formant.sdk.agent.v1 import Client as FormantClient

if __name__ == "__main__":
    fclient = FormantClient()

    image = cv2.imread("/path/to/example.jpg")
    encoded = cv2.imencode(".jpg", image)[1].tostring()

    fclient.post_image("example.image", encoded)

By default, post_image accepts JPEGs. To send a PNG image, add the keyword argument content_type="image/png", e.g. fclient.post_image("example.image", encoded, content_type="image/png")

formant/send_image_datapoint.py at master · FormantIO/formant

github.com

formant/send_image_datapoint.py at master · FormantIO/formant

Streaming video to telemetry or teleop

The Formant Agent can automatically encode high-frequency image streams to H264 video data. It can also accept a raw H264 stream directly.

🚧
Consider using a hardware video stream
Hardware streams can automate the process of reading and streaming USB and IP cameras.
See Add a video stream.

Streaming from a USB camera

import sys
import cv2
from formant.sdk.agent.v1 import Client as FormantClient

if __name__ == "__main__":
    fclient = FormantClient(ignore_throttled=True, ignore_unavailable=True)

    cap = cv2.VideoCapture(0)  # usb cam may be on a different index: try 1, 2 ...
    if cap is None:
        sys.exit()

    while True:
        _, image = cap.read()
        encoded = cv2.imencode(".jpg", image)[1].tostring()
        fclient.post_image("usb_cam", encoded)

When encoding is turned on in device telemetry or teleop configuration for the stream "usb_cam", the JPEG stream sent with post_image in this script will be automatically encoded into an H264 stream by the Formant Agent and streamed as telemetry and/or teleop data.

If you have direct access to a raw H264 stream and wish to ingest it, use content_type="video/h264".

formant/stream_usb_cam.py at master · FormantIO/formant

github.com

formant/stream_usb_cam.py at master · FormantIO/formant

Configuration

The stream that is sent on can be configured in both telemetry and teleop configuration for a device.

Telemetry image API stream configuration

No configuration is required for images. Consider using video to preserve bandwidth unless the images are meant to be single frame shots.

Telemetry video API stream configuration

Navigate to Settings->Devices
Locate the name of the stream
Make sure "enable encoding" is toggled on in the stream's configuration.
Save

Navigate to Settings->Streams
Find the stream in the list of streams for your organization.
Make sure the "type" dropdown is set to "video".
Save

Teleop video API stream configuration

Navigate to Settings->Devices
Add a new stream under the "Image" section
Enter the name of the stream
Make sure "enable encoding" is toggled on
Save