Teleoperation is a major functionality in Formant which allows you to remotely operate, control, and observe your robot. You can configure realtime video streams, localization data, joysticks, and command buttons to create a custom operation interface.

Formant creates a teleoperation connection to your device via WebRTC, allowing you to connect from anywhere over an internet connection.

📘

WebRTC connections are encrypted.

This section will define teleoperation and provide step-by-step instructions to configure realtime data streams and interfaces to your robot. We will also describe the key differences between telemetry and realtime streams.

The following section is specific to web connection via WebRTC in the Formant web application. You can also connect to your device over a local network using the Data SDK, as seen in Connect over a local network

Connection requirements

Twilio connectivity

Run the following test on both the operator computer and on your robot: Twillio: Network test.

If all tests pass, a teleoperation connection can be established between Formant and your device.

On restricted networks, the TURN UDP Connectivity test may fail, which means that UDP traffic is being blocked. See the Allowed ports and domains section below for more details.

Bandwidth

Realtime teleoperation connections can often require considerable bandwidth, especially when handling rich data types such as high-quality video and point clouds. You'll want to make sure that both your operator computer and your robot have sufficient bandwidth to handle your teleoperation streams.

You can test the upload and download bandwidth of both your operator computer and your robot using speed test tools such as this one: GitHub: speedtest-cli.

The following table approximates the upload requirements of a single video stream by video quality:

👍

A good rule of thumb is that the robot and operator should have a consistent 3Mbps connection in both directions, plus whatever the additional upload requirements of your video streams and point clouds are.

Allowed ports and domains

For best performance, allow the following ports and domains:

Ports

• TCP: 443, 3478
• UDP: 443, 1024 - 65535

🚧

Avoiding relays via TURN

UDP traffic over these ports is outgoing only.

Your browser and operating system will dynamically establish a UDP connection on a port in this range. If ports on this range are blocked, your teleoperation connection will be relayed via TURN, which will degrade your connection quality and performance.

If your teleoperation connection is being relayed despite these ports being open, check your firewall or other network security settings.

Domains

• Allow *.twilio.com
• Allow *.formant.io

Choose a network interface

By default, any network interface may be used to make a WebRTC connection between your device and the Formant web application.

You can use the WEBRTC_INTERFACE environment variable to force WebRTC traffic over a specified network interface. This can be used to route WebRTC traffic around a VPN, or to keep traffic on a consistent interface if there are multiple internet links.

For more information, see Environment variables for the agent.

Next steps

Continue through the following pages under the Teleoperation header for step-by-step instructions to build a teleoperation interface.

See also

• Issues connecting to device via teleop
• Video performing poorly
• Images black, glitchy, out of order when ingesting pre-encoded video
• Connect over a local network

👋

If you notice an issue with this page or need help, please reach out to us! Use the 'Did this page help you?' buttons below, or get in contact with our Customer Success team via the Intercom messenger in the bottom-right corner of this page, or at [email protected].