Issue details

• Image resolution is grainy or pixelated: Often related to how the video is configured.
• Latency makes the video lag behind real-life circumstance: Often network or CPU related.
• Artifacting, or pixels blending between frames (frames out of order or lost): Often network related.
• Low frames per second (FPS)

Troubleshooting steps

Check the network topology

📘

For more information on diagnosing network issues during teleoperation, see Monitor network statistics during teleoperation.

Are you using tailscale or some other VPN? If so, set the WEBRTC_INTERFACE environment variable in /var/lib/.bashrc to use the network interface used to connect to the public internet.

Formant optimizes latency over quality

What is the intended quality of the video? The intended bitrates for each resolution are shown below. Poor network quality may cause the agent to reduce bitrate to reduce latency. You can also set the intended bitrate higher than 2048 by using the FORMANT_ENCODING_BITRATE environment variable.

Agent logs may provide context helpful for debugging

Check the agent log; this may provide context pointing to one of the other issues on this page.

• Is there something suspicious about the video encoding logs?
• Gstreamer pipeline string is not what you expect (correct interface, quality, etc)
• If it is not using accelerated encoding, but you’re on a jetson, check /var/lib/formant for FORMANT_ACCELERATED_VIDEO_ENCODING flag and remove it
• If you’re on a jetson you may need to force accelerated encoding off (envvar mentioned above).
• Encoder crashing or restarting

Adaptive Bitrate

When you have adaptive bitrate on, packet loss will cause the Formant Agent to crush the bitrate until there is no packet loss. Adaptive bitrate is enabled by default. To disable adaptive bitrate:

export FORMANT_ENABLE_ADAPTIVE_BITRATE=false

Source Video Quality

Is the source video good quality? If the source only has low quality video, Formant will pass on the low quality video.

Low frames per second (FPS)

To address the low FPS issue, it’s crucial to understand how the video is being captured. For USB cameras, running the following command could provide valuable information to optimize settings:

v4l2-ctl --device /dev/videoX --list-formats-ext

If you're using an IP camera, adjusting the camera's settings directly could lead to improvements. The method of connection and configuration can significantly impact the FPS, so these steps are worth exploring.

Teleop session type

Is the teleop session occurring via TURN (would be displayed in bottom left corner)? This does not guarantee that performance will be poor, but can be a cause. All data is being relayed rather than transiting P2P. This is more common with some firewalls / routers; specifically: double or more NAT, ISP implemented CGNAT, or lack of UDP connectivity (either the robot side, client side, or both).

Ping/Packet Loss

What is the ping/packet loss? High ping/packet loss network would make for a poor video experience. Try a tool like Packet Loss Test – Test Your Connection Quality . Note that even a few percent of packet loss can cause significant issues.

WebRTC internals

Check chrome://webrtc-internals to check if there are other issues with the network. Make sure the proper session is selected in teal at the top. Every WebRTC connection has a separate “tab” you can click on.
Check round trip latency for the selected (bold) candidate pair; there are numerous graphs that can be utilized to deduce network issues.

Telemetry Stream

Try configuring the same stream for telemetry; when you see poor performance over teleop do you also see poor performance in the telemetry stream?

Is the device’s CPU super high? This can affect the performance of the video, especially latency.

Reduce Quality

If possible, turn down the video quality. It may be that the network can’t handle the data; if nothing else it is a good datapoint to test this.

A speed test should be executed on the robot side to confirm the WAN speed. It is common to see performance issues when a robot enters a cellular dead spot.

IP Camera

Is the video camera an IP camera that does not have h264b (h264 with baseline or constrained-baseline profile)?
You may need to use constant quality instead of constant bitrate.

See also

• Add a video stream
• Add a teleoperation image stream
• Images black, glitchy, out of order when ingesting pre-encoded video
• Difference in video quality between telemetry and teleoperation

👋

If you notice an issue with this page or need help, please reach out to us! Use the 'Did this page help you?' buttons below, or get in contact with our Customer Success team via the Intercom messenger in the bottom-right corner of this page, or at [email protected].