Handle disconnections safely

In mission critical teleop applications, it is necessary to detect and react to session disconnections within milliseconds. This can be achieved by listening to a high frequency streaming heartbeat that is continually sent from the operator to device during teleop. This heartbeat is communicated over SCTP and requires less than 0.004Mbps. The robot application can listen to the teleop’s heartbeat and handle disconnects immediately, or time out when the teleop heartbeat stalls.

Monitor heartbeats and safe shutdown using the Agent SDK

Using the Formant Agent SDK, you can run an application on your device which monitors heartbeat messages from the Formant UI and responds to lags or disconnections in a teleoperation session. You can also distinguish between session types connected to your device from the Formant UI.

register_teleop_heartbeat_callback() will receive a heartbeat continuously from the frontend, every 50ms from each connection in perfect network. This will allow you to detects disconnect with logic like 'Have I not received a heartbeat in the last X ms?' (e.g. teleop_heartbeat_callback.py )

You won't miss a disconnection using this method, because the heartbeat is sent from the frontend, however you may need to tune the sensitivity to ensure you're not accidentally calling something a disconnection when it's actually just a small lag-spike.

get_teleop_info will let you know how many sessions are currently connected, and uses more complicated logic internal to the formant agent for detecting disconnections (intentional disconnections are instant and unintentional make take longer while we try to re-establish connection)

This method is easier to implement, and you can be sure the connection is indeed disconnected, it just may take longer (few seconds in the worst case)

For more information, including a code example, see Receiving teleop heartbeats.

👋

If you notice an issue with this page or need help, please reach out to us! Use the 'Did this page help you?' buttons below, or get in contact with our Customer Success team via the Intercom messenger in the bottom-right corner of this page, or at [email protected].