Intervention Requests

Intervention requests are the ability for a device or server to request some kind of user interaction to occur on a device. We have APIs that represent the list of intervention requests, creation of new intervention requests, and the appending of data to those requests.

See a video of an intervention request in which the Spot asks for help labeling a QR code:

View full API documentation here.

Getting a list of intervention requests

GET https://api.formant.io/v1/admin/intervention-requests

Creating a new intervention request

POST https://api.formant.io/v1/admin/intervention-requests

{
    "message": "A teleop for a customer is requested",
    "interventionType": "teleop",
    "time": "2022-02-17T11:41:33.389-08:00",
    "deviceId": "b306de84-33ca-4917-9218-f686730e24e0",
    "tags": {},
    "data": {
        "instruction": "Look at the users item on the table"
    }
}

Or, use the SDK:

from formant.sdk.agent.v1 import Client

client.create_selection_intervention_request(
        self,
        title,  # type: str
        instruction,  # type: str
        options,  # type: List[str]
        hint,  # type: int
        url=None,  # type: str
        content_type=DEFUALT_IMAGE_CONTENT_TYPE,
        # type: Literal["image/jpg", "image/png"]
        timestamp=None,  # type: Optional[int]
        severity=DEFAULT_SEVERITY_TYPE,
        # type: Literal["info", "warning", "critical", "error"]
    ):
        # type : (...) -> intervention_pb2.InterventionRequest
        """
        Creates an intervention request based on type "selection".
        Takes an image url, options and an integer with an optional
        addition of instructions, and title.
        :param title: The name of the intervention
        :param instruction: The instructions detailing how to resolve the intervention
        :param options: The list with options to select from
        :param hint: The index of the suspected correct answer
        :param url: The path to local file or valid remote URL for remote files
        :param content_type: The format of the encoded image or frame.
            Defaults to "image/jpg"
        :param timestamp: Unix timestamp in milliseconds for the posted datapoint.
            Uses the current time by default
        :param severity: The severity level of the event
        .. highlight:: python
        .. code-block:: python
            from formant.sdk.agent.v1 import Client
            fclient = Client()
            fclient.create_selection_intervention_request(
                "Which fruit is best?",
                "Select the best grape",
                ["fruit_1", "fruit_2", "fruit_3"],
                hint=1,
                url=/home/my_user/data/test-image.jpeg
                severity=critical
            )
        """
        
client.create_teleop_intervention_request(
        self,
        instruction,  # type: str
        timestamp=None,  # type: Optional[int]
        severity=DEFAULT_SEVERITY_TYPE,
        # type: Literal["info", "warning", "critical", "error"]
    ):
        # type : (...) -> intervention_pb2.InterventionRequest

        """
        Creates an intervention request based on type "teleop".
        requires an instruction.
        :param instruction: The instructions detailing how to resolve the intervention
        :param timestamp: Unix timestamp in milliseconds for the posted datapoint.
            Uses the current time by default
        :param severity: The severity level of the event
        .. highlight:: python
        .. code-block:: python
            from formant.sdk.agent.v1 import Client
            fclient = Client()
            fclient.create_teleop_intervention_request(
            "Get that gold",
            )
        """
      
 client.create_labeling_intervention_request(
        self,
        title,  # type: str
        instruction,  # type: str
        labels,  # type: Dict[str, str]
        hint,  # type:List[intervention_pb2.LabeledPolygon]
        url=None,  # type: str
        content_type=DEFUALT_IMAGE_CONTENT_TYPE,
        # type: Literal["image/jpg", "image/png"]
        timestamp=None,  # type: Optional[int]
        severity=DEFAULT_SEVERITY_TYPE,
        # type: Literal["info", "warning", "critical", "error"]
    ):
        # type : (...) -> intervention_pb2.InterventionRequest

        """
        :param title: The name of the intervention
        :param instruction: The instructions detailing how to resolve the intervention
        :param labels: An Array of labels
        :param hint: An array of label polygons, X and Y coordinates with a label
        :param url: The path to local file or valid remote URL for remote files
        :param content_type: The format of the encoded image or frame.
            Defaults to "image/jpg"
        :param timestamp: Unix timestamp in milliseconds for the posted datapoint.
            Uses the current time by default
        :param severity: The severity level of the event
        Creates an intervention request based on type "labeling".
        it takes an image url, an instructions, 'labels'
        which are shown as:
                Label = {
                        value = string;
                        string display_name = string;
                        }
        and a hint which is an array of "LabeledPolygon",
        that are shown as:
                hint = {
                    List of vertex,
                    List of labels
                }
        and a vertex that appears as:
                vertex = {
                    x = float,
                    y = float
                }
        """

Adding response to intervention request

POST https://api.formant.io/v1/admin/intervention-responses

{
    "interventionId": "518e24fc-64ef-47bb-be5e-036a97aeafaa",
    "interventionType": "teleop",
    "data": {
        "state": "success",
        "notes": "looks good!"
    }
}