Intervention Requests
Intervention requests are the ability for a device or server to request some kind of user interaction to occur on a device. We have APIs that represent the list of intervention requests, creation of new intervention requests, and the appending of data to those requests.
See a video of an intervention request in which the Spot asks for help labeling a QR code:
View full API documentation here.
Getting a list of intervention requests
GET https://api.formant.io/v1/admin/intervention-requests
Creating a new intervention request
POST https://api.formant.io/v1/admin/intervention-requests
{
"message": "A teleop for a customer is requested",
"interventionType": "teleop",
"time": "2022-02-17T11:41:33.389-08:00",
"deviceId": "b306de84-33ca-4917-9218-f686730e24e0",
"tags": {},
"data": {
"instruction": "Look at the users item on the table"
}
}
Or, use the SDK:
from formant.sdk.agent.v1 import Client
client.create_selection_intervention_request(
self,
title, # type: str
instruction, # type: str
options, # type: List[str]
hint, # type: int
url=None, # type: str
content_type=DEFUALT_IMAGE_CONTENT_TYPE,
# type: Literal["image/jpg", "image/png"]
timestamp=None, # type: Optional[int]
severity=DEFAULT_SEVERITY_TYPE,
# type: Literal["info", "warning", "critical", "error"]
):
# type : (...) -> intervention_pb2.InterventionRequest
"""
Creates an intervention request based on type "selection".
Takes an image url, options and an integer with an optional
addition of instructions, and title.
:param title: The name of the intervention
:param instruction: The instructions detailing how to resolve the intervention
:param options: The list with options to select from
:param hint: The index of the suspected correct answer
:param url: The path to local file or valid remote URL for remote files
:param content_type: The format of the encoded image or frame.
Defaults to "image/jpg"
:param timestamp: Unix timestamp in milliseconds for the posted datapoint.
Uses the current time by default
:param severity: The severity level of the event
.. highlight:: python
.. code-block:: python
from formant.sdk.agent.v1 import Client
fclient = Client()
fclient.create_selection_intervention_request(
"Which fruit is best?",
"Select the best grape",
["fruit_1", "fruit_2", "fruit_3"],
hint=1,
url=/home/my_user/data/test-image.jpeg
severity=critical
)
"""
client.create_teleop_intervention_request(
self,
instruction, # type: str
timestamp=None, # type: Optional[int]
severity=DEFAULT_SEVERITY_TYPE,
# type: Literal["info", "warning", "critical", "error"]
):
# type : (...) -> intervention_pb2.InterventionRequest
"""
Creates an intervention request based on type "teleop".
requires an instruction.
:param instruction: The instructions detailing how to resolve the intervention
:param timestamp: Unix timestamp in milliseconds for the posted datapoint.
Uses the current time by default
:param severity: The severity level of the event
.. highlight:: python
.. code-block:: python
from formant.sdk.agent.v1 import Client
fclient = Client()
fclient.create_teleop_intervention_request(
"Get that gold",
)
"""
client.create_labeling_intervention_request(
self,
title, # type: str
instruction, # type: str
labels, # type: Dict[str, str]
hint, # type:List[intervention_pb2.LabeledPolygon]
url=None, # type: str
content_type=DEFUALT_IMAGE_CONTENT_TYPE,
# type: Literal["image/jpg", "image/png"]
timestamp=None, # type: Optional[int]
severity=DEFAULT_SEVERITY_TYPE,
# type: Literal["info", "warning", "critical", "error"]
):
# type : (...) -> intervention_pb2.InterventionRequest
"""
:param title: The name of the intervention
:param instruction: The instructions detailing how to resolve the intervention
:param labels: An Array of labels
:param hint: An array of label polygons, X and Y coordinates with a label
:param url: The path to local file or valid remote URL for remote files
:param content_type: The format of the encoded image or frame.
Defaults to "image/jpg"
:param timestamp: Unix timestamp in milliseconds for the posted datapoint.
Uses the current time by default
:param severity: The severity level of the event
Creates an intervention request based on type "labeling".
it takes an image url, an instructions, 'labels'
which are shown as:
Label = {
value = string;
string display_name = string;
}
and a hint which is an array of "LabeledPolygon",
that are shown as:
hint = {
List of vertex,
List of labels
}
and a vertex that appears as:
vertex = {
x = float,
y = float
}
"""
Adding response to intervention request
POST https://api.formant.io/v1/admin/intervention-responses
{
"interventionId": "518e24fc-64ef-47bb-be5e-036a97aeafaa",
"interventionType": "teleop",
"data": {
"state": "success",
"notes": "looks good!"
}
}
Updated over 1 year ago