Query datapoints in Analytics

Within the custom SQL editor in Formant Analytics, you can view and query datapoints and streams in your organization.

Viewing available data point parameters to query in the custom SQL editor.

Viewing available datapoint parameters to query in the custom SQL editor.

This guide will walk you through the table structure for stream data in Formant Analytics, and teach you how to query stream data by the available parameters.

Stream table structure explained

Data stream types

Data streams in Formant are categorized in the following way:

Table name
query_battery
query_bitset
query_health
query_json
query_numeric
query_numeric_set
query_text

Universal parameters

All data tables can be queried by the following parameters:

ParameterDescription
organization_idOrganization ID related to this datapoint.
device_idDevice ID related to this datapoint.
nameName of the stream associated with this datapoint.
tagsTags associated with this device or datapoint, used for filtering.
timeTimestamp associated with this datapoint.

The value parameter

Each table has a unique format for the value parameter, which represents the data structure of that stream's datapoints:

Table namevalue parameter format
query_batteryquery_battery does not have a value parameter, but instead has the following parameters:

- percentage: number between 0 and 100
- voltage: number
- charge: number
query_bitsetKey-value pair of the form: Map(String, Bool)
query_healthquery_health does not have a value parameter, but instead has a status parameter of type string.
query_jsonstring containing JSON data.
query_numericnumber
query_numeric_setArray of:

- label: string
- value: number
- unit: string Map(String,Tuple(value Float64, unit String))
query_textstring

📘

Parsing JSON data

When using the query_json table, the value parameter will be a JSON object. You can use the ClickHouse JSONExtract functions to navigate the JSON structure and extract specific values of various data types.

For example, if you want to query and plot an integer radius from your JSON object, you would use JSONExtractInt( value, "radius" ) to query only the radius column.

For more information, see ClickHouse: json-functions.

Example: Query a numeric stream

In the context of Formant Analytics, we can think of a stream's data as the value column of a table when the namecolumn is filtered for a given stream name.

For example, consider the following query_numeric table:

organization_iddevice_idnametagstimevalue
org adevice 1berry_radiusberry-stats12:00 PM12.2
org adevice 1berry_diameterberry-stats12:00 PM28.8
org adevice 1pick_durationberry-stats12:01 PM2.2
org adevice 1pick_durationberry-stats12:02 PM1.7
org adevice 1berry_radiusberry-stats12:02 PM14.5
org adevice 1berry_diameterberry-stats12:03 PM25.9
org adevice 1pick_durationberry-stats12:03 PM5.3
org adevice 1berry_radiusberry-stats12:05 PM10.7

The query_numeric table contains all stream data for numeric streams in org a. To query the data for the stream berry_radius, we filter query_numeric by name berry_radius for the following result:

organization_iddevice_idnametagstimevalue
org adevice 1berry_radiusberry-stats12:00 PM12.2
org adevice 1berry_radiusberry-stats12:02 PM14.5
org adevice 1berry_radiusberry-stats12:05 PM10.7

The above structure for stream names applies for all tables with a name and value field.

Example: Query by tags

All datapoint tables have a tags column, so we can query by tags.

For example, suppose we want to track the amount of time lost due to mechanical problems with a robot. We already have a tag key called Time Lost, and the value for that tag is the number of minutes lost to a mechanical issue. This value is assigned within an annotation.

A very simple query might look like this:

SELECT
   device_id AS device,
   tags ['Time Lost']
FROM
   query_annotation
WHERE
   query_annotation.message = 'Task completion delayed due to mechanical issue.'
Querying annotations by the tag key `Time Lost`.

Querying annotations by the tag key Time Lost.

In a more advanced case, you might want to sum Time Lost by device ID. We'll convert the values for Time Lost into unsigned integers, and then group the result by device:

SELECT
   device_id AS device,
   sum(
      CAST(
         SUBSTRING_INDEX(tags ['Time Lost'], ' ', 1) AS UNSIGNED
      )
   ) AS time_lost
FROM
   query_annotation
WHERE
   query_annotation.message = 'Task completion delayed due to mechanical issue.'
GROUP BY
   device;
Summing `Time Lost` by device ID.

Summing Time Lost by device ID.

See also

👋

If you notice an issue with this page or need help, please reach out to us! Use the 'Did this page help you?' buttons below, or get in contact with our Customer Success team via the Intercom messenger in the bottom-right corner of this page, or at [email protected].