Gst-nvdsanalytics

This plugin performs analytics on metadata attached by nvinfer (primary detector) and nvtracker. Analytics refer to Region of Interest (ROI) Filtering, Overcrowding Detection, Direction Detection and Line Crossing. The plugin operates in batch mode, it handles the context for each stream independently. ROI based analytics work on primary detector output, but Direction Detection and Line Crossing requires a tracker-id as previous history/state is required to do both the analytics. The configuration for the rules for analytics is achieved using a configuration file. For all the analytics calculations bottom center coordinate of bounding box of an object is being used. If bounding box is defined as (x_left, y_top, width, height) then the bottom center coordinate would be (x_left + width/2, y_top + height)

The gst-nvdsanalytics plugin extracts the metadata from the batched buffer sent by the upstream (nvtracker/nvinfer) element and passes it to the low-level nvdsanalytics library. The low-level library performs analytics based on the configured rules for each feature for each of the streams, the library returns the analytics output, which is for each tracked object as well for the complete frame. The analytics output (Frame level) is encapsulated in the NvDsAnalyticsFrameMeta structure and is added as user meta with meta_type set to NVDS_USER_FRAME_META_NVDSANALYTICS.The user meta is added to the frame_user_meta_list member of NvDsFrameMeta. The analytics output (per detected Object) is encapsulated in the NvDsAnalyticsObjInfo structure and is added as user meta with meta_type set to NVDS_USER_OBJ_META_NVDSANALYTICS. The user meta is added to the obj_user_meta_list member of NvDsObjectMeta.

The definition of NvDsAnalyticsObjInfo NvDsAnalyticsFrameMeta is available in sources/includes/nvds_analytics_meta.h header file. For guidance on how to access user metadata, see the User/Custom Metadata addition in the NvDsBatchMeta and Tensor Metadata.

Gst-nvdsanalytics

Inputs and Outputs

  • Inputs

    • GStreamer buffer containing NV12/RGBA frame(s) with attached metadata

  • Control Parameters

    • enable: True Enables the plugin, False sets it in pass-through mode

    • config-file: Configuration file which consist of the analytics rules

  • Output

    • Original NV12/RGBA batched buffer

    • NvDsAnalyticsFrameMeta (meta containing information about individual frames in the batched buffer)

    • NvDsAnalyticsObjInfo (meta containing information about individual objects in the individual frames of batched buffer)

Features

The following table summarizes the features of the plugin.

Gst-nvdsanalytics plugin features

Feature

Description

Release

ROI Filtering

This feature detects presence of objects in the ROI, the result is attached as metadata for every object, as well as total count of objects in ROI per frame.

DS 5.0

Overcrowding Detection

This feature detects presence of objects in the ROI, the result is attached as metadata for every object, as well as total count of objects in ROI per frame along with overcrowding status, i.e. no of objects in ROI exceed a pre-configured threshold.

DS 5.0

Direction Detection

Checks using history of position of object and the current frame information if it is following a preconfigured direction. The result is attached as metadata for every object.

DS 5.0

Line Crossing

This feature checks if object is following a preconfigured direction for the virtual line and if it has crossed the virtual line. The result is attached to object when Line crossing also frame meta where cumulative count along with per frame count of line crossing is attached.

DS 5.0

Gst Properties

The following table describes the Gst properties of the Gst-nvdsanalytics plugin.

Gst-nvdsanalytics plugin gst properties

Property

Meaning

Type and Range

Example Notes

unique-id

Unique identification unsigned integer value for element to identify the output attached by the element

Integer, 0 to 4,294,967,295

unique-id=17

enable

True Enables the plugin, False sets it in pass-through mode

Boolean

enable=1

config-file

Configuration file which consist of the analytics rules

String

config-file=config_nvdsanalytics.txt

Configuration File Parameters

The configuration file is located at sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt.

The Gst-nvdsanalytics configuration file uses a “Key File” format described in: https://specifications.freedesktop.org/desktop-entry-spec/latest

  • The [property] group configures the general behavior of the plugin. It is the only mandatory group.

  • The [roi-filtering-stream-<n>] group configures ROI filtering rules parameters for stream specified by <n>. For example, classes.[roi-filtering-stream-0] configures ROI filtering rules for stream ID 0.

  • The [overcrowding-<n>] group configures Overcrowding parameters for a stream specified by` <n>`.

  • The [direction-detection-<n>] group configures Direction Detection parameters for a stream specified by <n>.

  • The [line-crossing-<n>] group configures Line Crossing parameters for a stream specified by <n>.

The following table respectively describe the keys supported for [property] groups and [<feature>-stream-<n>] groups.

Gst-nvdsanalytics plugin config properties

Property

Applicable Groups

Meaning

Type and Range

Example Notes

enable

[property] | [<feature>-stream-<n>]

Common property to enable or disable the plugin or feature

Boolean

enable=1

config-width

[property]

Width for which the analytics rules are configured

Integer, >0

config-width=1280

config-height

[property]

Height for which the analytics rules are configured

Integer, >0

config-height=720

osd-mode

[property]

0: Disables display of any lines, ROIS and text 1: Display only lines, rois and static text i.e. labels 2: Display all info from 1 in addition, information for all the events

Integer0,1,2

osd-mode=2

display-font-size

[property]

Set OSD font size for the analytics data

Integer > 0

display-font-size=12

roi-<label>

[roi-filtering-stream-<n>] | [overcrowding-stream-<n>]

The ROI can be specified using roi key followed by its corresponding label (roi-<label>) the value consist of semi-colon delimited coordinates of the polygon (roi-<label>=x1;y1;x2;y2;x3;y3;x4;y4;x5;y5) . The user can specify a simple polygon with > 3 points. The points should be sequentially specified, as the relative position of points is required to check object in polygon.

String

roi-RF= 295;643;579;634;642;913;56;828 roi-OC= 295;643;579;634;642;913;56;828

inverse-roi

[roi-filtering-stream-<n>]

If the flag set true, it indicates the object outside the configured ROI be attached with the ROI label, and count the objects outside the ROI and send as frame level output

Boolean

inverse-roi=0

class-id

[<feature>-stream-<n>]

The class ID separated by semi-colon for which analytic rules should be applied. | -1 indicate that the rules are to be applied to all the classes

String

class-id=0;1;2 or class-id=-1

object-threshold

[overcrowding-stream-<n>]

The total count after which overcrowding flag should be indicated in the frame metadata.

Integer, >=0

object-threshold=10

direction-<label>

[direction-detection-stream-<n>]

The value indicates the direction (x1;y1;x2;y2)  which corresponds to the configured <label>. When object is moving in configured direction, the label of the particular rule is attached to the object as user metadata

String

direction-South=284;840;360;662

line-crossing-<label>

[line-crossing-stream-<n>]

The value indicates 2 coordinates of direction followed by 2 coordinates of virtual line. These coordinates are delimited using semi-colon (line-crossing-<label>=x1d;y1d;x2d;y2d;x1c;y1c;x2c;y2c). The direction indicates the direction of object in which it should be moving that can be considered for counting, so the object should be in direction x1d,y1d to x2d,y2d, which indicates a direction vector. The line crossing coordinates x1c,y1c and x2c,y2c indicate a virtual line which when an object following a configured direction crosses is then accounted for line crossing. <label> is string identification of the rule. Multiple line-crossings can be defined for a single stream

String

line-crossing-Exit=789;672;1084;900;851;773;1203;732

extended

[line-crossing-stream-<n>]

Boolean parameter to check crossings only on the configured line or for the extended line covering the video frame. If extended is 0, the line crossing only the configured line would be counted

Boolean

extended=0

mode

[line-crossing-stream-<n>]

Parameter to allow user configuration of adherence of object direction with configured direction for crossing. Possible options strict, balanced, loose. Operate as per name suggestion, where loose will just check if the object has crossed the line and would need the object just to be in same direction with very high tolerance

String
mode=loose
mode=strict
mode=balanced

Note

The config-width, config-height, indicate the resolution at which analytics rules have been configured. If the runtime stream resolution is different than the configuration resolution. The plugin handles the resolution change and scales the rules for the runtime resolution. The configuration rules are not present in absolute pixel values.

The following pipelines demonstrate plugin’s usage.

  • For dGPU:

    gst-launch-1.0 filesrc location=streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0  nvstreammux name=m width=1920 height=1080 batch-size=1 live-source=1 ! nvinfer config-file-path= sources/apps/sample_apps/

deepstream-nvdsanalytics-test/nvdsanalytics_pgie_config.txt ! nvtracker ll-lib-file = /opt/nvidia/deepstream/deepstream/lib/libnvds_nvdcf.so ll-config-file= tracker_config.yml tracker-width=640 tracker-height=384   ! nvdsanalytics config-file=

sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt ! nvmultistreamtiler  ! nvvideoconvert  ! nvdsosd ! nveglglessink

  • For Jetson:

    gst-launch-1.0 filesrc location=streams/sample_1080p_h264.mp4 ! qtdemux ! h264parse ! nvv4l2decoder ! m.sink_0 nvstreammux name=m width=1920 height=1080 batch-size=1 live-source=1 ! nvinfer config-file-path= sources/apps/sample_apps/

deepstream-nvdsanalytics-test/nvdsanalytics_pgie_config.txt ! nvtracker ll-lib-file = /opt/nvidia/deepstream/deepstream/lib/libnvds_nvdcf.so ll-config-file= tracker_config.yml tracker-width=640 tracker-height=384  ! nvdsanalytics config-file=

sources/apps/sample_apps/deepstream-nvdsanalytics-test/config_nvdsanalytics.txt ! nvmultistreamtiler  ! nvvideoconvert  ! nvdsosd ! nv3dsink