Reference graphs

This section provides details about the sample graphs for the DeepStream extensions. Most of these sample graphs are equivalents of the sample apps released as part of the DeepStreamSDK and demonstrate how to port/convert various portions of the “C/C++” based DeepStream applications into graphs and custom components/extensions.

Installing the reference graphs

Download reference graphs:

https://catalog.ngc.nvidia.com/orgs/nvidia/resources/gxf_and_gc

Install reference graphs:

sudo dpkg -i deepstream-reference-graphs-6.3.deb

Graphs are installed to:

/opt/nvidia/deepstream/deepstream/reference_graphs

deepstream-test1

Simplest example of using DeepStream for object detection. Demonstrates decoding video from a file, performing object detection and overlaying bounding boxes on the frames.

Graph Files

deepstream-test1.yaml – The main graph file
parameters.yaml – File containing parameters for the various components in the graph
README - Contains detailed graph description and execution instructions
ds_test1_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_test1_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-test1

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-test1.yaml \
      parameters.yaml -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-test1.yaml \
      parameters.yaml -d ../common/target_aarch64.yaml

Graph

Sample Output

deepstream-test2

Builds on top of deepstream-test1 and demonstrates object tracking and cascaded inferencing.

Graph Files

deepstream-test2.yaml – The main graph file
parameters.yaml – File containing parameters for the various components in the graph
README - Contains detailed graph description and execution instructions
ds_test2_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_test2_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-test2

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-test2.yaml \
      parameters.yaml -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-test2.yaml \
      parameters.yaml -d ../common/target_aarch64.yaml

Graph

Sample Output

deepstream-test3

Builds on top of deepstream-test1 to demonstrate use of multiple sources in the pipeline.

Graph Files

deepstream-test3.yaml – The main graph file
parameters.yaml – File containing parameters for the various components in the graph
README - Contains detailed graph description and execution instructions
ds_test3_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_test3_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-test3

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-test3.yaml \
      parameters.yaml -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-test3.yaml \
      parameters.yaml -d ../common/target_aarch64.yaml

Graph

Sample Output

deepstream-test4

Builds on top of deepstream-test1 to demonstrate how to send the metadata generated by the DeepStream components to the cloud using messaging components.

Graph Files

deepstream-test4.yaml – The main graph file
parameters.yaml – File containing parameters for the various components in the graph
README - Contains detailed graph description and execution instructions
ds_test4_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_test4_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-test4

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-test4.yaml \
      parameters.yaml -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-test4.yaml \
      parameters.yaml -d ../common/target_aarch64.yaml

Note

A small note on what minimum parameter changes are needed to run the graph on any system

Note

By default the sample is configured to send messages over Kafka and thus needs a Kafka broker running. The graph files assume that the server is running on “localhost:9092”. The server to send messages to along with the topic can be changed by modifying msg-broker-conn-str and topic parameters in the parameters.yaml file.

Graph

Sample Output

Following is the snapshot from output video. However, the app also sends messages over Kafka which can be viewed using the console consumer utility in the Kafka package or a similar alternative.

deepstream-test5

DeepStream reference application which demonstrates device-to-cloud and cloud-to-device messaging, Smart Record and model on-the-fly update.

Graph

Sample Output

deepstream-runtime-src-add-del

Demonstrates how sources can be dynamically added/removed at runtime. Also, demonstrates the use of action/signal components. The “NvDsSourceManipulationAction” of “NvDsMultiSrcInput” component is used to add/remove the sources. This action is triggered by another sample component “NvDsSampleSourceManipulator” every fixed interval which can be configured. Or this action can also be triggered by an Http service “NvDsStreamManager” upon add source or remove source Http request.

Graph Files

deepstream-runtime-src-add-del.yaml – The main graph file
deepstream-runtime-src-add-del-as-a-service.yaml – The main graph file with stream manager service
parameters.yaml – File containing parameters for the various components in the graph
README - Contains detailed graph description and execution instructions
ds_runtime_src_add_del_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_runtime_src_add_del_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-runtime-src-add-del

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-runtime-src-add-del.yaml \
      parameters.yaml -d ../common/target_x86_64.yaml
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-runtime-src-add-del-as-a-service.yaml \
      -d ../common/target_x86_64.yaml
* To add / remove a stream using a client on the same host
   Add stream with id 1:
   $ curl -X POST "http://localhost:8082/AddStream/stream" -d "{\"sensor\":{\"id\": \"1\", \"uri\": \"file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4\"}}"
   Add stream with id 2:
   $ curl -X POST "http://localhost:8082/AddStream/stream" -d "{\"sensor\":{\"id\": \"2\", \"uri\": \"file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h265.mp4\"}}"
   ......
   Until the max number of streams configured in NvDsStreamManager is reached

   Remove stream with id 2:
   $ curl -X POST "http://localhost:8082/RemoveStream/stream" -d "{\"sensor\":{\"id\": \"2\"}}"
   Remove stream with id 1:
   $ curl -X POST "http://localhost:8082/RemoveStream/stream" -d "{\"sensor\":{\"id\": \"1\"}}"
   ......
   Until all streams are removed

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-runtime-src-add-del.yaml \
      parameters.yaml -d ../common/target_aarch64.yaml

Graph

Sample Output

deepstream-template-plugin

Demonstrates the usage of configuration components used as configuration providers for other components. These graphs are meant to run only on DGPU platforms since the DeepStream template plugins are available only on DGPU platforms.

Graph Files

deepstream-videotemplate.yaml – Graph demonstrating usage of DS video template plugin and a configuration provider for the plugin
deepstream-audiotemplate.yaml – Graph demonstrating usage of DS audio template plugin and a configuration provider for the plugin
ds_audiotemplate_plugin_container_builder_dgpu.yaml, ds_videotemplate_plugin_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_audiotemplate_plugin_container_builder_jetson.yaml, ds_audiotemplate_plugin_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform
README - Contains detailed graph description and execution instructions

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-template-plugin

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-audiotemplate.yaml \
      -d ../common/target_x86_64.yaml
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-videotemplate.yaml \
      -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-audiotemplate.yaml \
      -d ../common/target_aarch64.yaml
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-videotemplate.yaml \
      -d ../common/target_aarch64.yaml

Graph

Sample Output

The sample output consists of the input video scaled by “scale-factor” mentioned in parameters of NvDsSampleVideoTemplateLib component in the graph.

The sample output consists of the addition of noise specified by “noise-factor” mentioned in parameters of NvDsSampleAudioTemplateLib component in the graph to the input audio.

deepstream-app

DeepStream reference application, showing a wide variety of features such as kitti dump, performance measurement, handling tiler events. Two graphs corresponding to the two sample configurations in the DeepStreamSDK are provided.

Graph Files

source30_1080p_dec_infer-resnet_tiled_display_int8.yaml – Graph file for 30 file inputs + Primary Detector + Tiled Output
source30_1080p_dec_infer-resnet_tiled_display_int8.parameters.yaml – File with parameters for various components in the above graph
source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.yaml – Graph file for 4 file inputs + Primary Detector + Tracker + 3 x Secondary Classifiers + Tiled Output
source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.parameters.yaml – File with parameters for various components in the above graph
README - Contains detailed graph description and execution instructions
ds_app_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_app_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-app

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh \
      source30_1080p_dec_infer-resnet_tiled_display_int8.yaml \
      source30_1080p_dec_infer-resnet_tiled_display_int8.parameters.yaml \
      -d ../common/target_x86_64.yaml

$ /opt/nvidia/graph-composer/execute_graph.sh \
      source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.yaml \
      source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.parameters.yaml \
      -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh \
      source30_1080p_dec_infer-resnet_tiled_display_int8.yaml \
      source30_1080p_dec_infer-resnet_tiled_display_int8.parameters.yaml \
      -d ../common/target_aarch64.yaml

$ /opt/nvidia/graph-composer/execute_graph.sh \
      source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.yaml \
      source4_1080p_dec_infer-resnet_tracker_sgie_tiled_display_int8.parameters.yaml \
      -d ../common/target_aarch64.yaml

deepstream-audio

Demonstrates audio classification using DeepStream.

Graph files

deepstream-audio.yaml – The main graph file
parameters.yaml – File containing parameters for the various components in the graph
README - Contains detailed graph description and execution instructions
ds_audio_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_audio_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-audio

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-audio.yaml \
      parameters.yaml -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-audio.yaml \
      parameters.yaml -d ../common/target_aarch64.yaml

Graph

Sample Output

Since this is an audio only app, the graph outputs audio classification results in a textual form on the terminal.

deepstream-triton

Demonstrates usage of triton server in a simple DeepStream pipeline along with the use of NVIDIA Graph Container Builder for creating use case based containers. The graph shows object detection using SSD Inception V2 Tensorflow model via the Triton server. For DGPU, the graph must be executed inside the container built using the container builder, since Triton is available only in docker format. For Jetson, the graph can be run directly on the device.

Graph and related files

deepstream-triton.yaml – The main graph file
deepstream-triton.parameters.dgpu_container.yaml – File containing parameters for executing the graph on DGPU
deepstream-triton.parameters.jetson.yaml – File containing parameters for executing the graph on Jetson
ds_triton_container_builder_cfg_dgpu.yaml – Container Builder configuration file for building a container for the graph
README - Contains detailed graph description and execution instructions
ds_triton_container_builder_cfg_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_triton_container_builder_cfg_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-triton

Sample Commands:

On x86:
Triton samples for DGPU need to be run in containers based on Triton. This
sample uses the NVIDIA Container Builder to build a container for the sample.

Steps:
* Build the container
$ container_builder build -c ds_triton_container_builder_cfg_dgpu.yaml \
      -d target_triton_x86_64.yaml -wd $(pwd)

* Start the container
$ docker run -it --rm  -e DISPLAY=:0 -v /tmp/.X11-unix/:/tmp/.X11-unix \
      --gpus all deepstream_triton_dgpu

On Jetson:
Triton samples for Jetson can be run natively or in a container.

Steps for running natively:
* Setup the Triton Server.
- $ cd /opt/nvidia/deepstream/deepstream/samples
- $ sudo ./triton_backend_setup.sh

* Prepare the triton model repo. Downloads the model files.
- $ cd /opt/nvidia/deepstream/deepstream/samples/
- $ ./prepare_ds_triton_model_repo.sh # prepare the triton model repo

* Launch the graph
- $ /opt/nvidia/graph-composer/execute_graph.sh deepstream-triton.yaml \
      deepstream-triton.parameters.jetson.yaml -d target_triton_aarch64.yaml

Note

Sometime there can be following error while running the graph.

"unable to load backend library: /usr/lib/aarch64-linux-gnu/libgomp.so.1: cannot allocate memory in static TLS block"

To solve the issue the libgomp.so.1 needs to be preloaded which can be done as follows before running the sample:

$ export LD_PRELOAD=/usr/lib/aarch64-linux-gnu/libgomp.so.1:$LD_PRELOAD

Graph

Sample Output

deepstream-camera

Demonstrates usage of a camera source in a simple DeepStream pipeline

Graph and related files

deepstream-camera.yaml – The main graph file
v4l2-usb-camera.parameters.yaml - Parameters file for using V4L2 USB camera. Works on both x86 and jetson
argus-csi-camera.parameters.yaml - Parameters file for using CSI camera using NVIDIA Argus API. Works on jetson only
README - Contains detailed graph description and execution instructions
ds_camera_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_camera_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-camera

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-camera.yaml \
      v4l2-usb-camera.parameters.yaml -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-camera.yaml \
      v4l2-usb-camera.parameters.yaml -d ../common/target_aarch64.yaml
OR
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-camera.yaml \
      argus-csi-camera.parameters.yaml -d ../common/target_aarch64.yaml

Graph

Sample Output

deepstream-action-recognition

Demonstrates usage of NvDsPreProcess component to do pre-processing outside the NvDsInferVideo component. In this sample, it is used to demonstrate temporal batching and pre-processing required by the NVIDIA TAO Action Recognition models.

Graph and related files

deepstream-action-recognition.yaml – The main graph file
parameters.yaml - File containing parameters for the various components in the graph
resources.yaml - List of resources required to execute the graph. This is required when using remote graph execution
README - Contains detailed graph description and execution instructions
ds_action_recognition_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_action_recognition_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform
config_preprocess_3d_custom.txt - NvDsPreProcess component configuration file for the 3D Action Recogntion model
config_preprocess_2d_custom.txt - NvDsPreProcess component configuration file for the 2D Action Recogntion model

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-action-recognition

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-action-recognition.yaml \
      parameters.yaml -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-action-recognition.yaml \
      parameters.yaml -d ../common/target_aarch64.yaml

Note

When executing on a remote target, additional argument “–resources resources.yaml” must be provided to execute_graph.sh script.

Graph

Sample Output

deepstream-subgraph

Demonstrates usage of subgraphs with DS components.

Graph and related files

main_graph.yaml – The main graph file
inference_subgraph.yaml - Inference subgraph used by thr main graph
resources.yaml - List of resources required to execute the graph. This is required when using remote graph execution
README - Contains detailed graph description and execution instructions
ds_subgraph_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform
ds_subgraph_container_builder_jetson.yaml - Configuration file for building application specific container for Jetson platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-subgraph

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh main_graph.yaml \
      -s inference_subgraph.yaml -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh main_graph.yaml \
      -s inference_subgraph.yaml -d ../common/target_aarch64.yaml

Note

When executing on a remote target, additional argument “–resources resources.yaml” must be provided to execute_graph.sh script.

Graph

Main graph:

Inference Subgraph

Sample Output

deepstream-3d-camera

Demonstrates capture, processing and rendering of 3D data from a 3D camera.

Graph and related files

deepstream-3d-camera.yaml – The main graph file
parameters-2drender.yaml - Parameters to render as 2D instead of 3D
ds_3d_loader_realsense.yaml - Configuration file for Data loader component (source)
ds_3d_filter_depth2cloud.yaml - Configuration file for Data filtering component (nvds3dfilter)
ds_3d_render_depth2d.yaml - Configuration file for Data render component (sink) - 2D rendering
ds_3d_render_pointcloud3d.yaml - Configuration file for Data render component (sink) - 3D rendering
resources.yaml - List of resources required to execute the graph. This is required when using remote graph execution
README - Contains detailed graph description and execution instructions
ds_3d_depth_camera_container_builder_dgpu.yaml - Configuration file for building application specific container for dGPU platform

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-3d-camera

Sample Commands:

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-3d-camera.yaml \
      -d ../common/target_x86_64.yaml

On Jetson:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-3d-camera.yaml \
      -d ../common/target_aarch64.yaml

NOTE:

Note

“parameters-2drender.yaml” can be added to the commandline to render 2D depth/color images instead of the default 3D point cloud rendering.
Mouse interactions are possible with default 3D point render.

Graph

Sample Output

deepstream-ucx-test1

Demonstrates how to use DeepStream UCX communication components for data transfer:

Graph and related files

deepstream-ucx-test1-server.yaml – Server application graph file (sender)
deepstream-ucx-test1-client.yaml – Client application graph file (receiver)
server-parameters.yaml - Configurable parameters for various components in the server application.
client-parameters.yaml - Configurable parameters for various components in the client application.
README - Contains detailed graph description and execution instructions

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-ucx-test1

Sample Commands:

Run the server first:

Update `addr` parameter in `server-parameters.yaml` to mellanox NIC address on
which the server listens.

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-ucx-test1-server.yaml \
      server-parameters.yaml -d ../common/target_x86_64.yaml

Run the client next:

Update addr` parameters in `client-parameters.yaml` to address on which the
server is listening. This may be executed on the same device in another terminal
or on another device with a mellanox NIC connected to the server side mellanox
NIC.

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-ucx-test1-client.yaml \
      client-parameters.yaml -d ../common/target_x86_64.yaml

Note

This sample is supported only for x86.

Graph

Server Graph

Client Graph

Sample Output

deepstream-ucx-test2

Demonstrates how to use DeepStream UCX communication components for data transfer including DS metadata.

Graph and related files

deepstream-ucx-test2-server.yaml – Server application graph file (sender)
deepstream-ucx-test2-client.yaml – Client application graph file (receiver)
server-parameters.yaml - Configurable parameters for various components in the server application.
client-parameters.yaml - Configurable parameters for various components in the client application.
README - Contains detailed graph description and execution instructions

Path - /opt/nvidia/deepstream/deepstream/reference_graphs/deepstream-ucx-test2

Sample Commands:

Run the server first:

Update `addr` parameter in `server-parameters.yaml` to mellanox NIC address on
which the server listens.

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-ucx-test2-server.yaml \
      server-parameters.yaml -d ../common/target_x86_64.yaml

Run the client next:

Update addr` parameters in `client-parameters.yaml` to address on which the
server is listening. This may be executed on the same device in another terminal
or on another device with a Mellanox NIC connected to the server side Mellanox
NIC.

On x86:
$ /opt/nvidia/graph-composer/execute_graph.sh deepstream-ucx-test2-client.yaml \
      client-parameters.yaml -d ../common/target_x86_64.yaml

Note

This sample is supported only for x86.

Graph

Server Graph

Client Graph