DeepStream Components

DeepStream/Gstreamer pipeline is implemented using the Graph Specification by wrapping GStreamer element in Graph Component and adding supportive Components to enable communication with GStreamer pipeline from Graph components. This section explains components types used in DeepStream graphs.

Interfaces

DeepStream extensions provide interfaces (abstract base types) for some commonly used types of components and functionality. These interfaces provide a way for other components to interact with a variety of components implementing or inheriting from the interface without depending on the actual type/implementation. For e.g. NvDsInfer component has a parameter infer-model-config with handle type INvDsInferModelConfigComponent. The INvDsInferModelConfigComponent base type is for components representing a NN model (the model files, associated configuration, post-processing algorithm). Thus, NvDsInfer component can load any type of model without actual depending on the concrete type of the model component by calling APIs of INvDsInferModelConfigComponent.

The interfaces also act as base types which can be used by tools like registry and composer to filter and group together components inheriting from the interface. Thus in the above example, registry command registry comp list -b nvidia::deepstream::INvDsInferModelConfigComponent can be used to find all the components that can be connected to the parameter “infer-model-config” of NvDsInfer. Composer also uses this base type to provide quick access context-menu to on a handle type parameter create a component inheriting from the base type and attaching the newly created component to the parameter. However interfaces themselves are abstract and won’t be listed in the component list from the Composer for users to create the instances through drag&drop.

The rest of the section provides a high-level overview of the interfaces provided by the DeepStream extensions. These interfaces are part of the NvDsInterfaceExt extension. The detailed API and parameter description is part of the document Graph Composer Extensions Manual.

Element - INvDsElement

This interface is used by components representing a GStreamer element in the graph. An element is the basic building block of a GStreamer based pipeline. It represents a unit of processing. These elements can perform very specific tasks like video decoding or encoding or they might perform a set of related tasks together e.g. Object detection - pre-processing, inference and post-processing to get bounding box co-ordinates of detected objects.

This interface is primarily used by the NvDsScheduler component to manage the lifecycle of the elements and construct a GStreamer pipeline from the elements represented by the components. It should only be used as a base class and derived from to create a component representing a GStreamer element. Ideally, no other component should ever use this interface for interacting with the elements. Other mechanisms are provided which are described later. This interface provides following main virtual methods.

`gxf_result_t create_element()`	Concrete types must use this to create the GStreamer element and return the error/success status
`gxf_result_t bin_add(GstElement *pipeline)`	Concrete types must use this to add the created element to the provided GStreamer pipeline and return the error/success status
`GstElement *get_element_ptr()`	Concrete types must return the raw pointer to the GStreamer element

Most components implementing this interface are thin wrappers created over corresponding GStreamer element. In such cases, there would be a 1-to-1 mapping between the Component parameter and the GStreamer element property. Special cases like NvDsMultiSrcInput exist which are not wrappers over a single element but elaborate implementations using GStreamer API to combine multiple elements together.

I/Os - INvDsIO/INvDsInput/INvDsOutput

The INvDsIO interface is used to represent to represent an input/output of an INvDsElement based component. It is used to represent a GStreamer Pad which is the input / output port of a GStreamer element through which it receives/transmits data. The direction input or output is from the perspective of the component and not the application. An element receives data from an input and transmits data on an output. The base type for an input component is INvDsInput and that for an output is INvDsOutput. Both of these inherit from INvDsIO. Depending on the I/O (data) availability type, multiple concrete types are derived from INvDsInput and INvDsOutput, which are described in detail later. The complete hierarchy looks like

Any concrete pad type can interact with via INvDsIO interface or INvDsInput/INvDsOutput interface or the concrete pad classes themselves. INvDsElement based components have parameters with the handle type of the concrete pad types (depending on the direction – input/output and availability – static/on-request/dynamic/multi). In the graph, for such a component to send/receive data, an I/O component of the corresponding type must be attached to the parameter. For e.g., the NvDsStreamMux component has a “on-request” input and a “static” output. Thus, for it to receive data a NvDsOnRequestInput component must be attached to its video-in-%u parameter and for it to send data a NvDsStaticOutput component must be attached to its “video-out” parameter.

An I/O is identified by its template name in the element. GStreamer APIs can be used to find more details about the underlying pad details using this template name. These interfaces expose the following main virtual methods:

Interfaces	Details
`void set_template_name(const char *templ)`	Called by INvDsElement based element to set the template name of the i/o.
`INvDsElement *get_element()`	Get the owning INvDsElement based element of the i/o.
`const char *get_template_name()`	Get the template name for the i/o.
`GstPadSPtr get_pad(const char *requested_name)`	Get the underlying GstPad pointer as a shared_ptr. Detailed behavior of this API for each type of I/O has been documented in the “Pads” subsection of “Basic Components” section
`const std::vector<GstPadSPtr> get_all_pads()`	Get a list of all the underlying GstPad objects as vector of shared_ptrs. In case of multi/on-request/dynamic type of I/Os, element might have multiple underlying pads associated with the I/O. This API can be used to get the raw pointers to all such pads.

Custom extensions will almost never need to implement a I/O based component from these interfaces. All the concrete types that would be required are already provided as a part of the NvDsBaseExt extension. Custom components wanting to interact with pads directly can use APIs from this interface. However, if only access to the data being received/transmitted is required “Probe” based mechanism is recommended.

Connections - INvDsConnection

The INvDsConnection interface is used by components representing a link between an output component and an input component i.e., a link between two INvDsElement based components. Concrete implementations do the actual work of connecting the components.

The main virtual methods provided by this interface are:

Methods	Details
`gxf_result_t connect()`	Concrete implementations must link the source and target I/Os and return the status
`void disconnect()`	Concrete implementations must unlink the source and target I/Os
`bool is_io_compatible(INvDsIO *io, GstPadSPtr gstpad)`	Method to check if an I/O is compatible with the I/O on the other side of the connection.
`void on_pad_added(INvDsIO *io, GstPadSPtr gstpad)`	Notification from dynamic I/Os that a new underlying GstPad object has been added.
`void on_pad_removed(INvDsIO *io, GstPadSPtr gstpad)`	Notification from dynamic I/Os that an existing underlying GstPad object has been removed.
`void on_no_more_pads(INvDsIO *io)`	Notification from dynamic I/Os that no more new underlying GstPad objects would be added.

The interface APIs are primarily used by the NvDsScheduler component during pipeline construction to link elements. Ideally, custom components and extensions will never need to implement a component based on this interface or use the APIs provided by this interface.

The “output” component of the upstream element component must be attached to the “source” parameter of the INvDsConnection based component and the “input” component of the downstream element component must be attached to the “target” parameter.

DeepStream Domain Component - INvDsComponent

The INvDsComponent interface should be used by custom components working with DeepStream components in the graph. The main feature of this interface is that it adds start() and stop() virtual methods on top of the Component interface.

The component gets a notification with start() method that all other components in the graph have been initialized, underlying pipeline constructed and ready to start. In this method it is safe to call APIs of other components through their handles. This is not guaranteed to be safe in initialize() method provided by Component because the other component may not be initialized by then.

stop() method is called when the underlying pipeline has been stopped and before components are de-initialized.

Probe - INvDsProbe

The INvDsProbe interface is used by components representing a probe. A probe is used for monitoring dataflow and in-place buffer modifications by adding callbacks on the I/O components of and INvDsElement based component.

Depending on the flags used, these callbacks are called for every buffer/query/event that flows through the I/O. INvDsProbe based components are used to only install a probe. The components that want to implement these callbacks need to implemented the INvDsInPlaceDataHandler interface described later.

Ideally, custom components and extensions will never need to implement this interface or use APIs of this interface, it is required by the NvDsScheduler for setting up the probes.

Probe Callback Implementation - INvDsInPlaceDataHandler

Custom components wanting to monitor data flow or to do in place data modifications at a particular I/O of a component must inherit from this interface. The methods of this interface are called by the INvDsProbe based component when a buffer/event/query arrives at the I/O.

The interface provides the following virtual methods. Concrete implementations can implement any of the methods as required.

Methods	Details
`bool handle_buffer(GstPad *pad, nvidia::gxf::Entity buffer_data)`	Notification to the component to handle buffer data
`bool handle_event(GstPad pad, GstEvent event)`	Notification to the component to handle the passed event
`bool handle_query(GstPad pad, GstQuery query)`	Notification to the component to handle passed query

Returning true from these methods tells the probe component to let them pass through. Returning false implies that the objects should be dropped at that point. The buffer_data argument of the handle_buffer method is an Entity that contains the following data:

Key	Value Type
`BUF_DATA_KEY_GST_BUFFER`	GstBufferHandle
`BUF_DATA_KEY_NVBUFSURFACE`	NvBufSurfaceHandle
`BUF_DATA_KEY_NVBUFAUDIO`	NvBufAudioHandle
`BUF_DATA_KEY_GST_VIDEO_BATCH_META`	NvDsBatchMetaHandle
`BUF_DATA_KEY_GST_AUDIO_BATCH_META`	NvDsBatchMetaHandle

The value types are explained in the Data components section.

Depending upon the type of data flowing through the I/O, not all values might be present in the buffer_data entity. For e.g for a I/O handling video frames, buffer_data can contain NvBufSurfaceHandle and NvDsBatchMetaHandle for video but not NvBufAudioHandle and NvDsBatchMetaHandle for audio.

The handle_buffer implementations must check for existence of the values before directly using the values.

INvDsInPlaceDataHandler based components must register a handle parameter of type NvDsProbeConnector and in the initialize() method must call set_handler() and set_flags() method of the NvDsProbeConnector to receive callbacks.

For more information on GStreamer queries and events and the APIs, refer to the following links:

Action - INvDsAction

The INvDsAction interface is used to represent a GstElement action. An action is a trigger that signals an INvDsElement based component to perform some action corresponding to that trigger.

Every concrete INvDsElement based component that supports actions will have a corresponding concrete type derived from INvDsAction. This concrete type will expose a method to trigger the action.

The INvDsAction based components act only as helper components for providing a prototype for the trigger and connecting a component that triggers an action to a component that reacts to the trigger.

The custom component that triggers the action must be implemented separately. It must register a handle parameter of the type of concrete INvDsAction based component. The component that reacts to the trigger would also do the same. The concrete INvDsAction based component must be attached to the parameters of the other two components. When required, the custom component can trigger the action by calling the action trigger method on the handle of the INvDsAction based component, for example, NvDsRecordAction component. This action is used to toggle recording on/off recording of a camera source. Thus this component has methods start_record(), stop_record() and take_snapshot(). This action is supported by NvDsMultiSrcInputWithRecord component, thus it has a handle parameter “record-action” of type NvDsRecordAction. To trigger the action, a custom component must have a Parameter of type Handle<NvDsRecordAction>. The component NvDsRecordAction must be added to the graph and attached to the parameters of both, the NvDsMultiSrcInputWithRecord component and the custom component. The custom component can then call start_record(), stop_record() and take_snapshot() methods as required.

Signal - INvDsSignal

The INvDsSignal interface is used to represent a GstElement signal. A signal is a callback from an INvDsElement based component on the occurrence of some event. Every concrete component that supports signals will have a corresponding concrete type derived from INvDsSignal. This concrete type will expose the prototype of the callback function and a method to set the callback function.

The INvDsSignal based components act only as helper components for providing a prototype for the callback and connecting a component that emits the signal to a component that implements the callback handling the signal.

The custom component that handles the signal must be implemented separately. It must register a handle parameter of the type of concrete INvDsSignal based component. It must implement the Handler interface exposed by the signal component and call set_handler() method on the handle of the signal component. The component that emits the signal would similarly register a handle parameter of type of the concrete signal component. The concrete INvDsSignal based component must be attached to the parameters of the other two components. When the INvDsElement based element emits the corresponding signal, the methods of the Handler interface will get called via the signal component for example, NvDsModelUpdateSignal component. This signal is used to notify the status of on the fly model update. Thus the Handler interface for this signal has the method on_model_updated(int errorCode, char *configFilePath). This signal is emitted by NvDsInfer component, thus it has a handle parameter model-updated-signal of type NvDsModelUpdateSignal.

To receive the signal callback, a custom component must implement NvDsModelUpdateSignal::Handler interface. It must register a parameter of type Handle<NvDsModelUpdateSignal> and call set_handler() on the handle. The component NvDsModelUpdateSignal must be added to the graph and attached to the parameters of both NvDsInfer and the custom component.

Element Property Controller – INvDsPropertyController

The INvDsPropertyController interface is used by components acting as runtime property(parameter) controllers for INvDsElement based components. Every concrete INvDsElement based component that supports runtime property control will have a corresponding concrete type derived from INvDsPropertyController. This concrete type will expose methods to get/set the properties(parameters) of the element component. A method for each property that can be controlled will be exposed. The INvDsPropertyController based components act only as helper components for providing prototypes for the get/set methods and connecting a component that can control the property to the element component that the property belongs to.

The custom component that controls the properties must be implemented separately. It must register a handle parameter of the type of concrete INvDsPropertyController based component. The component the properties belong to would also do the same. The concrete INvDsPropertyController based component must be attached to the parameters of the other two components. When required, the custom component can call the get/set methods of the properties on the handle of the INvDsAction based component for exmaple, NvDsOSDPropertyController component. This component can be used to get/set properties of NvDsOSD component, thus it has a handle parameter property-controller of type NvDsOSDPropertyController.

To controls properties of the NvDsOSD component, a custom component must have a Parameter of type Handle<NvDsOSDPropertyController>. The component NvDsOSDPropertyController must be added to the graph and attached to the paramers of both, the NvDsOSD component and the custom component. The custom component can then call set/get methods exposed by NvDsOSDPropertyController.

Configurations – INvDsConfigComponent template and specializations

The INvDsConfigComponent interface is used as a base class for components acting as configuration providers (collection of settings, associated files, binaries etc.) for other components. The aim of this interface and the components that would derive from it is to:

Simplify specification of group of parameters and their values that are fixed for a use case.

Package together all related assets (e.g. model files, custom implementation libraries)

Remove the need of knowing beforehand the paths to files during deployment and thus hardcoding these paths to related asset files in the yaml files. Since the files are packaged with the extension library, the component can internally determine the correct absolute path to the files.

The INvDsConfigComponent interface is a templated type. It provides a standard virtual method fill_config(Config *config) for any type of configuration, Config being the template type. It also provides a helper function get_absolute_path(std::string path) which can convert a path relative to the extension’s binary to an absolute path at runtime.

Components wanting to use this template interface for reading configurations must declare a structure with supported configuration parameters and specialize the template by using the struct as a template argument. These components must have a parameter with handle type of the specialized template. The configuration can then be read by calling the fill_config() method of the handle.

Components wanting to act as configuration providers must inherit from the specialized template and implement the fill_config() method. The configuration provider component is responsible for populating the config structure appropriately. These components can use the get_absolute_path() method to convert paths relative to extension binary to absolute paths.

The DeepStream extensions currently have the following configuration provider interfaces:

INvDsInferModelConfigComponent

This interface acts as a configuration provider for the NvDsInferVideo and NvDsInferAudio components. Components implementing this interface must populate the following parameters of the configuration structure in the fill_config() method.

Method	Details
`std::string config_file_path`	Absolute path to the nvinfer configuration file for the model
`std::string model_engine_path`	Absolute path to the pre-generated TensorRT engine file for the model (OPTIONAL)

NvDsInferVideo and NvDsInferAudio components expose a handle parameter infer-model-config of type INvDsInferModelConfigComponent to which components implementing this interface can be attached.

Components implementing this interface are model configuration providers. Thus they will package the DeepStreamSDK Gst-nvinfer/ Gst-nvinferaudio configuration file along with the model files. The specification of the Gst-nvinfer configuration file can be found at https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvinfer.html#gst-nvinfer-file-configuration-specifications.

INvDsVideoTemplatePluginConfigComponent / INvDsAudioTemplatePluginConfigComponent

These interfaces act as configuration providers for the NvDsVideoTemplate and the NvDsAudioTemplate components respectively. Components implementing these interfaces must populate the following parameters of the configuration structure in the fill_config() method.

Method	Details
`std::string customlib_name`	Absolute path to the custom library that the audio/video template component should load
`std::unordered_map<std::string, std::string> customlib_props`	name-value pairs of properties to be passed to the custom library.

NvDsVideoTemplate and NvDsAudioTemplate components expose handle parameters video-template-config and audio-template-config respectively to which components implementing the interfaces INvDsVideoTemplatePluginConfigComponent or INvDsAudioTemplatePluginConfigComponent can be attached. Components implementing these interfaces will package the custom library that must be provided to the template components. NvDsVideoTemplate and NvDsAudioTemplate components are wrappers for the DeepStreamSDK Gst-nvdsvideotemplate and Gst-nvdsaudiotemplate plugins respectively. For information on these plugins, refer to https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvdsvideotemplate.html and https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvdsaudiotemplate.html.

Data Components

DeepStream extensions provide some Data components which can be passed around as part of entities. This removes the need of having fixed structures / function prototypes to pass data, thus removing the problem of API breakage with changing versions. Entity provides a key-value mechanism for adding and retrieving data components. The key is of string type. Both the component producing these entities and consuming the entities must know the mapping between a key and the type of the associated value. Components consuming such entities must also check the existence of the key value pair by having a Boolean check on the return value of the get() method of the entity. The rest of the section provides details on the data components provided by DeepStream extensions. These data components are part of the NvDsInterfaceExt extension and are defined in the interfaces.hpp header file of the extension.

GstBufferHandle

Wraps the pointer to GstBuffer which is the GStreamer data structure for buffers - https://gstreamer.freedesktop.org/documentation/gstreamer/gstbuffer.html . The key usually associated with this data component is defined by BUF_DATA_KEY_GST_BUFFER.

NvBufSurfaceHandle

Wraps the pointer to NvBufSurface which is the DeepStream data structure for batched video frames - https://docs.nvidia.com/metropolis/deepstream/dev-guide/sdk-api/structNvBufSurface.html . The key usually associated with this data component is defined by BUF_DATA_KEY_NVBUFSURFACE.

NvBufAudioHandle

Wraps the pointer to NvBufAudio which is the DeepStream data structure for batched audio data – LINK TBD. The key usually associated with this data component is defined by BUF_DATA_KEY_NVBUFAUDIO.

NvDsBatchMetaHandle

Wraps the pointer to NvDsBatchMeta which is the DeepStream data structure for metadata associated with batched video frames or batched audio data – https://docs.nvidia.com/metropolis/deepstream/dev-guide/sdk-api/struct__NvDsBatchMeta.html . The key usually associated with this data component is defined by BUF_DATA_KEY_VIDEO_BATCH_META or BUF_DATA_KEY_AUDIO_BATCH_META depending on whether the NvDsBatchMeta structure contains information for video or audio.

Basic Components

This section explains the basic concrete components that are required to create any type of DeepStream based graph. All these basic components are part of the NvDsBaseExt extension.

I/Os

As explained in the previous section of the INvDsIO interface, I/Os are the input/output ports of an INvDsElement based component. An I/O is defined by a template which consists of:

the direction (input or output)

it’s availability

the format of the data that flows through it.

An I/O is identified by the template’s name. An element component will have one or more I/Os depending on it’s functionality and input/output requirements. The following types of I/O availabilities are defined along with the behavior of the GstPadSPtr get_pad(char *requested_name) method of INvDsIO interface for each of the type.

Static – The I/O is always available on the element:

get_pad() always returns the underlying GstPad object (single). requested_name argument is ignored

OnRequest – The underlying I/O objects must be requested from the element by the application:

e.g. NvDsStreamMux component which aggregates frames from multiple upstream components. Underlying GstPad objects for pushing data to the component must be requested from it, one for each upstream component.

e.g. NvDsTee component which broadcasts input data to multiple outputs. Underlying GstPad objects for pushing data to multiple downstream components must be requested from it, one for each downstream component.

If a non-NULL requested_name is provided to get_pad(), the method will try to create an underlying GstPad with the supplied name if it does not already exist and return it. If a pad with the provided requested_name cannot be created, NULL will be returned. If requested_name is not supplied, it creates a underlying GstPad with an internally generated name from the I/O’s template name and return it.

Dynamic – The underlying I/O object is made available by the element at runtime when some conditions are satisfied:

e.g. NvDsSingleSrcInput component which reads from an input like FILE or RTSP and can decide whether it can output audio or video or both only after parsing the file headers/initial communication with the RTSP server.

get_pad() tries to get an existing underlying GstPad object having name requested_name, returns NULL if no such pad is found. requested_name is mandatory.

Multi – A special I/O availability created for components which output data from multiple sources. This availability exposes itself as a single I/O but internally groups together multiple outputs, one for each of the sources. Due to the group representation, it makes the graph and components that would use the I/O independent of the number of sources. This makes it easy to have a variable number of sources, and to dynamically add/remove sources at runtime without any changes to the graph.

e.g. NvDsMultiSrcInput component which outputs data from multiple sources.

get_pad() tries to get an existing underlying GstPad object having name requested_name, returns NULL if no such pad is found. requested_name is mandatory.

Based on these concepts, the following concrete I/O types are provided:

NvDsStaticOutput
NvDsOnRequestOutput
NvDsDynamicOutput
NvDsMultiOutput
NvDsStaticInput
NvDsOnRequestInput

An element component exposes a handle parameter of a concrete I/O type for each of the I/O template it supports. For any kind of interaction with an I/O, like connecting two elements, in-place manipulation of data flowing through the I/O, a concrete type of I/P component must be attached to the parameter. Depending on the function of the element component, a variety of I/O templates and thus handle parameters will be present on the component.

Connections

The connection components are responsible for connecting two elements. Connecting two elements means linking an output of one of the elements (called upstream element) to an input of the other element (called downstream element) which enables flow of data between the two elements through the linked I/Os. Two types of connection components are provided:

NvDsConnection

This connection component can be used to link any type of output component to any type of input component with the exception of NvDsMultiOutput. It provides two handle parameters “source” and “target”. To connect, an output I/O component attached to an upstream element must be attached to the “source” parameter of an NvDsConnection component and an input I/O component attached to a downstream element must be attached to the “target” parameter of the NvDsConnection component.

NvDsMultiSrcConnection

As with NvDsMultiOutput, this connection type has been specially created for handling the use case of variable number of sources and dynamic addition/removal of sources at runtime. It can only be used to link a NvDsMultiOutput component to a NvDsOnRequestInput component, i.e. link an element which outputs data from multiple sources to aggregation type of elements. It has two parameters, “source” to attach the NvDsMultiOutput component to and “target” to attach the NvDsOnRequestInput component.

Probes

As mentioned earlier probes are used for in-place manipulation and monitoring of data flowing through the I/Os of an element component. Two components are provided to achieve this.

NvDsProbe

This component is responsible for intercepting data (buffers/events/queries) flowing through an I/O and calling the callback functions of the target handler component whenever data arrives at the I/O. It has two handle type properties. “io” to which an I/O component must be attached which needs to be probed and “probe-handler” to which a NvDsProbeConnector component must be attached.

NvDsProbeConnector

This component is used to link a NvDsProbe component to a target component derived from INvDsInPlaceDataHandler which makes it capable of handling callbacks from the probe. This component provides the following public methods, defined in nvds_probe_connector.hpp header file as part of NvDsBaseExt:

Method	Details
`void set_handler(INvDsInPlaceDataHandler *handler)`	Set the component that will be handling the probe callbacks. Must be called by INvDsInPlaceDataHandler based components.
`INvDsInPlaceDataHandler *get_handler()`	Get the component that will be handling the probe callbacks.
`void set_flags(NvDsProbeFlags flags)`	Set the types of information to receive probe callbacks for. Must be called by INvDsInPlaceDataHandler based components.
`NvDsProbeFlags get_flags()`	Get the types of information to receive probe callbacks for.
`void set_io(INvDsIO *io)`	Set the IO this component is connected to. This method is called by NvDsProbe.
`INvDsIO *get_io()`	Get the IO this component is connected to.

The flags must be a bitwise OR combination of the following:

Method	Details
`NvDsProbeFlags::NONE`	The target does not wish to handle any data
`NvDsProbeFlags::BUFFER`	The target component wants to handle buffers
`NvDsProbeFlags::EVENT`	The target component wants to handle events
`NvDsProbeFlags::QUERY`	The target component wants to handle queries

The target component must register a handle parameter of type NvDsProbeConnector. The target component in it’s initialize() method must call set_handler() and set_flags() method of the NvDsProbeConnector to receive callbacks.

NvDsScheduler

The NvDsScheduler component is responsible for setting up the underlying pipeline and managing its state, connecting and scheduling components in the graph and managing their lifecycle. It is also responsible for setting up the probe handlers.

There must be a NvDsScheduler in a graph. Without the presence of a NvDsScheduler in the graph, the DeepStream based components would never get scheduled. Additionally, there must be only one NvDsScheduler in a graph. In case multiple NvDsSchedulers are added to a graph, only one gets actually executed others become redundant and get skipped.