DeepStream Reference Application - deepstream-audio app#

Audio application builds on top of the deepstream-app reference application to demonstrate inference of audio data.

deepstream-audio application is open sourced and packaged with DeepStream 6.4 as an alpha product. The AI model used to demonstrate audio inference is an ONNX model. The model file is available within the DeepStream 6.4 package at:

/opt/nvidia/deepstream/deepstream/samples/models/SONYC_Audio_Classifier/sonyc_audio_classify.onnx

Quick run instructions:

$ cd /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-audio/configs
$ deepstream-audio -c ds_audio_sonyc_test_config.txt

DeepStream Audio Reference Application Architecture and Sample Graphs#

The image below shows the expected output:

DeepStream Audio Reference Application Architecture

The image below shows sample graph that could be constructed for audio rendering:

DeepStream Audio Sample Graph

GStreamer pipeline demonstrating the above sample graph without nvinferaudio component:

USE_NEW_NVSTREAMMUX=yes gst-launch-1.0 \
uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sonyc_mixed_audio.wav name=source_0 ! queue ! audioconvert ! audioresample ! mux.sink_0 \
uridecodebin uri=file:///opt/nvidia/deepstream/deepstream/samples/streams/sonyc_mixed_audio.wav name=source_1 ! queue ! audioconvert ! audioresample ! mux.sink_1 \
nvstreammux name=mux batch-size=2 ! \
nvstreamdemux name=demux \
demux.src_0 ! audioconvert ! audioresample ! alsasink async=false \
demux.src_1 ! audioconvert ! audioresample ! alsasink async=false
On this page