Overview#
Graph Composer is a tool suite to enable users to build and deploy high-performance and low-latency AI applications from edge to the cloud. These applications can be created using a rich C++ or Python API’s and also in a GUI environment. These API’s and tools are based on the core principles of GXF GXF (Graph eXecution Format) provide a complete ecosystem for a developer to create and deploy AI applications with ease.
Concepts#
GXF (Graph eXecution Format)#
GXF specification follow an entity-component design pattern implementing the “composition over inheritance” paradigm. An entity itself is just a light-weight object which owns components. Components define how an entity interacts with the rest of the application.
The GXF specification consists of five things:
A definition of a compute graph using an extensible entity-component architecture
A definition of APIs which allow developers to attach code and data to a compute graph
A specification of scheduling rules and the data flow between sources and sinks
A file format description for storing a compute graph in a file
A list of analytics, monitoring and debugging info produced during runtime
GXF framework streamlines the process of application development by
Enabling code reuse amongst developers with minimal integration overhead
Establishes common streaming media data types with customization hooks / interfaces
A rich tool suite to help analyze, debug, optimize and deploy high performance AI applications
Graph#
A graph is a data-driven representation of an AI application. Implementing an application by using programming code to create and link objects results in a monolithic and hard to maintain program. Instead a graph object is used to structure an application. The graph can be created using specialized tools and it can be analyzed to identify potential problems or performance bottlenecks. The graph is loaded by the GXF runtime in order to be executed.
The functional blocks of a graph are defined by the set of nodes which the graph is owning. Nodes can be queried via the graph using certain query functions. For example it is possible to search for a node by its name.
Node#
GXF uses an entity-component design principle for nodes and a GXF compute node is a GXF entity. This means that a node is a light-weight object whose main purpose is to own components. A node is a composition of components. Components are the “things” which define the aspects and behavior of the node. In order to customize a GXF node a developer does not derive from node as a base class, but instead composes objects out of components. Components can be used to provide a rich set of functionality to a node and thus to an application.
Every node in the graph is uniquely identifiable as described in more detail in the section about object identification.
Components#
Components are the main functional blocks of an application. GXF provides a couple of standard components such as transmitter, receiver, codelet, subgraph. GXF also allows a developer to extend the GXF runtime by injecting its own custom components with custom features to fit a specific use case.
Codelets#
The most common component is a codelet which is used for data processing and code execution. In order to implement a custom codelet the developer implements a certain set of functions like start and stop. A special system called scheduler - The GXF Scheduler - will call these functions at the appropriate time as specified by the developer. Typical examples of triggering code execution are: receiving a new message from another node, or performing work on a regular schedule based on a time trigger.
Subgraph#
Subgraph allows a node to wrap a whole graph and treat it like a sub-processing unit. Such subgraphs help greatly to increase the modularity of an application and allow reuse of compute graphs across applications. A subgraph’s internals are opaque to the outside world, and it can employ its own scheduler.
System#
Systems are used to create, govern and destroy components. Standard systems provided by GXF are for example: Greedy Scheduler, Multi-thread scheduler, Event Based Scheduler, GraphWorker, GraphDriver.
Edges / Connection#
A GXF dataflow edge is a GXF entity. Thus an edge is a light-weight, uniquely identifiable container of (edge) components. By default every edge has a DirectedEdge component which stores source and target. By default all edges of a compute graph are governed by the MessageRouter. A developer can inject custom (edge) components and (edge) systems into GXE.
Messages#
A GXF message is an entity. Thus a message is a light-weight, uniquely identifiable container of (message) components. Standard message components are: Header, Checksum, GPU Buffer, CPU Buffer. A typical message systems is a data pool which handels buffer allocation. Standard message systems provided by GXE are: GPU Buffer Pool, CPU Buffer Pool. A developer can inject custom (message) components and (message) systems into GXE.
Extension#
An extension is a compiled shared library of a logical group of component type definitions and their implementations along with any other asset files that are required for execution of the components. Some examples of asset files are model files, shared libraries that the extension library links to and hence required to run, header and development files that enable development of additional components and extensions that use components from the extension.
An extension library is a runtime loadable module compiled with component information in a standard format that allows the graph runtime to load the extension and retrieve further information from it to:
Allow the runtime to create components using the component types in the extension.
Query information regarding the component types in the extension:
The component type name
The base type of the component
A string description of the component
Information of parameters of the component – parameter name, type, description etc.,
Query information regarding the extension itself - Name of the extension, version, license, author and a string description of the extension.
The section Extension Development Workflow talks more about this with a focus on developing extensions and components.
Graph eXecution Engine (GXE)#
GXE accepts an application file which has the GXF components listed and a manifest file that has dependent extensions. GXE loads the graph and the dependent extensions and activates all the entities that are defined in the application file. Below are the steps that describes the functionality of GXE.
Create GXF context
Load the manifest file containing the GXF extensions
Load the application file containing the GXF entities and their connections
Activate the entities
Wait for the graph to complete
Destroy GXF context
Tools#
Graph Composer#
The Composer is an GUI application designed to create AI Application pipeline through an easy-to-use graphic interface, reducing the complexity of application development significantly and thus reducing time to market.
The Composer is based on the https://docs.omniverse.nvidia.com/ Kit which provides a highly responsive hardware-accelerated GUI.
Registry#
The Registry is an integral part of the Graph Composer ecosystem and is responsible for providing unified interface between the extensions and tools. Registry service can be accessed using registry CLI.
Container Builder#
Container Builder (CB) is used to build docker images for AI Application graphs created using Composer. In addition to docker images, it can also push the final image into the cloud for deployment.
Container Builder interacts with Registry: to:
Download extensions and other related files into your local system.
Copy other required files specified in the config file to generate an intermediate work folder and an optimized dockerfile.
Convert archives/packages dependencies and instructions into docker and try to build a minimal sized local image. For optimization, you can easily configure container builder to support multi-stage docker build.
Container Builder supports graph installing and container image building on x86 Ubuntu systems. It can also build arm64 images from x86_64 platforms - to do this, you will need to install QEMU and bintutils. Additionally, CB tool can now build multiarch images with the prerequisite that the user needs to specify a remote docker repository where the images can be pushed.
GXF Server#
The GXF server is a grpc server that integrates multiple tools, including registry and container builder, and provides services for various command line tools such as container builder cli, and the graph composer.
The gxf server starts automatically on installation of graph composer debian. The server starts using 50051 as the default port. The users can change this by setting the environment variable GXF_SERVER_PORT prior to running the server.
GXF CLI#
GXF CLI tool is used to connect GXF runtime remotely for monitoring or configuring tasks.
Workflow#
Graph Composer workflow enables AI application development using extensions from different contributors along with custome extension development. They enable AI Application developers to create the applications using GUI based tool and deploy them using Container Builder tool without developing any code. It provides central repository of NVIDIA� optimized extensions for different hardware platforms.
