GXF Component Interfaces#

Codelet#

class nvidia::gxf::Codelet#

Codelets are special components which allow the execution of custom code. The user can create her own codelets by deriving from this class and overriding the functions initialize(), start(), tick(), stop(), and deinitialize().

virtual ~Codelet() = default#: Destructor.

virtual gxf_result_t start()#

This function is called during the start phase of the codelet. It allows derived classes to execute custom code during the start phase. This is a good place to obtain resources which are necessary for ticking the codelet. This function is guaranteed to be called before the first call to tick().

Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t tick() = 0#

This function is called whenever the codelet is expected to do work, e.g. when an event was received or periodically. The tick() method can be specified with various other member functions. This function is the main work horse of the codelet.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t stop()#

This function is called during the stop phase of the codelet. It allows derived classes to execute custom code during the stop phase. This is a good place to clean up any resources which where obtained during ‘start’. After the codelet is stopped it should be in the same state as it was before ‘start’ was called. Be careful to not leave any unintended left overs as ‘start’ might be called again afterwards. It is guaranteed that stop is called after the last call to tick. When start was called stop will be called, too.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

int64_t getExecutionTimestamp() const#

Timestamp (in nanoseconds) of the beginning of the start, tick or stop function. The execution timestamp does not change during the start, tick or stop function.

Returns:: The execution timestamp in nanoseconds.

double getExecutionTime() const#

Similar to getExecutionTimestamp() but returns time as a floating point number and using seconds as unit. Equivalent to ‘ToSeconds(getExecutionCount())’.

Returns:: The execution time in seconds.

double getDeltaTime() const#

The delta between the current execution time and the execution time of the previous execution. During the start function this will return 0.

Returns:: The delta time in seconds.

int64_t getExecutionCount() const#

Returns the number of times a codelet is executed. This will return 0 during start and 1 during the first tick.

Returns:: The execution count.

bool isFirstTick() const#

Returns true if this is the first time tick is called after start.

Returns:: True if this is the first tick, false otherwise.

void beforeStart(int64_t timestamp)#

Called by EntityExecutor before each ‘start’

Parameters:: timestamp – The current timestamp from the clock. Timestamps are measured in nanoseconds.

void beforeTick(int64_t timestamp)#: Called by EntityExecutor before each ‘tick’ :param timestamp: The current timestamp from the clock. Timestamps are measured in nanoseconds.

void beforeStop()#: Called by EntityExecutor before each ‘stop’

Allocator#

enum nvidia::gxf::MemoryStorageType#

Enum representing the type of memory storage.

enumerator MemoryStorageType::kHost#: Page locked / Pinned memory allocated on the host

enumerator MemoryStorageType::kDevice#: Memory allocated on the device / GPU

enumerator MemoryStorageType::kSystem#: Memory allocated on the Heap

enum nvidia::gxf::AllocatorStage#

Specifies the stage of the allocator.

enumerator AllocatorStage::kUninitialized#

enumerator AllocatorStage::kInitializationInProgress#

enumerator AllocatorStage::kInitialized#

enumerator AllocatorStage::kDeinitializationInProgress#

class nvidia::gxf::Allocator#

Provides allocation and deallocation of memory.

virtual ~Allocator() = default#: Destructor.

virtual gxf_result_t is_available_abi(uint64_t size) = 0#

Returns whether the allocator can provide a memory block of the given size.

Parameters:: size – The size of the memory block.
Returns:: GXF_SUCCESS if the memory block can be provided, an error code otherwise.

virtual gxf_result_t allocate_abi(

uint64_t size,

int32_t type,

void **pointer,

) = 0#

Allocates a memory block of the given size and type.

Parameters:

size – The size of the memory block.
type – The type of memory to allocate.
pointer – A pointer to the allocated memory block.

Returns:

GXF_SUCCESS if the memory block was allocated successfully, an error code otherwise.

virtual gxf_result_t free_abi(void *pointer) = 0#

Frees a previously allocated memory block.

Parameters:: pointer – A pointer to the memory block to be freed.
Returns:: GXF_SUCCESS if the memory block was freed successfully, an error code otherwise.

virtual uint64_t block_size_abi() const = 0#

Returns the block size of the allocator.

Returns:: The block size of the allocator.

Expected<byte*> allocate(uint64_t size, MemoryStorageType type)#

Allocates a memory block of the given size and type.

Parameters:

size – The size of the memory block.
type – The type of memory to allocate.

Returns:

Expected<byte*> pointer to the newly created memory or error code on failure.

Expected<void> free(byte *pointer)#

Frees a previously allocated memory block.

Parameters:: pointer – A pointer to the memory block to be freed.
Returns:: Expected<void> Success or error code on failure

const char *allocator_stage_str(AllocatorStage stage) const#

Returns the string value of the allocator stage.

Parameters:: stage – The allocator stage.
Returns:: The string value of the allocator stage.

CudaAllocator#

class nvidia::gxf::CudaAllocator : public Allocator #

Provides allocation and deallocation of memory with Stream order memory allocators.

virtual ~CudaAllocator() = default#: Destructor.

virtual Expected<byte*> allocate( uint64_t size, cudaStream_t stream, ) = 0#

Allocates a memory block of the specified size and type on the given CUDA stream. It performs the allocation on the CUDA device asynchronously, meaning it does not synchronize the CUDA stream following the allocation. This is used to allocate memory on device.

Parameters:

size – The size of the memory block.
stream – Cuda stream which is used to allocate memory.

Returns:

Expected<byte*> pointer to the newly created memory or error code on failure

virtual Expected<void> free(void *pointer, cudaStream_t stream) = 0#

Frees a previously allocated memory block on given cuda stream asynchronously.

Parameters:

pointer – A pointer to the memory block to be freed.
stream – The CUDA stream that requires memory deallocation.

Returns:

Expected<void> Success or error code on failure

gxf_result_t allocate_async_abi(

uint64_t size,

void **pointer,

cudaStream_t stream,

)#

Allocates a memory block of the specified size and type on the given CUDA stream.

Parameters:

size – The size of the memory block.
pointer – A pointer to the allocated memory block.
stream – Cuda stream which on which memory is allocated.

Returns:

GXF_SUCCESS if the memory block was allocated successfully, an error code otherwise.

gxf_result_t free_async_abi(void *pointer, cudaStream_t stream)#

Frees a previously allocated memory block on given cuda stream asynchronously.

Parameters:

pointer – A pointer to the memory block to be freed.
stream – The CUDA stream that requires memory deallocation.

Returns:

GXF_SUCCESS if the memory block was allocated successfully, an error code otherwise.

Expected<size_t> get_pool_size(MemoryStorageType type) const#

Retrieve the current size of the memory pool for the specified type.

Parameters:: type – The memory type from which we derive the pool size.
Returns:: Expected<size_t> current memory pool size of specified type or error code on failure.

Receiver#

class nvidia::gxf::Receiver#

Interface for receiving entities.

virtual gxf_result_t receive_abi(gxf_uid_t *uid) = 0#

Receives the next entity from the main stage.

Parameters:: uid – The UID of the received entity.
Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual size_t back_size_abi() = 0#

The total number of entities which have recently arrived but are not yet on the main stage.

Returns:: The number of entities in the back stage.

virtual gxf_result_t peek_back_abi(gxf_uid_t *uid, int32_t index) = 0#

Peeks into back stage at a specific index.

Parameters:

uid – The UID of the entity at the index.
index – The index of the entity to peek.

Returns:

GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t sync_abi() = 0#

Moves entities which recently arrived to the main stage.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t sync_io_abi()#

Syncs I/O.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t wait_abi()#

Waits for entities to arrive.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

Expected<Entity> receive()#

Receives the next entity from the main stage.

Returns:: The received entity.

size_t back_size()#

The total number of entities which have recently arrived but are not yet on the main stage.

Returns:: The number of entities in the back stage.

Expected<void> sync()#

Moves entities which recently arrived to the main stage.

Returns:: Expected<void> Success or error code on failure

Expected<void> sync_io()#

Sync I/O.

Returns:: Expected<void> Success or error code on failure

Expected<void> wait()#

Wait for entities to arrive.

Returns:: Expected<void> Success or error code on failure

Expected<Entity> peekBack(int32_t index = 0)#

Peeks into back stage at a specific index.

Parameters:: index – The index of the entity to peek.
Returns:: The peeked entity.

Expected<void> setTransmitter(Handle<Transmitter> tx)#

Sets the transmitter to form a connection

Parameters:: tx – Handle to a transmitter to connect to.
Returns:: Expected<void> Success or error code on failure

Transmitter#

class nvidia::gxf::Transmitter#

Interface for publishing entities.

virtual gxf_result_t publish_abi(gxf_uid_t uid) = 0#

Publishes the entity with the given UID.

Parameters:: uid – The UID of the entity to publish.
Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual size_t back_size_abi() = 0#

The total number of entities which have previously been published and were moved out of the main stage.

Returns:: The number of entities in the back stage.

virtual gxf_result_t sync_abi() = 0#

Moves entities which have been published to the main stage.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t sync_io_abi()#

Sync I/O.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t pop_io_abi(gxf_uid_t *uid)#

Pops the next entity.

Parameters:: uid – The UID of the popped entity.
Returns:: GXF_SUCCESS if successful, an error code otherwise.

Expected<void> publish(const Entity &other)#

Publishes the given entity.

Parameters:: other – The entity to publish.
Returns:: Expected<void> Success or error code on failure

Expected<void> publish(Entity &other, const int64_t acq_timestamp)#

Publishes the given entity with the specified acquisition timestamp.

Parameters:

other – The entity to publish.
acq_timestamp – The acquisition timestamp of the entity.

Returns:

Expected<void> Success or error code on failure

size_t back_size()#

The total number of entities which have been published but are not yet on the main stage.

Returns:: The number of entities in the back stage.

Expected<void> sync()#

Moves entities which have been published to the main stage.

Returns:: Expected<void> Success or error code on failure

Expected<void> sync_io()#

Syncs I/O.

Returns:: Expected<void> Success or error code on failure

Expected<void> pop_io()#

Pops the next entity.

Returns:: Expected<void> Success or error code on failure

Expected<Entity> pop()#

Pops the next entity.

Returns:: The popped entity.

System#

class nvidia::gxf::System#

Component interface for systems which are run as part of the application run cycle.

virtual gxf_result_t schedule_abi(gxf_uid_t eid) = 0#

Schedules the entity with the given UID.

Parameters:: eid – The UID of the entity to schedule.
Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t unschedule_abi(gxf_uid_t eid) = 0#

Unschedules the entity with the given UID.

Parameters:: eid – The UID of the entity to unschedule.
Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t runAsync_abi() = 0#

Runs the system asynchronously.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t stop_abi() = 0#

Stops the system.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

virtual gxf_result_t wait_abi() = 0#

Waits for the system to complete execution.

Returns:: GXF_SUCCESS if successful, an error code otherwise.

Expected<void> event_notify_abi(gxf_uid_t eid, gxf_event_t event)#

Notifies the system of an event.

Parameters:

eid – The UID of the entity associated with the event.
event – The event to notify the system of.

Returns:

Expected<void> Success or error code on failure

Expected<void> schedule(const Entity &entity)#

Schedules the given entity.

Parameters:: entity – The entity to schedule.
Returns:: Expected<void> Success or error code on failure

Expected<void> unschedule(const Entity &entity)#

Unschedules the given entity.

Parameters:: entity – The entity to unschedule.
Returns:: Expected<void> Success or error code on failure

Expected<void> runAsync()#

Runs the system asynchronously.

Returns:: Expected<void> Success or error code on failure

Expected<void> stop()#

Stops the system.

Returns:: Expected<void> Success or error code on failure

Expected<void> wait()#

Waits for the system to complete.

Returns:: Expected<void> Success or error code on failure

Expected<void> event_notify(gxf_uid_t eid, gxf_event_t event)#

Notifies the system of an event.

Parameters:

eid – The UID of the entity associated with the event.
event – The event to notify the system of.

Returns:

Expected<void> Success or error code on failure

Scheduler#

class nvidia::gxf::Scheduler : public System #

An interface which extends the nvidia::gxf::System interface to create schedulers which can execute codelets.

virtual gxf_result_t prepare_abi(EntityExecutor *executor) = 0#

Prepares the scheduler for execution by providing access to the entity executor

Parameters:: executor – The entity executor to prepare for.
Returns:: GXF_SUCCESS if successful, an error code otherwise.

SchedulingTerm#

class nvidia::gxf::SchedulingTerm#

Base class for scheduling terms. Scheduling terms are used by a scheduler to determine if codelets in an entity are ready for execution.

virtual ~SchedulingTerm() = default#: Destructor.

virtual gxf_result_t check_abi( int64_t timestamp, SchedulingConditionType *type, int64_t *target_timestamp, ) const = 0#

Get the condition on which the scheduling waits before allowing execution. If the term is waiting for a time event, target_timestamp will contain the target timestamp.

Parameters:

timestamp – The current timestamp.
type – A pointer to a variable that will contain the scheduling condition type.
target_timestamp – A pointer to a variable that will contain the target timestamp if the scheduling condition is waiting for a time event.

Returns:

GXF_SUCCESS if the function was executed successfully, an error code otherwise.

virtual gxf_result_t onExecute_abi(int64_t dt) = 0#

Called each time after the entity of this term was executed.

Parameters:: dt – The current timestamp.
Returns:: GXF_SUCCESS if the function was executed successfully, an error code otherwise.

virtual gxf_result_t update_state_abi(int64_t timestamp)#

Checks if the state of the scheduling term can be updated and updates it.

Parameters:: timestamp – The current timestamp.
Returns:: GXF_SUCCESS if the function was executed successfully, an error code otherwise.

Expected<SchedulingCondition> check(int64_t timestamp)#

Checks the scheduling condition and returns the result.

Parameters:: timestamp – The current timestamp.
Returns:: An expected scheduling condition if the function was executed successfully, an unexpected error otherwise.

Expected<void> onExecute(int64_t timestamp)#

Called each time after the entity of this term was executed.

Parameters:: timestamp – The current timestamp.
Returns:: Expected<void> Success or error code on failure

Router#

class nvidia::gxf::Router#

A base class for objects which are routing messages in and out of entities.

virtual Expected<void> addRoutes(const Entity &entity) = 0#

Notifies the router about a new entity. This function is called when a new entity is added to the system. The router uses this function to set up any necessary routing for the entity’s inbox and outbox.

Parameters:: entity – The entity to add routes for.
Returns:: Expected<void> Success or error code on failure

virtual Expected<void> removeRoutes(const Entity &entity) = 0#

Notifies the router about the removal of an entity. This function is called when an entity is removed from the system. The router uses this function to clean up any routing that was set up for the entity’s inbox and outbox.

Parameters:: entity – The entity to remove routes for.
Returns:: Expected<void> Success or error code on failure

virtual Expected<void> syncInbox(const Entity &entity) = 0#

Synchronizes the inbox of an entity and prepares it for execution. This function is called when an entity is scheduled for execution. The router uses this function to synchronize the entity’s inbox, ensuring that any new messages are available for processing.

Parameters:: entity – The entity to synchronize the inbox for.
Returns:: Expected<void> Success or error code on failure

virtual Expected<void> wait(const Entity &entity)#

This function causes the router to wait until the entity’s inbox has new messages.

Parameters:: entity – The entity to wait for.
Returns:: Expected<void> Success or error code on failure

virtual Expected<void> syncOutbox(const Entity &entity) = 0#

Synchronizes the outbox of an entity after successful execution

Parameters:: entity – The entity to synchronize the outbox for.
Returns:: Expected<void> Success or error code on failure

virtual Expected<void> setClock(Handle<Clock> clock) = 0#

Sets the clock to be used to for updating the pubtime while publishing messages

Parameters:: clock – The clock to set.
Returns:: Expected<void> Success or error code on failure

virtual Expected<void> addNetworkContext( Handle<NetworkContext> context, ) = 0#

Sets the network context to be used by network router

Parameters:: context – The network context to set.
Returns:: Expected<void> Success or error code on failure

Clock#

class nvidia::gxf::Clock#

A class for keeping track of time

virtual double time() const = 0#

The current time of the clock. Time is measured in seconds.

Returns:: The current time in seconds.

virtual int64_t timestamp() const = 0#

The current timestamp of the clock. Timestamps are measured in nanoseconds.

Returns:: The current timestamp in nanoseconds.

virtual Expected<void> sleepFor(int64_t duration_ns) = 0#

Waits until the given duration has elapsed on the clock

Parameters:: duration_ns – sleep duration in nanoseconds
Returns:: Expected<void> Success or error code on failure

virtual Expected<void> sleepUntil(int64_t target_ns) = 0#

Waits until the given target time has elapsed on the clock

Parameters:: target_ns – target time duration to wait until in nanoseconds
Returns:: Expected<void> Success or error code on failure

Benchmark#

BenchmarkController#

class nvidia::gxf::benchmark::BenchmarkController#: A benchmark controller that governs the entire benchmark flow

BenchmarkPublisher#

class nvidia::gxf::benchmark::BenchmarkPublisher#

A benchmark publisher that publishes buffered benchmark messages

gxf::Handle<EntityBuffer> getEntityBuffer()#

Getter of the underlying entity buffer component

Returns:: gxf::Handle<EntityBuffer> Valid handle or error code on failure

std::vector<std::chrono::nanoseconds> &getPublishedTimestamps();#

Getter of the recorded published timestamps

Returns:: std::vector<std::chrono::nanoseconds> Vector of published timestamps

gxf::Handle<gxf::AsynchronousSchedulingTerm> getAsyncSchedulingterm();#

Getter of the associated async scheduling term

Returns:: gxf::Handle<gxf::AsynchronousSchedulingTerm> Valid handle or error code on failure

void setNumOfMessagesToPublish(uint64_t num_of_messages_to_publish);#

Setter of the number of benchmark messages to publish

Parameters:: num_of_messages_to_publish – No of messages to publish. 0 means no limit

void clearRecordedTimestamps();#: Clear the runtime state. Calling this fucntion is sufficient to reset state for a new benchmark iteration

BenchmarkSink#

class nvidia::gxf::benchmark::BenchmarkSink#

A benchmark sink that records message arrival timestamps

gxf::Expected<void> begin() override;#

Signal the start of a benchmark iteration

Return:

gxf::Expected<void> end() override;#

Signal the end of a benchmark iteration

Return:

gxf::Expected<void> reset() override;#

Reset states of the benchmark sink and the associated perf calculators

Return:

gxf::Expected<nlohmann::json> compute() override;#

Compute performance outcome for the recorded timestamps. The results are expected to be buffered in the associated perf calculators.

Return:

nlohmann::json conclude() override;#

Conclude the performance results from the associated perf calculators

Return:

std::vector<std::chrono::nanoseconds> &getReceivedTimestamps( ) override;#

Getter of the recorded received timestamps

Return:

gxf::Expected<std::vector<gxf::Handle<PerformanceCalculatorBase>>> getPerformanceCalculators( ) override;#

Getter of the associated performance calculator component handles

Return:: gxf::Expected<std::vector<gxf::Handle<PerformanceCalculatorBase>>>

void clearRecordedTimestamps() override;#

Clear the runtime state Calling this fucntion is sufficient to reset state for a new benchmark iteration