TensorRT new feature: how to add Debug Tensor?

Question

In tensorRT > 10.8, there is a new feature to add the debug tensor while network is executing. https://docs.nvidia.com/deeplearning/tensorrt/10.8.0/inference-library/advanced.html#debug-tensors

I followed the link above to add required API into my code but there is nothing related debug tensor is printed out. Can I get help with this?

Below is the steps how I changed my code:

Add mark_debug to the layer's output that I want to treat it as a debug tensor, for example

    shuffle_layer.get_output(0).name = "debug_tensor_shuffle"
    network.mark_debug(shuffle_layer.get_output(0))

Define a DebugListener class deriving from IDebugListener and implement the virtual function for processing the tensor.

    class DebugPrinter : public nvinfer1::IDebugListener {
    public:
     bool processDebugTensor(
            const void* addr,
            nvinfer1::TensorLocation location,
            nvinfer1::DataType type,
            const nvinfer1::Dims& shape,
            const char* name,
            cudaStream_t stream) noexcept override {
                std::cout << "[DEBUG] Tensor Name: " << name << std::endl;

            return true;
        }
   };

... 3. Attach the listener to IExecutionContext.

    static DebugPrinter debug_listener;
    context->setDebugListener(&debug_listener);

... I also have set DEBUG to build config.

    config.set_flag(trt.BuilderFlag.DEBUG)

where config is in type of IBuilderConfig

After running my code, the function processDebugTensor is not triggered so it didn't print any information about the debug tensor. Did I miss something? Thank you.

debug_all_the_time · Accepted Answer · 2025-05-21 18:56:04Z

2

Solved.

Need to add

context->setAllTensorsDebugState(true);

after

static DebugPrinter debug_listener;
context->setDebugListener(&debug_listener);

answered May 21 at 18:56

debug_all_the_time

6441 gold badge8 silver badges21 bronze badges

Sign up to request clarification or add additional context in comments.

Collectives™ on Stack Overflow

TensorRT new feature: how to add Debug Tensor?

1 Answer 1

Comments

Your Answer

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Comments

Your Answer

Sign up or log in

Post as a guest

Related