3,759 questions
0
votes
0
answers
22
views
what causes geometry to be corrupted in the viewport like this after loading/meshletizing sponza.gltf? (DX12_2)
Video: https://drive.google.com/file/d/1ZOL9rXo6wNLwWAu_yjkk_Gjg1BikT7E9/view?usp=sharing
I moved the camera to show culling in all four directions
sponza: https://github.com/toji/sponza-optimized
GPU ...
1
vote
1
answer
89
views
How to correctly install JAX with CUDA on Linux when `jax[cuda12_pip]` consistently falls back to the CPU version?
I am trying to install JAX with GPU support on a powerful, dedicated Linux server, but I am stuck in what feels like a Catch-22 where every official installation method fails in a different way, ...
3
votes
1
answer
98
views
Unable to run CUDA program in google colab
I am trying to run basic CUDA program in google colab but its not giving kernel output.
Below are the steps what I tried:
Changed run type to T4 GPU.
!pip install nvcc4jupyter
%load_ext ...
- 3,351
1
vote
1
answer
56
views
How to debug cuda in Visual Studio with "step over"
I installed NVIDIA Nsight Visual Studio Edition 2025.01 in Visual Studio 2022.
I want to debug code, but I can't debug with step over(F10), The debugger always stops at a location without a breakpoint....
0
votes
1
answer
229
views
TensorFlow not detecting NVIDIA GPU (RTX 3050, CUDA 12.7, TF 2.20.0) [duplicate]
I’ve been trying to get TensorFlow to use my GPU on Windows, and even though everything seems installed correctly, it shows 0 available GPUs.
System setup
Windows 11
RTX 3050 Laptop GPU
NVIDIA driver ...
1
vote
0
answers
120
views
Why does “Command Buffer Full” appear in PyTorch CUDA kernel launches?
I’m using the PyTorch profiler to analyze sglang, and I noticed that in the CUDA timeline, some kernels show “Command Buffer Full”. This causes the cudaLaunchKernel time to become very long, as shown ...
- 143
-1
votes
0
answers
63
views
Visual Studio Code contains a visual corruption error [duplicate]
The screenshot shows the visual corruption that is plaguing my Visual Studio Code
Whenever I am running VS Code, there are small pixel deformations on the window that make it very difficult to ...
2
votes
0
answers
202
views
jax plugin configuration error: Exception when calling jax_plugins.xla_cuda12.initialize()
I am using WSL2 on windows 10. I have NVIDIA graphics card. I recently installed GPU jax using the command pip install -U "jax[cuda12]". This completed successfully, but when I run any jax ...
- 51
2
votes
1
answer
153
views
Executing a CUDA Graph from a CUDA kernel
I’m trying to launch a captured CUDA Graph from inside a regular CUDA kernel (i.e., device-side graph launch).
From the NVIDIA blog on device graph launch, it seems this should be supported on newer ...
- 1,292
0
votes
1
answer
98
views
CPU-GPU producer-consumer pattern using unified memory but GPU is in spin loop
I am trying to implement producer consumer problem in GPU-CPU. Required for some other project. GPU requests some data via Unified memory to CPU. CPU copies that data to a specific location in global ...
3
votes
1
answer
119
views
TensorRT PWC-Net Causing 2.4km Trajectory Error in iSLAM - Original PyTorch Works Fine
Problem Statement
My iSLAM system works correctly with the original PyTorch PWC-Net but produces catastrophic trajectory errors (2.4km ATE RMSE) when I replace it with a TensorRT-converted version. ...
- 705
0
votes
0
answers
148
views
TensorRT DLA Engine Build Fails for PWC-Net on Jetson NX - Missing Layer Support?
I'm converting a PWC-Net optical flow model to run on Jetson NX DLA using the iSLAM framework, but the TensorRT engine build fails during DLA optimization.
Environment
Hardware: NVIDIA Jetson NX
...
- 705
0
votes
0
answers
65
views
Using a scalar tensor as image source for a Holoscan HolovizOp
I am attempting to write my own holoscan::Operator for creating some images that should be displayed as a short video using a holoscan::ops::HolovizOp.
So I compose()-d an application flow: add_flow(...
- 1,061
0
votes
1
answer
125
views
How to correctly monitor a program’s GPU memory bandwidth utilization and SM utilization? (DCGM DRAM_ACTIVE vs in-program bandwidth differs a lot)
I want to quantitatively measure the memory bandwidth utilization and SM utilization of a CUDA program for performance analysis and regression testing.
My approach so far:
Compute the theoretical ...
- 143
2
votes
1
answer
146
views
How to define "pool" for Nvidia holoscan::ops::FormatConverterOp
I am trying to get the holoscan example "bring your own model"
https://docs.nvidia.com/holoscan/sdk-user-guide/examples/byom.html
to run, translating it from Python into CPP.
One necessary ...
- 1,061
1
vote
1
answer
268
views
How are fp6 and fp4 supported on NVIDIA Tensor Core on Blackwell?
I am writing PTX assembly code on CUDA C++ for research. This is my setup:
I have just downloaded the latest CUDA C++ toolkit (13.0) yesterday on WSL linux.
The local compilation environment does not ...
- 11
1
vote
1
answer
99
views
What is the actual maximum nesting depth of dynamic parallelism in CUDA?
Without getting into too much detail, the project I'm working on needs three different phases, each corresponding to a different kernel. I only know the number of threads needed in the second phase ...
- 308
2
votes
1
answer
67
views
How to correctly pass float4 vector to kernel using PyCUDA?
I am trying to pass a float4 as argument to my cuda kernel (by value) using PyCUDA’s make_float4(). But there seems to be some misalignment when the data is transferred to the kernel. If I read the ...
- 308
1
vote
0
answers
46
views
Unresolved extern function '__write_pipe_2' when building an OpenCL program
I'm using the OpenCL clBuildProgram() API function on a program created from a source string. The source is:
kernel void foo(int val, write_only pipe int outPipe)
{
write_pipe(outPipe, &val);
}...
- 137k
2
votes
0
answers
40
views
What do shuffle instructions do on the hardware? [duplicate]
https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#warp-shuffle-functions
https://docs.nvidia.com/cuda/parallel-thread-execution/index.html#data-movement-and-conversion-instructions-shfl-...
- 3,705
1
vote
1
answer
107
views
Doing inference for Nvidia's onnx model (PeopleNet)
I am implementing PeopleNet onnx model at my cpp application.
The following are preprocessing and postprocessing functions.
void preprocessGpuBatch(const std::vector<cv::cuda::GpuMat>& ...
- 7,346
1
vote
1
answer
188
views
(NVIDIA/nv-embed-v2) ImportError: cannot import name 'MISTRAL_INPUTS_DOCSTRING' from 'transformers.models.mistral.modeling_mistral'
My code:
from transformers import AutoTokenizer, AutoModel
model_name = "NVIDIA/nv-embed-v2"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(...
- 21
0
votes
1
answer
306
views
Distinction CuTe and NVIDIA Cutlass
I'm confused what exactly is handled by CuTe and by Cutlass.
From my understanding Cutlass handles the following:
Gemm computation of CuTe Tensors
Communication between CPU and GPU
Abstract memory ...
- 254
0
votes
0
answers
50
views
Problem about copying data from one gpu to another using torch
Recently，I upgraded my Ubuntu from 22.04 to 24.04 and found the performence of my trained deep network written by torch degrade. After debug, I found the problem is copying data from one gpu to ...
- 49
0
votes
0
answers
38
views
SLURM service in invalid state
I have a 1 gpu machine with this configuration:
This is my slurm.conf:
NodeName=TechdivAISLURM CPUs=4 Boards=1 SocketsPerBoard=1 CoresPerSocket=4 ThreadsPerCore=1 RealMemory=32096 Gres=gpu:1 State=...
- 835
-4
votes
1
answer
42
views
For a large project, how should I search for a specific kernel in nsys?
When I was analyzing a large project, there were many kernel files. I wanted to find a specific kernel in the file obtained from nsys analysis. How should I operate
0
votes
1
answer
150
views
how to use nsys when the program is running
I want to use nsys to profile a sever，how can I use nsys when the server is running，should I use nsys launch or nsys profile to restart the sever？Is there any way for me not to restart the service？
2
votes
0
answers
81
views
Reusing __shared__ data between __global__ functions
Is there an officially sanctioned way to reuse shared data between global functions?
Consider the following code https://cuda.godbolt.org/z/KMj9EKKbf:
#include <cuda.h>
#include <stdio.h>
...
- 77.4k
1
vote
0
answers
32
views
DOCA Switch Application segmentation fault
Environment:
Hardware: BlueField-2, model MBF2H516A-CEEOT
OS: Linux version 5.15.0-1060-bluefield (buildd@bos03-arm64-114)
DOCA SDK: 2.10.0087
Description:
I'm trying to run the doca_switch sample ...
- 11
-5
votes
1
answer
264
views
Is Bryce Lelbach's claim regarding progress guarantees on non-NVIDIA GPUs true?
In a talk on The C++ Execution Model, from the cppunderthesea 2024 conference, at around 44:50, NVIDIA's Bryce Adelstein Lelbach claims, that non-NVIDIA GPUs give no guarantee of threads progressing (&...
- 137k
1
vote
1
answer
117
views
Unexpected result with cublasStrmm (cublas triangular matmul)
With the following test example, the output matrix doesn't give the desired output or maybe I'm misunderstanding certain parameters:
#include <cstdio>
#include <cublas_v2.h>
#include <...
- 39.3k
0
votes
0
answers
253
views
docker-compose.yaml: services.OD Additional property device_requests is not allowed
I am trying to run a docker compose and failing. I have here a minimal reproducible example.
First with this docker-compose.yml
services:
hello-app:
image: python:3.10-slim
command: python ...
- 10.6k
0
votes
0
answers
63
views
Why is my OpenCL optimized convolution kernel slower than the naive version at higher workgroup sizes?
I'm working on a GPU-accelerated 2D convolution in OpenCL for a 2048x2048 image using a 3x3 Sobel filter. I implemented two versions of the kernel:
A naive version that uses only global memory.
An ...
- 19
0
votes
0
answers
86
views
Trouble Detecting NIC Ports in DPDK Program with ConnectX-6
I'm encountering an issue while developing a DPDK-based program using a dual-port ConnectX-6 NIC on Ubuntu 24.04. Despite following the setup instructions, my program fails to detect the NIC ports. ...
1
vote
0
answers
39
views
Why does ML.NET Image Classification with Ampere Gpu return fixed results when it otherwise works with CPU and Turing Gpu support?
We currently have a trained ResnetV250 image classification model that performs as expected on CPU and with GPU support on Turing based cards with CUDA 10.1 and cudnn 7.6.4.
When transferring this to ...
- 935
0
votes
0
answers
58
views
How to deal with a process holding nvidia GPU memory after termination？ [duplicate]
I am facing an issue with a process that holds GPU memory even after I have terminated it. Here's a detailed breakdown of the situation:
The process (a CUDA application) is running and occupies GPU ...
2
votes
0
answers
162
views
CUDA 12.8 causes #error: MFC requires use of Winsock2.h in MFC project, but works fine with CUDA 12.4
I'm working on a Windows MFC GUI application that uses CUDA for high-performance image processing.
Previously, I was using CUDA 12.0 on a system with an RTX 3090 (Ampere), and everything compiled and ...
- 163
2
votes
2
answers
97
views
Standard way of calling math functions in C when using OpenMP & its offloading feature(s)?
I am writing some code in C in which I want to add the optional ability to have certain sections of the code accelerated using OpenMP, and with an additional optional ability to have them accelerated ...
- 124
0
votes
0
answers
13
views
CrashLoopBackOff for NVIDIA VSS Pod on MicroK8s – Troubleshooting Deployment Issues
I'm working on the NVIDIA VSS project as detailed in the official documentation: NVIDIA VSS Run Guide. I'm deploying the service on a MicroK8s cluster. However, one of the pods—named similar to vss-...
0
votes
1
answer
73
views
Why is tensorflow not recognizing my gpu after installing it with anaconda
So im trying to use tensorflow with my yolov8 project but for some reason it is not recognizing my gpu. I had originally installed it using pip but i was told i should use conda instead, so i switched ...
2
votes
0
answers
43
views
NVIDIA webinar on parallel reduction gridsize
In the last example of Mark Harris' webinar I don't understand the indexing before the parallel reduction part. In "Reduction #6" the gridSize/number of dispatches was ceil[N (the size of ...
- 71
0
votes
0
answers
119
views
TensorRT Access Violation Error (0xC0000005) at nvinfer_10.dll - How to Resolve?
Environment:
OS: Windows Operating System
TensorRT Version: TensorRT-10.3.0.26
NVIDIA CUDA Version: 12.6
cuDNN Version: 9.8
GPU: RTX 3050ti laptop GPU
Issue Description:
I am encountering an "...
- 11
0
votes
0
answers
103
views
Got Segmentation fault (core dumped) after run IExecutionContext.execute_async_v3()
I used the following commands to convert an ONNX model to a TRT engine, where the input.onnx file is the original model:
polygraphy surgeon sanitize --fold-constants ./input.onnx -o output.onnx
...
- 31
1
vote
0
answers
50
views
DuplicateOutput Fails with NVIDIA set as Preferred graphic Adapter on Dual Graphics System
I have a laptop with an integrated Intel graphics card and an NVIDIA T1000 graphics card.
I set the NVIDIA card as the preferred graphic processor in the Managed 3D in NVIDIA Control Panel.
However, ...
- 21
2
votes
1
answer
228
views
Inconsistent results when training models using different GPUs
I've been trying to train a language model (text classification), our lab has two GPUs, a 4090 and a 3090. However, I encountered a perplexing phenomenon during training: the model's performance ...
- 21
2
votes
0
answers
251
views
Not able to access GPU within the docker container
I am using Ubuntu 22.04. I have nvidia-570 driver installed along with cuda 12.4 on my host machine. However, I am not able to access gpu in my container.
This is my docker-compose-file
version: '3.8'
...
0
votes
1
answer
92
views
TAO Command Not found, whenever trying to run nvidia model inside TAO TOOLKIT TENSORFLOW docker container on WSL2
Hello even though I am inside the TAO toolkit tensorflow I am still having the issue of decrypting a model with tao
docker run -it --rm -v /home/models:/workspace/model_dir -p 8888:8888 --runtime=...
- 84
1
vote
0
answers
134
views
Error response from daemon: could not select device driver "nvidia" with capabilities: [[utility compute]]
I'm trying to build a docker for realtime-whiper. The build process finishes successflly but at the end it gives this error:
Error response from daemon: could not select device driver "nvidia&...
2
votes
1
answer
103
views
Deploy TPU TF Serving Model to AWS SageMaker
I have a couple of pre-trained and tested TensorFlow LSTM models, which have been trained on Google Colab. I want to deploy these models with AWS as our entire application is deployed there.
I've ...
- 310
0
votes
0
answers
150
views
IsaacGym visualization seg faulting; Vulkaninfo giving BadMatch error with X_CreateWindow failed request
I am trying to visualize something on Isaac Gym, but when I run gym.draw_viewer(viewer, sim, True) I get a segmentation fault. I think it has something to do with Vulkan because when I run vulkaninfo ...
