100% GPU Utilization, 39% of Rated FP16

An author testing a matrix multiplication benchmark on an NVIDIA L4 GPU found that nvidia-smi reported 100% GPU utilization while the card performed at only 39% of its rated FP16 TFLOPS. Investigation revealed the GPU was heavily power-capped at its 72W limit, causing clocks to drop. The author explains how to use DCGM profiling metrics like SM_ACTIVE and tensor core activity to accurately diagnose workload bottlenecks and system health. Open article →