User Tools

Site Tools


gpu_monitor

GPU Monitor

Use nvidia-smi to monitor GPU use.

srun -p gpu nvidia-smi -q
srun -p gpu nvidia-smi -q -d UTILIZATION
srun -p gpu nvidia-smi -q -d MEMORY
srun -p gpu nvidia-smi -q -d PIDS

Using this program through srun depends on at least one core being free. You could also start a “watchdog” thread that runs it occasionally as part of your script.

gpu_monitor.txt · Last modified: 2014/04/22 18:04 by root