Monitoring Compute Node Activity

If you're interested in  tracking your batch job's CPU activity on an individual compute node, one approach is to write intermittent output from the mpstat command to a file.  You can then monitor the output from a login node while your job is running.  
*** Note: adding this monitoring command may impact the performance of your running job. ***
 

Step 1) Inside your batch job submission script, include the following prior to calling your executable or mpirun statement. In this example, t he "5" represents the interval between each run of mpstat, and the "100" is the number of times the command will be run.


unbuffer mpstat 5 100 > /u1/uaf/username/cpu_activity.log &

Step 2) After your job launches, monitor your process activity log by running the following on a login node:


tail -f /u1/uaf/username/cpu_activity.log


For more information regarding available mpstat options, view the "man mpstat" manual page.

Back to Top