Limitations of the resource logging

 

Determining CPU time and elapsed time usage under Unix is relatively straightforward. However there is no tidy and portable way of assessing memory usage and the solution used here is a little bit messy. For details of the method used and a better insight into the problems which can occur, see the inline comments in the script reslog. Broadly speaking however, the following comments apply:

• The logging script will not work well for assessing applications with a short run time or small memory footprint. Since the purpose of these scripts is to assess resource usage for large data reduction jobs we do not consider this a serious problem.
• The logging script relies heavily on the details of output from the ps command, amongst others; therefore it is likely to be sensitive to changes in operating system and environment.
• When very large jobs are running, the system can behave badly, response can be very slow, and processes can fail unexpectedly, even ones which do not themselves have extreme requirements.

For all the above reasons, CCDBIG is not a foolproof application, so its results should be regarded critically; if there are anomalous results they should be checked further (a monitoring tool like top may be of use here) before being believed!