Structure of the output file

 

As it runs, the logging process will append to a log file (by default ccdbig_time.log). Initially a header will be written indicating some of the operating parameters and details of the machine which is being run on. One subsequent line will be written for each program as it is run, giving the size of the image, the number of objects in the data frame, the name of the application being run, the CPU time and elapsed time in seconds, the maximum total (virtual memory) size and maximum resident (real memory) size of the process in kilobytes, and the remaining disk space. A section of a log file could therefore look something like this:

                            Elapsed(s)   CPU(s)   RSS(k)  Size(k) Diskfree(M)
SunOS cass58 5.6 Generic sun4u sparc SUNW,Ultra-1
 10:32am  up 19 day(s), 22:21,  0 users,  load average: 0.04, 0.14, 0.12
Tue Jul  7 10:32:57 BST 1998
714     /data/cass58a/mbt/scratch
 
Variance:           FALSE
Keepard:            FALSE 
Biasframe:          TRUE
Normalisation:      TRUE
Data type:          _REAL
Logging interval:   15

Pixels Objects     Command
                            Elapsed(s)   CPU(s)   RSS(k)  Size(k) Diskfree(M)
   3000   1012     ccdbgen:        403      349    39024    42024        2910
   3000   1012     display:          3        2    19496    45984        2910
   3000   1012    makebias:         43       36    39440    42840        2879
   3000   1012      debias:        593       54   174208   179784        2715
   3000   1012     display:         17        2    14664    41944        2715
   3000   1012    makeflat:        179       60    92384   101632        2635
   3000   1012     ardmask:          6        3    94800   135424         140
   3000   1012     flatcor:         17        4    89032   101616        2635
   3000   1012     findobj:         25       16    35944    42760        2635
   3000   1012     tranndf:        187      167    95568   105360        2600
   3000   1012 makemosnorm:        600      515    51704    59256        2567
   3000   1012  makmoscomb:         47        7     4920    12128        2546

The meanings of the values logged are as follows:

• Pixels: The test data frames are $N \times N$ pixels in size. This number is N.
• Objects: This is the number of objects placed in the star field; each frame occupies a subsection of this area and will contain around 0.4 times as many.
• Command: Name of the application to which the statistics apply. This is a label determined by the ccdbig script and in general is the name of a CCDPACK or KAPPA application, except for makemosnorm and makemoscomb, which are the normalisation and combination parts respectively of MAKEMOS (see the ccdbig script for details).
• Elapsed: This is the ``wall-clock'' time in seconds between the start of the process in question and its termination. Large data reduction processes are typically I/O bound, so for most of the applications the discrepancy between this and the CPU time will be accounted for mainly by waiting for I/O, either simple writes and reads to and from disk files, or swapping virtual memory between disk and core. On a heavily loaded system however it will reflect processor usage by other processes as well.
• CPU: Number of seconds of system plus user time spent by the program executing.
• RSS: The maximum amount of real memory used (resident set size) in kilobytes. This will not exceed the physical memory installed in the machine.
• Size: The maximum amount of real plus virtual memory in kilobytes used by the whole process. This can be as large as the whole of real memory plus the whole of swap space. As it approaches and exceeds the physical memory available (where availability takes account of requirements of other processes running concurrently) the machine's performance is likely to degrade significantly as much time is spent swapping pages to and from disk.
• Diskfree: The number of megabytes still available in the filesystem holding the data files. This is not a direct measure of the resource usage of the programs, but it is an important value to monitor, since if it falls to zero the programs will probably fail with a write error. If it falls to within a few percent of the total size of the disk then disk performance can sometimes be degraded.