• Data Persistence
• Raw CSV Logs
• SQL Database
• References and Memos
• Process and File Tracing
• Data Plotting and Graph Drawing

Data Persistence

Raw CSV Logs

SQL Database

ParaTrac uses sqlite3 to store and retrieve trace data, which makes the trace data portable and queryable by SQL commands or APIs. There are four tables in ParaTrac trace database: env, file, proc, and syscall.

Table env

Table file stores the paths of all files that have been accessed and their mappings from iid and fid to path.

• Column Definition

iid INTEGER, fid INTEGER, path TEXT

1. iid: Instance ID of ftrace trace.
2. fid: File ID.
3. path: The absolute path of the file.

Table proc stores the process information traced during the application.

• Column Definition

iid INTEGER, pid INTEGER, ppid INTEGER, live INTEGER, res INTEGER, 
btime FLOAT, elapsed FLOAT, cmdline TEXT, environ TEXT

1. iid: Instance ID of tracer
2. pid: Process ID (not real pid, instead, hashed from pid and process time.)
3. ppid: Parent process ID (not real pid, instead, hashed from ppid and process time.)
4. live: The liveness of process after the tracing period. 0 is dead and 1 is live.
5. res: Return value of the process exit status
6. btime: Began time of the process/task
7. elapsed: Elapsed time of the process/task
8. cmdline: Command line that starts the process/task.
9. environ: Environment variables.

Table syscall is used to store every system calls conducted during tracing time.

• Column Definition

stamp DOUBLE, iid INTEGER, pid INTEGER, sysc INTEGER, fid INTEGER, res INTEGER, 
elapsed DOUBLE, aux1 INTEGER, aux2 INTEGER

1. stamp: Timestamp when the system call was called.
2. iid: Instance ID of tracer, used to identify tracers. iid is used to differentiate process with the same pid but resides in different machines.
3. pid: Process ID of the process who invoked the system call.
4. sysc: The system call number.
5. fid: The file ID.
6. res: The return value of system call.
7. elapsed: The elapsed time (or latency) of the system call.
8. aux1: Auxiliary column 1 for multiple purposes, see below:
• If sysc is read or write, then aux1 indicates the data size requested in the system call.
• If sysc is link, then aux1 is the fid that is linked to.
• If sysc is open or close, then aux1 indicates the file size (in bytes) during the open/close time.
9. aux2: Auxiliary column 2 for multiple purposes, see below:
• If sysc is read or write, then aux2 indicates the file offset (in bytes) requested in the system call.

References and Memos

Process and File Tracing

• Linux /proc filesystem: http://en.wikipedia.org/wiki/Procfs
• Linux kernel process accouting
• Enable process accouting: http://www.tldp.org/HOWTO/Process-Accounting/index.html

• Task accounting: http://www.kernel.org/doc/Documentation/accounting/
• Userspace filesystem by FUSE: http://fuse.sourceforge.net/
• Filesystem watching by inotify: http://inotify.aiken.cz/

Data Plotting and Graph Drawing

• matplotlib Python Library: http://matplotlib.sourceforge.net/
• NetworkX Python Library: http://networkx.lanl.gov/
• Add interactivity to your SVG: http://www.ibm.com/developerworks/library/x-svgint/