Overview

Systems are often difficult to debug and to understand. A typical way of gaining insight into system behavior is by inspecting execution logs. Manual inspection of logs, however, is an arduous process. This project helps this problem with Synoptic, a tool to mine a finite state machine model representations of a system from its logs. Two features distinguish Synoptic from other tools.

1. Synoptic's models preserve key event ordering invariants mined from the log, making them more accurate.
2. Synoptic uses refinement to derive the model, which is more efficient than traditional coarsening algorithms like kTails.

For users

• Try out Synoptic in your browser: http://synoptic.cs.washington.edu
• Tutorial explains how to use the various web app features

• Installation instructions

• Command line Synoptic
• Tutorial : explains how to get started with using Synoptic from scratch
• Fast-paced usage primer : includes many examples but not as detailed as the tutorial
• Help screen : usage reference

For academics

• FSE'11 conference paper : describes Synoptic's approach at length and evaluates Synoptic both formally and with a few case studies.

• SLAML'11 workshop paper : motivates and defines temporal invariants for partially ordered logs, and gives three algorithms for mining these invariants.

• SLAML'10 workshop paper : introduces Synoptic's algorithms

For developers

• Development guide : general overview for developers working on the project

• Development practices : describes the project's socio-technical processes and workflows

• GWT Development guide : describes how to get the GWT interface up and running

Example

1. Lets say you are implementing the two phase commit protocol. You wrote your code, and as part of debugging you included print statements that output a log like the following:

src : 2, dst : 0, timestamp : 16, type : propose, txid : 1
src : 2, dst : 1, timestamp : 17, type : propose, txid : 1
src : 0, dst : 2, timestamp : 18, type : abort, txid : 1
src : 1, dst : 2, timestamp : 19, type : commit, txid : 1
src : 2, dst : 0, timestamp : 20, type : tx_abort, txid : 1
src : 2, dst : 1, timestamp : 21, type : tx_abort, txid : 1
..

2. You then thoroughly tested and debugged the code, and eventually deployed it. After a week of executing thousands of two-phase commit rounds you discover an application-level inconsistency. This leads you to look at the accumulated log:

3. Uh oh. Your first challenge -- to make sense of the thousands of lines in front of you. Your second challenge -- find clues that might lead you to a potential root cause. You fire up Synoptic with:

$ java -jar synoptic.jar -r '.+ timestamp :(?<TIME>), type :(?<TYPE>), txid :(?<txId>)' -m '\k<txId>' logfile.txt -o twopc-graph

And you get the following output:

4. You realize that this graph output differs from the graph that you generated while testing your code:

You then find exactly where this discrepancy occurs in the log file, and go on from there.

History

Synoptic began in January 2010 as part of a graduate course in software engineering at the CSE department at the University of Washington.

Support

YourKit is kindly supporting the Synoptic project with its full-featured Java Profiler. YourKit, LLC is the creator of innovative and intelligent tools for profiling Java and .NET applications. Take a look at YourKit's leading software products: YourKit Java Profiler and YourKit .NET Profiler.