Click here to install wtop & logrep

wtop for running statistics

wtop is like "top" for your webserver. How many searches or signups are happening per second? What is the response time histogram for your static files? wtop shows you at a glance.

$ wtop
               req/s   avg   min              max     3xx     4xx     5xx    slow
  ---------------------------------------------------------------------------------
      home      0.20   126   26  .........*   543      --      --      --      --
   landing     12.60    62    0  @*........  1319    1.20      --      --    0.07
    search      0.93   597   92  **........  1764      --      --      --      --
    signup      0.20   285   22  @*.......*  1415      --      --      --      --
       css      1.63     9    0  #.........   294    0.40      --      --      --
     image      3.40     2    0  #.........   192    0.40    0.07      --      --
        js      1.63    46    0  @*........   471    0.67      --      --    0.03

• req/s is requests per second
• avg is the average response time, in milliseconds
• min/max are the fastest and slowest hit
• the characters in the middle (#@-.......) form a histogram of the response times: lowest . - o O @ # highest
• 3xx is the number of redirects and '304 Not Modified' requests per second
• 4xx & 5xx are "404 not found" and "503 error" per second.
• slow is the rate of "slow" requests (> 5000msec)

logrep for webserver log analysis (Cookbook)

wtop also comes with logrep, a powerful command-line program for ad-hoc analysis and filtering.

Spot-check page performance, errors, aggregate statistics, etc:

$ logrep -o 'status,count(*),avg(msec),min(msec),max(msec)' access.log 
200	4196	244.58	3	589
302	5	79.75	17	42
404	1	9.00	9	9
304	798	158.76	0	694

See how robot traffic rises and falls by day:

$ logrep --robots-only --output 'botname,month,day,count(*),avg(msec),dev(msec)' --sort '30:1,2,3:asc' 
Googlebot	7	20	1090	1045.97	1.65
Googlebot	7	21	771	3082.58	2.08
Googlebot	7	22	1177	1278.14	1.89
Googlebot	7	23	1134	1841.48	2.59
Googlebot	7	24	1057	1636.69	2.81
Googlebot	7	25	536	1210.78	2.10
...

Query for specific strings and conditions:

$ logrep -f "status=200,bytes>1000,msec<1000,url~Paris" -o ts,msec,bytes,url
1213574430      125     47396   /Paris-Hilton
1213574892      126     47391   /Paris-Hilton
1213579556      393     23028   /Diane-Parish
1213582392      402     19757   /Paris-Kanellakis
1213582651      530     23751   /Paris-Bennett
1213584996      366     19295   /Tristan-Paris
1213586358      114     47295   /Paris-Hilton
1213587075      227     22424   /Steve-Pariso
...