PerfMetrics

System Performance Monitoring for Windows

PerfMetrics is a free/open source performance monitoring system for Windows. It collects, stores, and graphs performance counters from remote Windows machines.

It uses WMI to collect server stats and RRDTool for data storage and graphing.

A set of .PNG images is generated along with an HTML template for viewing reports:

sample screenshot

sample graphs

Download:

Latest Version: 1.001 (08/05/2009) - perfmetrics_1.001.zip
(for the very latest, checkout from SVN instead)

Prerequisites:

• Python 2 (2.6+)
• Python for Windows extensions

After you have everything installed, run wmi_test.py to make sure your environment is configured correctly.

Using:

1. create a configuration file (config.ini)
2. run the rrd_maker.py script to create your RRD files (data stores)
3. schedule the perfmetrics.py script to run at a set interval (Windows Task Scheduler works)
• the script collects remote performance counters, populates the data stores, and generates a graph
• .png graph images are written to the report/graphs directory each time it runs
4. run the report_maker.py script to create your report templates
• a report is created for each server you are monitoring
• view the html reports in your browser (located in report directory)
• images are refreshed at set interval

Components:

PerfMetrics consists of the following Python scripts:

• perfmetrics.py - main program. collects, stores, and plots data
• report_maker.py - generates html templates for viewing reports
• rrd_maker.py - creates rrd files (data stores)
• wmi_test.py - verifies environment setup

These use a configuration file named config.ini, where various options can be set.

Collectors:

Processor:

• cpu_util - utilization avg of all processors/cores
• cpu_util_maxcore - utilization max of all processors/cores
• cpu_queue_length - processor run queue length
• cpu_context_switches - context switches per sec

• net_bits_total - total bits per sec
• net_bits_in - input bits per sec
• net_bits_out - output bits per sec
• net_bytes_total - total bytes per sec
• net_bytes_in - input bytes per sec
• net_bytes_out - output bytes per sec

• mem_available_byte -
• mem_cache_bytes -
• mem_committed_bytes -
• mem_pages -
• mem_page_faults -

Disk:

• disk_queue_length_avg -
• disk_queue_length_current -
• disk_bytes_transferred -

System:

• uptime - hours of uptime

(more coming...)

Configuration Options (config.ini):

• global section
• interval: time interval for collecting data (seconds)
• graph_mins: comma separated list of minutes displayed on each graph. a graph is created for each minute value (a set for each machine/collector)
• machine sections
• create a section for each machine you wish to monitor
• collectors: comma separated list of data collectors

Sample config.ini file:

[global]
interval: 60
graph_mins: 60,180,1440
graph_width: 350
graph_height: 150
log_results: on
rrd_results: on

[machine1]
host: machine1.foo.com
username: username
password: secret
collectors: cpu_util

[machine2]
host: machine2.foo.com
username: username
password: secret
collectors: cpu_util,net_bits_total,mem_available_bytes

Round Robin Database Storage (RRD):

RRDtool is the Open Source industry standard, high performance data logging and graphing system for time series data. RRD stands for "Round Robin Database". It stores data in a very compact way that will not expand over time. RRDTool is a really good back-end for storing time-series data. The neat thing about this data storage is its "round robin" nature. A fixed binary file is created, which never grows in size over time. As you insert more data, it is inserted into each span. As results are collected, they are averaged and rolled into successive time spans. It makes a much more efficient system than using your own complex data structures, relational database, or file system storage.

Help/Tips:

• PerfMetrics does not come with it's own scheduler. You must use an external task scheduler to run the script at set intervals. Windows task scheduler can be used for this but runs at a minimum interval of 60 secs. 60 second interval seems to work well, but if you want to collect data faster, you will have to provide your own scheduler.
• PerfMetrics is multithreaded, so stats are collected from remote servers in parallel.
• If you change the interval at which you collect data, you must create new RRD's. The RRD data storage is based on the approximate interval at which you will insert data.
• RRDs are binary files of a fixed size. They will not grow over time.
• The server usernames/passwords you provide must have administrative rights on the remote machine.
• a log is written to: perf.log each time it runs

Copyright (c) 2009 Corey Goldberg - http://www.goldb.org
contact me with questions, bugs, patches, comments, etc