My favorites | Sign in
Logo
          
Search
for
Updated Apr 13, 2009 by jwzurawski
SNMPMA  
perfSONAR-PS SNMP MA Service Information

Introduction

The perfSONAR-PS SNMP based Measurement Archive (SNMP MA) is able to expose data collected via variables from the Simple Network Management Protocol (SNMP) protocol found on networked devices and stored in Round Robin Databases (RRD) archives. The measurements are collected through external means, normally through software such as MRTG, CACTI, or Cricket, and is commonly stored in RRD archives. External access to commonly requested data items, remains challenging due to physical and political boundaries.

The SNMP MA provides a simple interface that is cable of exposing these files after basic configuration for consumption by perfSONAR client applications and services alike. The Web Service front end provides a uniform method of access using the perfSONAR XML protocols and delivers the data in an unambiguous manner, thus eliminating the mystery associated with the backend storage.

Installation

There are several ways to install this service, the popular methods are listed below. Note that these are from the point of view of installing a particular version - this may or may not be the most recent.

Prerequisites

The SNMP MA is packaged as an RPM - a binary installation format that is geared towards use on Red Hat linux based systems. Examples include Fedora, Red Hat Enterprise Linux, CentOS, White Box, Scientific Linux, and others.

These Linux distributions normally rely on Package Management tools to install and maintain software. Examples of these tools include up2Date, YUM, and APT-RPM. Internet2 has created a repository to house the perfSONAR-PS software, related performance tools, and hard to find prerequisites. We encourage adopters of these tools to install a simple RPM package that gives the necessary configuration information for the YUM and APT-RPM tools. The RPM can be found here:

wget -c http://software.internet2.edu/Internet2-repo-0.1-2.noarch.rpm

To install this RPM first import the signing key:

rpm --import http://software.internet2.edu/rpms/RPM-GPG-KEY-Internet2

Then install the RPM:

rpm -ivh Internet2-repo-0.1-2.noarch.rpm

It is also recommended that an additional repository be installed for hard to find prerequisite software that may not be available in traditional repositories used by the Linux distributions. Two regarded repositories are:

Each site provides configuration instructions and packages similar to the Internet2 repository.

Other Forms of Linux

If using a package management system is not an option, or if your system is not RedHat based (e.g. Debian, Gentoo, etc.) it is still possible to install this software and related requirements by hand. Please see the TarPackage or Subversion sections for more details. This tool uses the following Perl libraries, libraries, and external tools to function:

  • Perl Libraries
    • Config::General
    • Cwd
    • DBI
    • Data::UUID
    • Date::Manip
    • Digest::MD5
    • Error
    • Exporter
    • File::Basename
    • File::Temp
    • Getopt::Long
    • HTTP::Daemon
    • IO::File
    • LWP::Simple
    • LWP::UserAgent
    • Log::Dispatch
    • Log::Dispatch::FileRotate
    • Log::Log4perl
    • Module::Load
    • Net::Ping
    • Params::Validate
    • RRDp
    • Socket
    • Sys::Hostname
    • Time::HiRes
    • XML::LibXML
  • Libraries
    • LibXML2
  • Tools
    • RRDTool

YUM

Like Apt, YUM installation also manages the task of finding dependencies. The following command is all that is needed:

yum install perl-perfSONAR_PS-SNMPMA-3.1-1.pSPS.noarch.rpm

There may be several prerequisites to install depending on the age of your system. To ensure that all are found, please be sure to install one of the alternative repositories above.

APT-RPM

Installation through APR-RPM is easy, simply let the package manager do the work of managing the dependencies:

apt-get update
apt-get install perl-perfSONAR_PS-SNMPMA-3.1-1.pSPS.noarch.rpm

There may be several prerequisites to install depending on the age of your system. To ensure that all are found, please be sure to install one of the alternative repositories above.

Standalone RPM

If you do not wish to install the Internet2 Repository package, you may simply find the appropriate RPM for your architecture without using a package manger. The first step is to install the signing key:

rpm --import http://software.internet2.edu/rpms/RPM-GPG-KEY-Internet2

Next download the service, for example if you are on an i386 architecture:

wget -c http://software.internet2.edu/rpms/i386/main/RPMS/perl-perfSONAR_PS-SNMPMA-3.1-1.pSPS.noarch.rpm

To install without package manager help:

rpm -ivh perl-perfSONAR_PS-SNMPMA-3.1-1.pSPS.noarch.rpm

Note that this RPM may need packages to install correctly: if they are not found on your system manual download and install will be required. An alternate method is to still use a package manager in a local way (YUM only):

yum localinstall perl-perfSONAR_PS-SNMPMA-3.1-1.pSPS.noarch.rpm

This will still search the available repositories for any necessary packages.

Tar Package

All RPM packages are also available in source packages that may be installed and used in the typical Unix/Linux fashion. There are helper scripts included that may be used to download the necessary prerequisite software packages:

wget -c http://software.internet2.edu/source/perl-perfSONAR_PS-SNMPMA-3.1-1.tar.gz
tar -zxfv perl-perfSONAR_PS-SNMPMA-3.1-1.tar.gz
make install
cd perl-perfSONAR_PS-SNMPMA-3.1-1/scripts
./prepare_environment.sh
./install_dependencies.sh

These two included scripts will configure the machine's users and startup scripts as well as contact CPAN for any necessary perl libraries.

In addition to perl requirements, there are other software requirements that should be installed. These are not controlled via the dependency scripts due to differences in package management. The tools are listed below:

  • RRDTool
    • Source is available here
    • Package management names:
      • Debian/Ubuntu: rrdtool, librrd2, librrd2-dev, librrds-perl, librrdp-perl
      • RedHat Systems: rrdtool, rrdtool-devel, perl-rrdtool
  • LibXML2
    • Package management names:
      • Debian/Ubuntu: libxml2, libxml2-dev, libxml2-utils
      • RedHat Systems: libxml2, libxml2-devel

Subversion

Software may be downloaded directly from our anonymous subversion server. This installation method allows for easy upgrades between versions and access to bugfixes immediately. There are two ways to download from our server, one that retrieves all software and another that retrieves only selective services.

To download the entire release:

svn checkout http://anonsvn.internet2.edu/svn/perfSONAR-PS/tags/RELEASE_3.1 perfSONAR-PS

The instructions are the same here as for the Tar package. Note that installation may require pre-requisite software. Read the preceding sections for necessary items and suggestions on how to install and configure them. When you wish to update the checkout (to retrieve new development or bugfixes):

cd perfSONAR-PS
svn update

If a single service is all that is required, a similar method is used but also note that the service must have access to a "Shared" directory that contains perfSONAR-PS libraries:

svn checkout http://anonsvn.internet2.edu/svn/perfSONAR-PS/tags/RELEASE_3.1/perfSONAR_PS-SNMPMA/
svn checkout http://anonsvn.internet2.edu/svn/perfSONAR-PS/tags/RELEASE_3.1/Shared

The instructions are also the same as above:

cd Shared
svn update

and

cd perfSONAR_PS-SNMPMA
svn update

Configuration

The following sections detail the necessary configuration of this service.

Configuration File Format

<port 9990>
    <endpoint /perfSONAR_PS/services/SNMPMA>
        <snmp>
            default_resolution              300
            rrdtool                         /usr/bin/rrdtool
            
            metadata_db_external            none
            metadata_db_external_source     

                                                        # Sample Cacti configuration
#            metadata_db_external                   cacti
#            metadata_db_external_source            /etc/cacti/cactid.conf

                                                        # Sample cricket configuration
#            metadata_db_external                   cricket
#            metadata_db_external_cricket_config    /home/cricket/cricket-config
#            metadata_db_external_cricket_cricket   /home/cricket/cricket
#            metadata_db_external_cricket_data      /home/cricket/cricket-data
#            metadata_db_external_cricket_home      /home/cricket

            metadata_db_file                /var/lib/perfsonar/snmp_ma/store.xml
            metadata_db_type                file

                                                        # Sample dbxml configuration
#            metadata_db_file                       snmpstore.dbxml
#            metadata_db_name                       /var/lib/perfsonar/snmp_ma/xmldb
#            metadata_db_type                       xmldb
                                                        
	                                            # 720 minute (12hr) registaration window
            ls_registration_interval        720
            enable_registration             1
                                                # fake LS instance (change to something real)
            ls_instance                     http://localhost:9995/perfSONAR_PS/services/hLS
            
				                                # Service options (please customize)
            service_accesspoint             http://localhost:9990/perfSONAR_PS/services/SNMPMA
            service_description             ORG SNMP MA (CITY, STATE, COUNTRY)
            service_name                    ORG SNMP MA
            service_type                    MA
        </snmp>
        disable                             0
        module                              perfSONAR_PS::Services::MA::SNMP
    </endpoint>
</port>
	                                            # 720 minute (12hr) registaration window
ls_registration_interval	                720
                                                # fake LS instance (change to something real)
ls_instance                                 http://localhost:9995/perfSONAR_PS/services/hLS

disable_echo	                            0
	                                            # Root hints info, url + local copy location
root_hints_url	                            http://www.perfsonar.net/gls.root.hints
root_hints_file	                            /var/lib/perfsonar/snmp_ma/root.hints
	                                            # Daemon management, shouldn't need changing
reaper_interval	                            20
max_worker_lifetime	                        300
max_worker_processes	                    30
pid_dir                                     /var/run
pid_file	                                snmp_ma.pid

Daemon Configuration File Options

There are several Global configuration items, an explanation of each follows:

Configuration Name Required Type Description
ls_registration_interval No Integer (Time in Minutes) How often the service registers with a Global Lookup Service
disable_echo Yes Boolean, 0 or 1 Turn on or off the echo response (recommended to keep this on)
root_hints_url Yes String (URL) Location of a root hints file (default is http://www.perfsonar.net/gls.root.hints)
root_hints_file Yes String (file path) Location to locally store the root hints file
ls_instance No String (URL) LS instance to register with
reaper_interval Yes Integer (Time in Seconds) Internal daemon setting to manage processes
max_worker_lifetime Yes Integer (Time in Seconds) Internal daemon setting to manage processes
max_worker_processes Yes Integer Internal daemon setting to manage processes
pid_dir Yes String (file path) Directory to store PID files
pid_file Yes String (file name) Name of PID file for this service

There are several configuration items that are directly related to the service, an explanation of each follows:

Configuration Name Required Type Description
port Yes Integer Port this service will listen on
endpoint Yes String (URL path) Endpoint path this service will listen on
default_resolution Yes Integer Default resolution for RRD tool archives
rrdtool Yes String (file path) Path to rrdtool binary
ls_registration_interval Yes Integer (Time in Minutes) How often the service registers with a Global Lookup Service
ls_instance No String (URL) LS instance to register with
enable_registration Yes Boolean, 0 or 1 Allow the service to register with an LS instance
metadata_db_name No String (file path) Name of the directory that houses the database
metadata_db_file Yes String (file name) File name of a database file
metadata_db_type Yes String Either file or xmldb
metadata_db_external Yes String Either none, cacti, or cricket
metadata_db_external_source No String Location of external configration file, Used only for Cacti currently
metadata_db_external_cricket_config No String (directory path) Path to cricket config, Used only for cricket
metadata_db_external_cricket_cricket No String (directory path) Path to cricket directory, Used only for cricket
metadata_db_external_cricket_data No String (directory path) Path to cricket data, Used only for cricket
metadata_db_external_cricket_home No String (directory path) Path to cricket home directory, Used only for cricket
service_accesspoint Yes String (URL) URL used to contact this service
service_description Yes String Description of this service
service_name Yes String Name of this service
service_type Yes String Type of this service (normally MA)
disable No Boolean, 0 or 1 Is this service active or not (normally 1)
module Yes String (module name) Module name that controls this service (normally will not change)

Logger Configuration Options

The following is an explanation of the logging file format.

Configuration Name Suggested Value Description
log4perl.logger.perfSONAR_PS INFO, A1 Log level to use (DEBUG, INFO, CRITICAL, FATAL) and the name of the logger
log4perl.appender.A1 Log::Dispatch::Screen, Log::Dispatch::Syslog, Log::Dispatch::FileRotate Where to direct logging messages
log4perl.appender.A1.facility local5 Syslog facility to handle log messages. Use with the Syslog option
log4perl.appender.A1.filename /var/log/perfsonar/snmp_ma.log Location of logging file. Use with the FileRotate option
log4perl.appender.A1.max 5 Maximum log files to maintain. Use with the FileRotate option
log4perl.appender.A1.size 16777216 Size of the logging file before rotation. Use with the FileRotate option
log4perl.appender.A1.mode append Append to a file instead of overwriting.
log4perl.appender.A1.layout Log::Log4perl::Layout::PatternLayout Use a specific patter generator
log4perl.appender.A1.layout.ConversionPattern %d (%P) %p> %F{1}:%L %M - %m%n Pattern for log files

Suggested Configuration Changes

The SNMP MA comes with a base configuration via the installation procedure. This is meant to allow for immediate testing as well as serve as a guide for customizing the service.

There are several files that may be adjusted manually to configure this service for a specific use case:

Daemon Changes

Contains information that identifies the service uniquely and allows it to run in a given environment. There are several options that should be changed:

There are also several options that may be changed for a configuration:

The rest of the items are normally not to be adjusted.

Logging Changes

Contains information related to logging, there are three modes:

The rest of the items are normally not to be adjusted.

Metadata Configuration Changes

The metadata configuration is stored at:

/var/lib/perfsonar/snmp_ma/store.xml: 

This file represents the explanation of the data that is being offered by the service. It is written in XML and is configured for a localhost environment with a single empty RRD file. This file may be modified by hand to add new existing RRD files that describe specific interfaces. It is also possible to configure your service to use external collection tools such as Cacti or Cricket.

Service Control

The service comes with init scripts that may be used to start and stop operation:

/etc/init.d/snmp_ma start
/etc/init.d/snmp_ma stop
/etc/init.d/snmp_ma restart

The service can also be run in a manual fashion (useful for debugging):

perl bin/daemon.pl --verbose --conf=etc/daemon.conf --logger=etc/daemon_logger.conf

Before doing this, be sure to edit the logging to enable Screen debugging.

Service Validation

The service comes with a client application and several test messages that can be used to test functionality. There are many messages provided as examples, to test the service there are four that are recommended:

etc/requests/EchoRequest.xml
etc/requests/MetadataKeyRequest-1.xml 
etc/requests/DataInfoRequest-1.xml 
etc/requests/SetupDataRequest-1.xml

These four will test the major functions of this service. To test:

  1. Ensure the service is started, it helps if you have configured logging to use the "Screen" so you may monitor the service's activities.
  2. Run the client:
  3.   perl bin/client.pl URL FILENAME
  4. Compare the results of the test with the comments in each file. Note there may be some differences (e.g. id values, key values).
  5. If everything looks OK, return logging to normal and restart service via the init system.

Service Maintenance

Subscribe to the user mailing lists if you are seeking help on some problem related to this software. Announcements regarding availability of new releases will also be sent to these locations:

List URL Audience
perfSONAR Users Group https://mail.internet2.edu/wws/info/perfsonar-user perfSONAR Users Group
perfSONAR Announcements https://mail.internet2.edu/wws/info/perfsonar-announce perfSONAR Announce (Low Volume)
Performance Node Users https://mail.internet2.edu/wws/info/performance-node-users Performance Node Users
Performance Node Announcements https://mail.internet2.edu/wws/info/performance-node-announce Performance Node Announce (Low Volume)

Also if you are using package management software and have installed the Internet2 software repository, a typical upgrade should download new releases:

yum update

or

apt-get dist-upgrade

Finally, check the following websites for news of software releases:

Last Updated

$Id$


Sign in to add a comment
Hosted by Google Code