|
NPToolkit
perfSONAR Network Performance ToolkitTOC
IntroductionThe perfSONAR Network Performance Toolkit (NPToolkit) is a customized version of a Knoppix Live-CD bootable disk. The software consists of an iso disk image that can be used in one of two ways:
After performing either step, the result is a fully configured system capable of running and using many advanced network tools and methodologies to serve as either a testing point or a means to solve performance issues. BackgroundThe original idea for the NPToolkit came from the Network Performance Workshop series sponsored by Internet2. The old format of this workshop consisted of a day of tool installation (e.g. installing the Web100-enhanced kernel and other performance tools) followed by some time to test and use them. This particular format did not allow enough time to get to the primary reason many participants choose to sign up: solving network performance problems. To lower the bar on this complex topic, this product offers a complete network performance suite pre-installed (and in many cases pre-configured) that is ready to be dropped directly into a network operations center (NOC) of any size. With the integration of perfSONAR tools, the results of both passive and active measurements can be shared in a federated environment with your peers enabling the early detection of performance issues. PurposeWhen serving as a measurement point, a use case adopted by Large Hadron Collider (LHC) affiliated Tier 2 sites in the United States, the disk becomes a part of a global registry of perfSONAR tools. Affiliated sites and other interested parties may perform any number of tests (on demand, or in some cases regularly scheduled) to the installation including:
When used in the role of a network debugging infrastructure, the various tools can be used to methodically find, diagnose, and aid in the correction of network performance issues. More InformationFor information regarding upcoming releases or important notices concerning quality assurance and security we encourage subscription to our announcement mailing list:
This list is expected to be low volume and subscription is open. To get (or contribute) help within the Performance Node community we encourage subscription to our users mailing list:
Non-subscribing users may still post without being a member. This list will be of a higher volume than the announcement list, and should be used as the first stop for help regarding installation, configuration, and use of the Performance Node software. A final piece of information is the developer's issue tracker which can be used to find, track, and file bugs related to this software: If your solution is not easily solved by members of the mailing list, we encourage the use of this tool to inform the developers of potential problems. Note that before filing a bug it is customary to search open issues to be sure something isn't being duplicated. Also be sure to use a the Component-NPToolkit tag or indicate somewhere in the report that the bug is related to this product. Previous ReleasesThe original version of the perfSONAR Network Performance Toolkit was released in the summer of 2008, at the Joint Techs conference in Lincoln Nebraska. Beta 1
Beta 2
Release
Installation and ConfigurationPlease visit NPToolkitQuickStart for a complete set of instructions regarding installation and configuration. Disk ContentsThe main feature is a Web100-enhanced Linux 2.6.23-9 kernel that facilitates the use of several network tools without the need to configure and compile this advanced component. This disk features several performance tools pre-installed and configured to a working state:
Additional supporting components include:
Performance ToolsThe following performance tools are packaged on the perfSONAR Network Performance Toolkit. Each tool comes configured to function with default options, although customization is possible and encouraged. Certain tools have begun the steps necessary to participate in the perfSONAR framework by exporting data and utilizing the information services. We expect many more will follow in the future as adoption continues to grow. BWCTLBWCTL is a command line client application and a scheduling and policy daemon that wraps tools such as Iperf, thrulay, and nuttcp. Currently BWCTL wraps these tools by actually executing the respective command line program on the system. The bwctl client application works by invoking instances on the two test endpoint systems. BWCTL will work as a 3-party application. The client can arrange a test between two servers on two different systems. If the local system is intended to be one of the endpoints of the test a local instance is not required, BWCTL will detect that there is no local server and execute the required functionality directly. The daemon manages and schedules the resources of the host on which it runs. The perfSONAR Network Performance Toolkit contains a release candidate of version 1.3 of BWCTL. The daemon is started by default with a wide open authentication and resource protection scheme. It is recommended that the deploying party review this to ensure it matches local security policies. CactiCacti is a complete network graphing solution designed to harness the power of RRDTool's data storage and graphing functionality. Cacti provides a easy to use interface that offers easy ways to pool data sources, manage the data, and graph the results. The perfSONAR Network Performance Toolkit contains version 0.8.7b of Cacti. Cacti is fully installed and will start by default. It can be reached by visiting http://HOST_OR_ADDRESS/admin/cacti*. The interested use can follow the instructions in Cacti Configuration to poll a local or remote SNMP enabled network device to gather passive measurements. The SNMP Measurement Archive is configured by default to read the cacti data and deliver this data. NDTThe Network Diagnostic Tool (NDT) is a client/server program that provides network configuration and performance testing to a user's desktop or laptop computer. The system is composed of a client program (command line or java applet) and a pair of server programs (a web server and a testing/analysis engine). Both command line and web-based clients communicate with a Web100-enhanced server to perform these diagnostic functions. Multi-level results allow novice and expert users to view and understand the test results. The perfSONAR Network Performance Toolkit contains version of 3.4.4a NDT. All installation, including integration of the Web100 enabled linux kernel, is complete and operational as the disk comes online. Minor configuration is necessary, but is performed as a part of the general configuration steps described in Step 2 - Console Configuration. NPADThe NPAD diagnostic server, Pathdiag, is designed to easily and accurately diagnose problems in the last-mile network and end-systems that are the most common causes of all severe performance degradation over long end-to-end paths. The overall goal is to make the test procedures easy enough and the report it generates clear enough to be suitable for end-users who are not networking experts. The perfSONAR Network Performance Toolkit contains version of 1.5.4 NPAD. All installation, including integration of the Web100 enabled linux kernel, is complete and operational as the disk comes online. Minor configuration is necessary, but is performed as a part of the general configuration steps described in Step 2 - Console Configuration. OWAMPOWAMP is a command line client application and a policy daemon used to determine one way latencies between hosts. It is an implementation of the OWAMP protocol as defined by http://www.rfc-editor.org/rfc/rfc4656.txt. With roundtrip-based measurements, it is hard to isolate the direction in which congestion is experienced. One-way measurements solve this problem and make the direction of congestion immediately apparent. Since traffic can be asymmetric at many sites that are primarily producers or consumers of data, this allows for more informative measurements. One-way measurements allow the user to better isolate the effects of specific parts of a network on the treatment of traffic. The perfSONAR Network Performance Toolkit contains version 3.0c of OWAMP. The daemon is started by default with a wide open authentication and resource protection scheme. It is recommended that the deploying party review this to ensure it matches local security policies. ThrulayThrulay is used to measure the capacity, delay, and other performance metrics of a network by sending a bulk TCP or UDP stream over it. Special features of thrulay include:
The perfSONAR Network Performance Toolkit contains version 0.9 of thrulay. Thrulay is offered as an add-on tester when used with BWCTL and the thrulay daemon is not started as a service by default. perfSONAR PS ServicesperfSONAR-PS is a set of independent software services that implement the perfSONAR protocols for network performance monitoring. perfSONAR-PS services are designed to be compatible with all other perfSONAR software that implements the perfSONAR protocols. perfSONAR-PS is able to federate between deployments, particularly those that span multiple domains, making the job of solving end-to-end performance problems on paths crossing several networks much easier to address. The perfSONAR-PS services provide Web Services (WS) based interfaces into already deployed network monitoring infrastructures and act as an intermediate layer, between the performance measurement tools and the diagnostic or visualization applications. The targeted audience for these services is network Network Operation Centers (NOCs) at Universities and Regional networks, however these services are broadly useful to the general public at large. The perfSONAR-PS software suite is developed entirely in the Perl programming language, taking full advantage of numerous language features and benefits including the Comprehensive Perl Archive Network (CPAN) distribution system. This software manager makes perfSONAR-PS the ideal choice for integration into typical NOC environments. The following perfSONAR-PS services are included on the Network Performance Toolkit. Each version is a pre-release of the upcoming 0.10 series. Lookup ServiceThe perfSONAR-PS Lookup Service (LS) addresses the always challenging problem of resource registration and discovery for the perfSONAR framework. Service instances that manage datasets are only useful when they can be contacted by consumers. Consumers can only function when there is data available. To manage these problems in a dynamic environment such as perfSONAR, it is necessary to register, maintain, and query for the services that may contain interesting data. The advent of the perfSONAR Global Lookup Service (gLS) now delivers a world wide view of all available perfSONAR and selected performance tools. Each LS instance is able to take an inventory of locally registered resources and share these with a well established infrastructure of globally deployed gLS services. Through the use of established APIs service and client applications alike can gain access to this powerful information source. The perfSONAR Network Performance Toolkit contains a version of this software that starts up on disk boot. Additional configuration is provided via Step 2 - Console Configuration. PingER Measurement Archive and Measurement PointThe PingER service is an evolution of the PingER project with more than 10 years experience in collecting and analyzing network performance across the world. The perfSONAR-PS PingER service is composed of both a storage backend (Measurement Archive) and measurement frontend (Measurement Point) to conduct and store ping measurements and to make available such data for consumption by interested parties. Network characteristics supported include availability, latency and jitter, which provide a broad spectrum of determining end-to-end network performance. The perfSONAR Network Performance Toolkit contains a version of this software that starts up on disk boot. Additional configuration is provided via Step 2 - Console Configuration. A default set of ping-able hosts are included on the installation, additional configuration can be performed via PingER Configuration. perfSONAR BUOY Measurement ArchiveThe perfSONAR-BUOY Measurement Archive service (pSB MA) exposes active measurement data, making the results available through perfSONAR web services interfaces. All performance tests are performed via the BWCTL tool (throughput measurements) or the OWAMP tool (one way delay) and currently consist of regularly-scheduled tests to a configurable list of source and destination hosts. Data collected is stored in a MySQL database. Active measurements between known hosts are particularly valuable when assuring connectivity, availability, and quality of the network. perfSONAR-BUOY allows for the easy configuration of a "mesh" of tests to hosts equipped with BWCTL or OWAMP. Based on a configurable schedule, the tests will be conducted autonomously and stored for consumption via the perfSONAR-BUOY interface as well as presentation through included web scripts. Using the same XML protocols as the the other perfSONAR services, perfSONAR-BUOY provides uncomplicated access methods to retrieve the data in an unambiguous manner, thus eliminating the mystery associated with the backend storage. The perfSONAR Network Performance Toolkit contains a version of this software that starts up on disk boot. Additional configuration is provided via the Console Configuration step. This software requires configuration via perfSONAR-BUOY Configuration before it can be used. SNMP Measurement ArchiveThe perfSONAR-PS SNMP based Measurement Archive (SNMP MA) is able to expose data collected via variables from the Simple Network Management Protocol (SNMP) protocol found on networked devices and stored in Round Robin Databases (RRD) archives. The measurements are collected through external means, normally through software such as MRTG, Cacti, or Cricket, and is commonly stored in RRD archives. A common way to diagnose network problems is to gain external access to commonly requested data items (such as interface utilization, errors, discards, etc.) remains challenging due to physical and political boundaries. The purpose of the SNMP MA is to deliver these items transparently and securely. The perfSONAR Network Performance Toolkit contains a version of this software that starts up on disk boot. Additional configuration is provided via Step 2 - Console Configuration. This software requires configuration via Cacti Configuration before it can be used. Supporting SoftwareThe following extra software packages are installed on the perfSONAR Network Performance Toolkit to aid the performance tools in the performance of their duties. Each piece of software named below is configured in a default manner to function when the disk starts, but interested parties may take additional steps to secure the disk further through additional configuration (beyond the scope of this document). Apache2Apache web server, version 2, is featured on this disk for use with the various GUIs and administrative tools. Apache has been configured with several major modules (including PHP, Perl, and MySQL) and matches the default upstream Knoppix version. By default all traffic will be passed over https (e.g. SSL), and traffic meant for http will be redirected. Users may control apache via the /etc/init.d/apache2 interface. K Desktop EnvironmentThe K Desktop Environment (KDE) is included as the default XWindows environment. There are many applications included by default including web browsers, and graphical interfaces to certain configuration tools. Users more comfortable with this environment when compared to a console should run startx to enable XWindows. MySQLMySQL, an open source relational database, is the default method for storing perfSONAR data collected via PingER and perfSONAR-BUOY, and Cacti. By default MySQL is configured to listen only on the local interface and does not feature a default root password. Users concerned with security are encouraged to set one immediately. All user accounts are protected by passwords. NTPNTP is a protocol designed to synchronize the clocks of computers over a network. NTP version 3 is an internet draft standard, formalized in RFC 1305. NTP version 4 is a significant revision of the NTP standard, and is the current development version, but has not been formalized in an RFC. Simple NTP (SNTP) version 4 is described in RFC 2030. The perfSONAR Network Performance Toolkit comes pre-loaded with several NTP servers, and users may adjust which servers their specific disk uses in the Step 7 - NTP step. Do note that a sane configuration of NTP consists of 4 to 5 servers geographically close to the deployment location. Oracle DB XMLOracle Berkeley DB XML is an open source, embeddable XML database with XQuery-based access to documents stored in containers and indexed based on their content. Oracle Berkeley DB XML is built on top of Oracle Berkeley DB and inherits its rich features and attributes. Like Oracle Berkeley DB, it runs in process with the application with no need for human administration. Oracle Berkeley DB XML adds a document parser, XML indexer and XQuery engine on top of Oracle Berkeley DB to enable the fastest, most efficient retrieval of data. Version 2.3.11 of this software comes pre-loaded, but does not need to be started (it functions more as a library than an executable). FAQ
openssl req $@ -new -x509 -days 365 -nodes -out /etc/apache2/apache.pem -keyout /etc/apache2/apache.pem Last Updated$Id$ |
Sign in to add a comment
