Implementation
The Probes , the Central Collector and one of the Traffic Generators are written in C-code.
Off-line data processing and reporting involves a mix of Python 2.x, MySQL SQL Stored Procedures, JavaScript (for BIRT tuning, and with AngularJS & JQuery for the web menu bar)
Packet capture libraries
The Probing software that is based on standard low-level software libraries (used in most common-off-the-shelf or freeware ‘capturing’ tools), is designed in the most ‘open’ way, allowing as much as possible the use of configuration files, while trying to achieve the best performances by using C-code programming and standard libraries. However, the specifics of the Linux ring buffer capture – with zero copy, is fully exploited. The same applies to the Linux Ethernet card tuning capability and the to separate the CPU affinity of the two probe processes in pipeline, in order to have them running in real parallelism
Signature hash
Packet Signature calculation has also been kept highly configurable, which allows the user to fine-tune the monitoring of his specific data flow in the most optimal way, in order to avoid false positive packet loss detection.
Observation reception acknowledgment
The mechanism of the Central Processing system to acknowledge the reception of observations sent by the probe, is essential to achieve a good level of reliability of the produced latency reports. One single missing datagram with observations would indeed induce tens of false positive ‘packet lost’ or ‘packets injected’, and would falsify the overall qualification of the data-link.
Efficient search for observation reconciliation
The Central Processing system uses a very fast binary search tree, to store, search, and consolidate the observations originating from multiple probes, but regarding the same packet.
Standard database
A standard database system is used (MySQL) in order to allow advanced users to get the most of the monitoring system, by digging deeper into the raw data that has been stored.
Standard report writer
Reports are built on the fly, using BIRT, an open source technology platform used to create data visualizations and reports that can be embedded into rich client and web applications.
Synthetic traffic generation
Each subsystem (Traffic Generator, Traffic Receiver, Probe, Central Processor and Reporter) is running on a separate Linux-based PC system, or they may share machines, depending on geographical location, traffic bandwidth and other criteria.
The traffic overhead created by the ‘observations’ has been kept as small as possible, to limit the bandwidth-cost of the monitoring system. Per monitored packet, one observation – packed and sent in bulks (default max size per bulk is 300 bytes) of around 60-70 observations – requires an average of 4.62 bytes. For example, for an application traffic with an average packet size of 1 kbit, the monitoring overhead is thus less than 0.5%.