DAG Support Package Documentation
Table of Contents
- Introduction to DAG capture cards
- DAG 3.2E capture card
Connecting the DAG 3.2E
Using the DAG 3.2E
- DAG 3.5E capture card
Connecting the DAG 3.5E
Using the DAG 3.5E
Introduction to DAG capture cards
DAG is a project of the University of Waikato Computer Science Department. Essentially the DAG card's internal clock is synchronised to the Global Positioning System's (GPS) time network, as such the DAG card can timestamp incoming/outgoing packets at GPS time precision which is accurate to 100ns. Whilst there is still some degree of error in the timestamps produced by the DAG, the card provides a very good estimation of the actual 'real' time that a packet was received/sent.
For further notes on the application of the DAG card here at CUBINlab see the article A Precision Infrastructure for Active Probing available on the active probing articles page. For a full description of the functionality of the DAG card and the latest version see the DAG home page above.
The DAG software produces .d3h files as records of the data capture. These files can then be filtered for the packets we are interested in using dagfilter and finally converted to tcpdump format. It is then possible to convert them to .dt files which can then be analysed using MATLAB or other mathematical tools. More details about post-processing are given on the active probing software documentation page.
This page serves as a basic user's guide and introduction to the DAG card. At CUBINlab we have two main DAG card series, the 3.2E and the 3.5E (both Ethernet based). As these cards do differ in some respects, we will describe the connections and usage separately below.
DAG 3.2E capture card
This section explains how to connect and use the DAG 3.2E capture cards which exist in CUBINlab.
Connecting the DAG 3.2E
The DAG 3.2E does not have the ability to act as a passive tap, but rather acts as a sink, which means all packets going into the DAG 3.2E are lost. To tap off packets you will require either a custom-made passive tap or a hub in order to tap off the network stream you are interested in capturing. As part of the Intelligent Traceroute project, a simply passive tap was built based on a previous design by Attila Pásztor, details of which can be found on the passive tap description page.
A basic setup for the DAG 3.2E card monitoring from a probing machine connected to a 10/100 Base-T Ethernet switch through a hub in CUBINlab is illustrated below. Note that in this case, only one capture port of the DAG3.2E is connected, the other is left unconnected, and the GPS pulse-per-second signal is connected via a custom-made cable directly to the DAG 3.2E board.
Hub connections with DAG 3.2E monitoring
The various connections are detailed in the following table:
| Connection | Cable Type |
|---|---|
| DAG card (upper port) to hub | Crossover cable |
| Probing machine to hub | Straight cable |
| Switch to Hub's Uplink port | Straight cable |
Note that the hub being used is an active, powered device and as such delays the layer-2 Ethernet frames that are passing through it. The hub being used is a store and forward device, hence it has a significant impact on the accurate monitoring of timestamps as a substantial delay is added to Ethernet frames and hence IP packets that pass through the hub. The non-invasive passive tap mentioned above was constructed in an effort to remove this delay. Details are available on the passive tap description page.
Using the DAG 3.2E
The following directories for the DAG 3.2E software are assumed (and are installed on snapshot):
| /usr/local/dag-x.x.x | x.x.x version of the DAG driver and measurement tools |
| /usr/local/dagtoolsx.x.x | x.x.x version of the binary dagtools - trace post-processing tools |
| /usr/local/src/dagtools-x.x.x | x.x.x version of the dagtools source (copy binaries to dir above) |
| /usr/local/dag | symbolic link to the current version of the dag package |
| /usr/local/dagtools | symbolic link to the current version of the dagtools package |
We assume the dagtools binaries have been copied from the /usr/local/src/dagtools-x.x.x directory to the directory pointed to by /usr/local/dagtools. You will have to do that manually as the make files for the dagtools package do not currently have an install directive. We also had to modify the dagtools source code in certain places to get them to compile, including changing following:
#include <time.h> to #include <sys/time.h> and
#include <pcap.h> to #include <pcap/pcap.h>
Once the dagtools and dag packages are compiled and the symbolic links created, connect the DAG to the network section you want to capture and run the following commands:
- Setup
First we need to setup the path, load the driver, and download the Xilinx image to the DAG card. This is done by executing the following command:
source /root/dagsetup.sh
This has to be done at least once after each reboot of the machine. The file dagsetup.sh can be found in the /root directory on snapshot. You can also download dagsetup.sh here.
- To capture a trace
To capture a 30 second trace of all packets passing through the DAG card:
dagsnap -e -s30 tracefile.d3h
Note: the -e denotes Ethernet, and the -s30 tells it to capture packets for 30 seconds, unlike the DAG 3.5E software, sending SIGINT (Ctrl-C) will result in an empty tracefile, you must wait for the program to exit.
You can pipe dagsnap to the program dagfilter and use libpcap-like expressions. The following command will capture a 30 second trace of only those packets which are UDP and on port 7775 using dag device /dev/dag0 (the default device):
dagsnap -e -d /dev/dag0 -s30 | dagfilter -e udp port 7775 > tracefile.d3h
Note: Both commands will store the capture data to the file tracefile.d3h, which can then be post-processed.
- To display the trace in tcpdump format
dagbpf -e < tracefile.d3h | tcpdump -r -
- Perform any further post-processing analysis
Check the active probing software documentation page for some useful post-processing tools and techniques.
DAG 3.5E capture card
This section explains how to connect and use the DAG 3.5E capture cards which exist in CUBINlab.
Connecting the DAG 3.5E
The two capture ports of the DAG 3.5E differ in functionality to the DAG 3.2E. Unlike the DAG 3.2E, the DAG 3.5E acts as a passive tap. Therefore you must connect the DAG 3.5E so that it is in the middle of the network path you want to analyse. This also means that unlike the DAG 3.2E, you don't require an external passive tap or hub to "tap off" the packet stream.
Connecting the GPS signal to the DAG 3.5E
The GPS RS422 pulse-per-second input on the DAG 3.5E also differs in comparison to the DAG 3.2E. Where as before the PPS signal was wired directly to the DAG board, the DAG 3.5E has a RJ45 connector available for the synchronisation input signal (the GPS PPS signal). In CUBINlab we use the Acutime 2000 GPS receiver (of which more details are available on the active probing testbed page). As the DAG 3.5E expects the PPS signal on pins 3 and 6 of the RJ45 connector, and the Acutime 2000 outputs the signal on these pins, we first tried simply connecting a straight-through Ethernet cable between these two devices. However, as the DAG card also emits a signal on pins 1 and 2 (which can be used to synchronise two DAG cards together using a cross cable), this feedback caused interference with the PPS signal of the GPS, and caused the DAG to quickly lose synchronisation once provided with the PPS signal using the standard straight-through Ethernet cable. To solve this problem we made a custom-built cable with only pins 3 and 6 connected straight through.
An example cable connection setup for the DAG 3.5E is shown below. Connecting the DAG 3.5E is a bit simpler, as only straight cables are required, due to the fact that the DAG 3.5E itself acts as a passive tap. Both capture ports of the DAG 3.5E are connected, as the packets "flow through" the DAG 3.5E card. The GPS pulse-per-second signal is connected via a custom-made cable as explained above to the RJ45 synchronisation input.
Network connections with DAG3.5E monitoring
The various connections are detailed in the following table:
| Connection | Cable Type |
|---|---|
| Probing machine to upper DAG capture port | Straight cable |
| Lower DAG capture port to Switch (or next network hop) | Straight cable |
| Lowest DAG port to GPS receiver | Custom cable |
Using the DAG 3.5E
The following directories for the DAG 3.5E software are assumed (and are installed on prober):
| /usr/local/dag-x.x.x | x.x.x version of the DAG driver and measurement tools |
| /usr/local/dagtoolsx.x.x | x.x.x version of the binary dagtools - trace post-processing tools |
| /usr/local/src/dagtools-x.x.x | x.x.x version of the dagtools source (copy binaries to dir above) |
| /usr/local/dag | symbolic link to the current version of the dag package |
| /usr/local/dagtools | symbolic link to the current version of the dagtools package |
We assume the dagtools binaries have been copied from the /usr/local/src/dagtools-x.x.x directory to the directory pointed to by /usr/local/dagtools. You will have to do that manually as the make files for the dagtools package do not currently have an install directive. We also had to modify the dagtools source code in certain places to get them to compile, including changing following:
#include <time.h> to #include <sys/time.h> and
#include <pcap.h> to #include <pcap/pcap.h>
Once the dagtools and dag packages are compiled and the symbolic links created, connect the DAG to the network section you want to capture and run the following commands:
- Setup
First setup the path
source /root/dagenv.sh
Now we need to load the driver, and download the Xilinx image to the DAG card. This is done by executing the following command:
source /root/dag_setup.sh
This has to be done at least once after each reboot of the machine. The files dagenv.sh and dag_setup.sh can be found in the /root directory on prober. You can also download dagenv.sh here and download dag_setup.sh here.
- To capture a trace
To simplify the process of capturing packets we wrote a small script based on the one supplied in the dag package. To capture a 30 second trace of all packets passing through the DAG 3.5E card run:
/root/dag_snap.sh 30 tracefile.d3h
You can download dag_snap.sh here.
Note: the 30 tells dagsnap to capture packets for 30 seconds. If you want to finish earlier than that you can send SIGINT or Ctrl-C to the dagsnap process. It is setup to run in verbose mode by the script and you should see output detailing the current amount of captured data in MB on the screen. This is useful to test if the DAG 3.5E is functioning properly.
Alternatively, without using the script you can pipe dagsnap to the program dagfilter and use libpcap-like expressions. The following command will capture a 30 second trace of only those packets which are UDP and on port 7775 using dag device /dev/dag0 (the default device):
dagsnap -e -d /dev/dag0 -s30 | dagfilter -e udp port 7775 > tracefile.d3h
Note: Both commands will store the capture data to the file tracefile.d3h, which can then be post-processed. We prefer to take a capture of all the data and then filter later using tcpdump.
- To display the trace in tcpdump format
dagconvert -T erf:pcap -c 32 -i tracefile.d3h | tcpdump -tt -r -
Where here you can filter expressions to the tcpdump command and direct the output to a file. Refer to the man page for tcpdump for more details about filter expressions.
- Perform any further post-processing analysis
Check the active probing software documentation page for some useful post-processing tools and techniques.
