Replay UDP packets from capture file.
28 August 2011
Two programs are presented here, one that takes UDP packets from a network capture file and sends them out to a live server, the capture file produced by either tcpdump or wireshark. The second program takes a capture file and prints out all UDP packets with their payload data into a text file; (I could not persuade the otherwise excellent tshark program to print out the payload data, so I did my own one).
Copyright notice
I did this on company time, so the source code of these two programs (all sources in this directory) is copyright of NDS Group, it is licensed by BSD license. Thanks for my managers for allowing me to release this program as an open source project.
Download instructions
Get my stuff chdir build ./build-iptools.sh make install the preceeding line will install to /usr/local make install INSTALL_PREFIX=/opt/iptools will install stuff to prefix /opt/iptools
Replay UDP packets with udpsim
Basically the simulator performs the following steps
- G iven a capture file, it loops over all packets included in capture file and selects a subset of packets to be sent out. If the capture file includes IP fragments, these are reassembled into whole packets (if option -n is not set).
- The values of IP header and UDP headers are modified in order to fit the current host / target host. IP and UDP header checksums are recomputed.
- Some attempt is made to check if outgoing packet does not collide with any real application running right now (this is not guaranteed to succeed though). The packets are sent out with the same timing as in capture file; the packet is delayed so that will be sent out at the same time, relative to the start of the capture, this is done by use of high-resolution sleep - actually high-resolution timers work amazingly well.
- The packet is sent out to the target host, if the packet is larger than MTU of outgoing interface, it is fragmented (by hand ;-)
The simulator also captures network traffic while the simulation is running; by default it is capturing all packets that go via the outgoing network interface.
The program has to be run with root / privileged access; it runs on Linux and worked on Windows under Cygwin (wonders of wonders).
Here is the documentation
udpsim [-d] -x <capture_file> -c <capture-host> -i <tx-interface>
[-p <tx-substitution-port>] [-r <tx-substitution-port-range>]
[-f <outgoing-filter-expression>] [-n] [-m]
Resends outgoing UDP traffic from capture file.
Only traffic that originates from one IP address in the capture can be
sent out by this tool, so that we will not be able to
collide with real traffic on the current network segment.
The simulator performs the following steps:
- The simulator loops over all packet contained in the input capture file
- if the source ip address of a packet is equal to <capture-host> value,
then it is selected. Only selected packets
will be sent by the simulator (Please see -c <capture-host> parameter)
- If a filter expression (-f option) is specified then only those packets
from the capture that satisfy the filter are selected.
- All selected packets are subjected to optional substitution rules (-r option);
The first mandatory substitution rule adjusts the source ip of outgoing
packet to that of an existing network interface (-i <tx-interface> options)
Substituion rules specified by -r <rule_syntax> option can modify
the destion port and destination ip address of outgoing packets.
- An effort is made to send the packets with the same timing as those
included in the capture is made.
- During the simulation run we capture all packets that conform to the
filter expression; this way the response is recorded.
An error is reported if source port of selected packet collides with already
bound port.
Arguments:
The substitution rule file includes the following directives:
-x <capture_file>
the capture file in libpcap format (written by tcpdump).
This parameter is mandatory.
-c <capture_host>
Only packets contained in the capture, where source ip address
is equal to <capture_host> will be sent out This parameter is mandatory.
-i <tx-interface>
IP address of a network interface on the current system.
When sending out a replayed packet, the <capture-host> is substituted
with <tx-interface> This parameter is mandatory.
-f <outgoing-filter-expression>
Filter expresssion that is applied to all packets on this capture;
only packets that fit this filter will be sent out.
This parameter is optional.
For syntax of filter expression see see man pcap-filter(7)
-r [src-port|dst-port],<capture-port>,<real-port>
This option specifies an port substituion rule for a an outgoing packet.
This parameter applies either to the destination port (if first token is dst)
or the source port (if first token is src)
When sending a packet, the <capture-port> port number from the capture
file will be substituted with value <real-port>
This parameter is optional, multiple occurences of this option add up.
-r [src-port-range|dst-port-range],<capture-port-start>,<capture-port-end>,<real-port-start>
This option specifies an port substitution rule for outgoing packets.
This parameter applies either to the destination port (if first token is dst)
or the source port (if first token is src)
When sending a packet, the range of port numbers starting with <capture-port-start>
and ending with <capture-port-end>
from the capture file will be substituted with value
<real-port>+<port in packet from capture>-<capture-port-start>
This parameter is optional, multiple occurrences of this option add up.
-r dst-ip,<destination-ip-address-in-capture>,<real-destination-ip-address>
This option specifies an port ip address substitution rule for outgoing packets.
The destination ip address of
-o <output capture file>
This parameter is optional.
File name of capture file of incoming packets.
If this option is missing then the capture file will be stored in a file name
derived from input file name as <input_capture_file>.in.cap
-y <input capture filter>
This parameter is optional.
An expression that is added to capture filter of incoming packets.
By default the capture filter of incoming packets is udp and dst <tx-interface>.
If this parameter is present then its value is added to the default capture filter
and it is set to udp and <input capture filter>
For syntax of filter expression see see man pcap-filter(7)
-n
don't reassemble IP fragments, (default is to reassemble them).
Setting this option means that the MTU of the outgoing interface is
not smaller than the MTU of the capture interface, this is relevant
if the capture file includes fragments, or outgoing packet will be
larger then tx interface MTU.
-m
No packet capture is performed during this run.
-d
Dry run. No packets are sent during dry run (enabled with -d option);
No capture files are created during dry run. Packets data is displayed if
it is sent during a simulation run. If the source port of an outgoing packet
collides with an UDP port in use, then this command exits with error (status 1).
Please note that the list of UDP ports that are used at the moment of
invoking this command is considered.
Limitations - Linux:
user of this program must have root access.
The program tests if outgoing packets would clash with a bound port, alas
list of ports is obtained at start of program run,
this information may not be up to date during the entire simulation run.
The tool gives the possibility to manipulate the ip addresses and port numbers of
outgoing packets, relative to that of the capture.
Not all application protocols may allow for such treatment; if your application
protocol creates a checksum over the ip or udp headers,
then modification of these parameters will break the checksum
(for example IPSEC like protocol over UDP would break).
On some kernel versions (2.6.18) you will have to run 32 bit version of udpsim
on the 32 bit kernel, and 64 bit version on 64 bit kernel.
This is since tpacket_hdr structure (from linux/if_packet.h) is shared between
kernel and user space and has long in it.
When in doubt then please rebuild libpcap and udpsim from the sources.
Limitations - Windows:
Cygwin must be installed (download here http://cygwin.org/ ) and
cygwin/bin directory must be in the path.
Winpcap must be installed (download here http://www.winpcap.org/ ) -
it is also installed as part of wireshark setup.
We can't capture traffic on loopback adapter (winpcap limitation)
General limitations
Currently filters are applied before reassembly of packets, so see section on IP fragments.
Ups this is probably a bug that has to be fixed.
Example usage:
./udpsim -x TEST10_HOST-B.cap -c 10.36.4.112 -i 127.0.0.1 -f 'dst 10.36.4.161' -r dst-port,2000,12321 -r dst-ip,10.36.4.161,127.0.0.1
Uses capture file TEST10_HOST-B.cap (option: -x TEST10_HOST-B.cap)
The following packets are selected to be sent, (all restrictions do apply).
Packet must be udp packet.
the source ip address must be 10.36.4.112 (option: -c 10.36.4.112)
the destination ip address must be 10.36.4.161 (option: -f 'dst 10.36.4.161')
The selected packets are then modified.
The source ip address is set to 127.0.0.1 (option: -f 127.0.0.1)
The destination port is changed from 2000 to 12321 (option: -r dst-port,2000,12321)
The destination address is changd to 127.0.0.1 (option: -r dst-ip,10.36.4.161,127.0.0.1)
Incoming packets are captured by the following default filter: udp and dst 127.0.0.1
Incoming packets are captured in file TEST10_HOST-B.cap.in.cap
./udpsim -x TEST10_HOST-B.cap -c 10.36.4.112 -i 127.0.0.1 -f 'dst 10.36.4.161' -r dst-port,2000,12321 -r dst-ip,10.36.4.161,127.0.0.1 -o in.cap -y "port 12321"
Differences to the previous case:
Incoming packets are captured by the following default filter: udp and dst 127.0.0.1 and port 12321
Incoming packets are captured in file in.cap
How does udpsim work.
-
Packet capture file is read by libpcap
- The packets are sent via RAW sockets (that’s why I have to do defragmentation of packet all on my own ;-)
- For capturing packets again libpcap is used (it does not fork out to tcpdump)
The program builds its own version of LIBPCAP build, the current release version 1.1.1 had some issues that had to be patched locally.
Known issues
On Linux one should use the 32 bit build with 32 bit kernels, and 64 bit build with 64 bit kernels. This is due to a beautiful libpcap issue - packet capture and filtering is running in kernel mode, and the result is passed to user mode via a memory mapped file (zero copy indeed). Now the structure had a long member, which ties it to the current built of the operating system, thankfully this issue was removed in recent versions of the kernel; so checking for this issue would make a total mess.
Windows has the issue that one cannot capture packets via the local interface, only real network interfaces do work.
Display packets from capture file
This is program takes a capture file and prints out all UDP packets with their payload data into a text file; (I could not persuade the otherwise excellent tshark program to print out the payload data, so I did my own one). It is priceless if you need to write your own quick traffic analysis script in PERL.
loop_udp -i <infile> [-s] [-f <filter>] [-n]
Print specific information on all udp packets in input capture file.
It shows the following information
seq(#) - sequence number
time(sec.usec) - time of packet
ipv4_src - source ip
udp_port - udp port
ipv4_dst - destination ip
udp_dst - destination port
data_len - length of UDP payload
data - hexadecimal dump of UDP payload.
By default, IP fragments are reassembled into packets, this requires
that fragments have been captured and are present in the capture file.
For whatever reason I could not bring tshark to show packet data,
so i did my own, by using libpcap;
-i <infile> - input capture file
-s - don't show header line
-f <filter> - filter expression (for syntax see man pcap-filter(7))
-n - don't reassemble IP fragments, (default is to reassemble them)
IP Fragmentation
An IP packet that is too big will be divided into fragments . Fragmentation starts when a packet is sent that is larger than the network cards MTU - maximum transmission unit, on Ethernet adapters this value is typically 1500 bytes;
One point to remember when the network capture is taken: One might be dealing with packets that can get larger than the interface MTU, in this case we would like to capture all fragments of an IP packet - all the pieces that make up the packet, so the capture filter should only be relative to IP information (i.e. source IP address and destination IP address);
Why? The capture filter is applied separately on each packet, just when the packet is arriving at the network card; so an IP fragment arrives without the UDP header, the UDP header is only included with the first fragment. Therefore a capture filter with the UDP port will just ignore an IP fragment that comes after the first fragment, as the UDP port number cannot be checked without the UDP header. Of course a network capture can grow insanely large, when all packets that go through it are captured.
Why can’t libpcap reassemble fragments? Libpcap network capturing is done by the kernel, so when you specify a capture filter in tcpdump, then tcpdump compiles the filter into a finite state machine that implements the capture filter; this compiled state machine is passed to the kernel and the kernel applies it to each incoming packet (for packets that go through the specified capture device). Now each packet that is filtered, goes into a circular buffer structure that is placed into a shared memory segment and can be seen directly in user space code; where tcpdump picks it up and places it into the capture file. Actually this is a form of zero copy networking, which is a form of Rocket science for networking; by mapping the data through share memory, the packet does not have to be copied back and forth from kernel address space to user address space on gazillion system calls (what you do have when calling recv and send though). Now of course memory is premium in kernel space, and the kernel component responsible for capturing can’t be bothered with IP reassembly.
Well, at least the TCPDUMP user space could be bothered with IP reassembly? Probably not, for gigabit network cards you would pass enormous amounts of data back and forth, so doing IP reassembly in user space on the fly would be a stretch. Well it might be possible but.
… but actually most applications that use UDP should very much avoid IP fragmentation, because it puts a large strain on the system, when packets are broken up on one side, and reassembled on the other side. Of course a fragment might be lost, in this case it has to be resent, so forth. IP fragmentation is generally a thing that an application would like to avoid.
With TCP/IP - IPv4 - there might be a case when IP fragmentation occurs; if the packet goes through multiple hops between routers, and two hops have a MTU that is smaller than the size of a packet. In IPv4 the router would do fragmentation on its own. Now luckily this situation is generally avoided - there is the magical size of 576 bytes, where IP packets are guaranteed to pass without fragmentation. (or supposed to be so).
So fragmentation is sort of an edge case that all systems try to avoid. All systems ? Well there is no such thing, always some exceptions exist ;-)