1 1 Network Traffic Analysis Using tcpdump Judy Novak Judy Novak Johns Hopkins University Applied Physics Laboratory jhnovak@ix.netcom.com Introduction to tcpdump All material Copyright  Novak, 2000, 2001. All rights reserved. 2 2 Table of Contents Topics Introduction to tcpdump Writing tcpdump Filters Examination of Datagram Fields Beginning Analysis Real World Examples Step by Step Analysis References 3 3 Course Objectives • Introduce the fundamentals of tcpdump • Explain how to write tcpdump filters • Examine fields in datagram for uses/misuses • Analyze traffic by placing it in categories • Demonstrate “real-world” analysis using tcpdump • Let you participate in the analysis process The objectives of this course are to introduce you to the fundamentals and benefits of using tcpdump as a tool to analyze your network traffic. We’ll start with introducing concepts and output of tcpdump. One of the most important aspects of using tcpdump is being able to write tcpdump filters to look for specific traffic. Filter writing is fairly basic unless you want to examine fields in an IP datagram that don’t fall on byte boundaries. So, that is why an entire section is devoted to the art of writing filters. Before we start to use tcpdump to analyze traffic, we’ll examine many of the fields found in the IP datagram. This is done to familiarize you with those fields in theory and also how they might be used in practice. We’ll study how and why fields might be changed and for what purpose. Next, we’ll start the basic analysis process by looking at tcpdump output and categorizing the kind of traffic that you can see. Then, we’ll take a look at some real-world examples and of how tcpdump was used on monitored networks to discover what was happening. Next, the analysis process will be inspected step by step often with missteps to get you comfortable with it. As a note, all tcpdump output shown in this course is activity that actually occurred. Source and destination hosts/IP’s have been altered to obfuscate the true identities. 4 4 Overview • Introduction to tcpdump • Writing tcpdump filters • Examination of Datagram Fields • Beginning Analysis • Real World Examples • Step by Step Analysis This page intentionally left blank. 5 5 Introduction to tcpdump • Introduction to tcpdump • Writing tcpdump Filters • Examination of Datagram Fields • Beginning Analysis • Real World Examples • Step by Step Analysis This page intentionally left blank. 6 6 Objectives • Examine the strengths/weaknesses of tcpdump • Organize collection/analysis process of tcpdump data via Shadow • Examine tcpdump output • Standard • Hexadecimal • Length fields and how to convert them to bytes • Application layer • Interpretation of payload/hex output This page intentionally left blank. 7 7 Introduction This page intentionally left blank. 8 8 Strengths • Provides audit trail/historical record of network activity • Provides absolute fidelity • Universally available and used A One of the most important parts of an arsenal in your security infrastructure is at least one tool or software package that captures an audit trail or a historical record of the traffic that enters or leaves your network. There will be times when you will be required to examine activity or connections that occurred in your network – not just traffic that caused an alarm to sound. For instance, what if you suspect that your packet filtering router that acts as your perimeter defense was acting strangely after some major network changes were made. You would have to examine the traffic that was allowed into your network to assist in determining the problem. That is where tcpdump is invaluable. Also, many tools - even logs from firewalls will display suspicious traffic, yet only partial data is displayed. What if you get a log of rejected traffic, but it doesn’t display or keep TCP flags? You’ll never know what kind of connection was attempted. tcpdump allows the analyst to examine all the bits and fields that are collected. If nothing is “wrong” with the connection, examination at the bit level is unnecessary. Yet, if you suspect something “foul” with the traffic, you really need access to all the data down to the bit level. And tcpdump is a tool that is universally used and very portable. If you become familiar with this software or its Windows counterpart, windump, it can be used on just about any platform to assist you in analysis of traffic. 9 9 Weaknesses • By default, doesn’t collect all the payload • Does not scale well on large networks • No idea of state • Limited operations • Do-it-yourself interpretations tcpdump will capture 68 bytes of data from the network interface. Some of this data might be used for the link layer frame header. For Ethernet, 14 bytes of the data are used to capture fields like the source and destination MAC address, along with the type of embedded data. That leaves only 54 bytes to capture the IP header and embedded protocol header as well as any data. Most of the time this size will allow you to capture the IP header and embedded protocol header. But, sometimes protocol headers or data will be truncated. And, if you are interested in the data payload, tcpdump is really not the tool to use for this. tcpdump can collect a large volume of data for larger networks. This can be alleviated by not collecting all the data on the network – perhaps omit web traffic (port 80). Or, another way to deal with this is more disk space and faster processors to analyze all the collected data. But, at some point, the volume gets unwieldy. tcpdump blindly collects packet after packet. It has no idea of state or being able to know that a given packet is anomalous because it does not follow the flow of a normal connection. And while tcpdump has some primitive arithmetic operations or ways to manipulate bits, it cannot do complex operations for analyzing data. Finally, while it is an excellent way to collect data, tcpdump does not attempt to make interpretations of what it sees. It does have some integrity checking operations for certain data to make sure that the data is not irregular, but the analyst has to have the training and savvy to interpret the data. For the sophisticated analyst, this is a bonus because she or he can make the correct call. Compare this with a tool that is prone to false positives that gives no way of verifying the alarmed event. But, for an analyst who has little training, tcpdump can be daunting since it does not interpret events. 10 10 tcpdump Versions • tcpdump: Unix version; official current version 3.4 • ftp://ftp.ee.lbl.gov/tcpdump.tar.Z • ftp://ftp.ee.lbl.gov/libpcap.tar.Z • windump: Windows version • http://netgroup-serv.polito.it/windump • http://netgroup-serv.polito.it/winpcap • Collective effort; current version 3.5: www.tcpdump.org • tcpdump-3.5.tar.gz • libpcap-0.5.tar.gz tcpdump is officially supported by the Lawrence Berkeley Labs. The current version is 3.4. There is an effort to improve tcpdump and patch known problems with tcpdump and libpcap that appears to be a collective effort of anyone interested. The software for this effort can be found at www.tcpdump.org. Their current version is 3.5 For the Unix versions of tcpdump, you need to download software known as libpcap that implements a portable framework for capturing low-level network traffic. windump is a Windows variant of tcpdump. It also requires an application program interface to collect the traffic known as winpcap. The unofficial version of tcpdump has some nice enhancements. It decrypts more of the applications at the application layer and has a very nice capability of converting hexadecimal payload to character output. [...]... processing tool Shadow turns tcpdump from a packet collecting tool into an intrusion detection system Shadow collects data from the network interface and stores it in hourly files in raw tcpdump compressed format It analyzes each hour’s collected data after-the-fact and runs a series of tcpdump filters against it looking for anomalies and one -to- many source IP to destination IP traffic Shadow will format into... formatted into html for viewing by the analyst 17 What is a Shadow Event of Interest? • The default filters will extract the following types of inbound traffic: • Traffic sent to broadcast address • Traffic from reserved private networks • Fragmentation • Initial SYN connections • Particular UDP ports • Specific ICMP traffic • Scans • Traffic to core infrastructure hosts A 18 Shadow comes with several tcpdump. .. or hosts may have to be excluded so as not to false alarm For UDP records, you have to maintain a list of UDP destination ports that are of interest to you Shadow looks for signs of a one -to- many relationship of source IP to multiple destination hosts – often indicative of a scan Finally, Shadow can be tuned to look at more granular activity to the core infrastructure hosts in your network 18 Sample... uses tcpdump output to assist in the understanding of TCP/IP One of the first things you will need to do upon looking at the hex output is to determine where the IP header is and how long it is We’ll see how to do that in upcoming slides Also, you want to examine the embedded protocol and determine where that header stops and starts Finally, you may have some kind of interest in the embedded protocol... in some way, you want to be able to inspect the entire datagram 16 Shadow Architecture hour 00 data tcpdump filters secure copy hour 01 data hour 02 data DMZ analysis host sensor html output 17 The Shadow architecture is a two-host system Typically, the sensor resides on the DMZ, but it can be placed anywhere on the network It collects the traffic from the network interface and stores the data in hourly... on this slide, a host running tcpdump and gathering records from the network interface We see the records that tcpdump has collected below tcpdump has a default standard output based on the protocol (TCP, UDP, ICMP) of the record that is displayed While each of the various protocols has a similar format to the other, they are also distinct in what is displayed By default, tcpdump will collect and print,... detected by the tcpdump filters and processed by some perl programs The analyst can examine the output with a browser and further investigate activity using some additional perl scripts to look through an hour’s or day’s worth of data Using Shadow relieves the analyst from having to worry about the collection of tcpdump data; it automates this process Further, it gives the analyst an automated way of... Still, the analyst has to interpret the output As with any other intrusion detection system, it requires a savvy analyst to accurately interpret the output However, since it is predicated upon tcpdump, the analyst has the ability to examine all the collected data down to the bit level 14 What is Shadow? • Intrusion detection system based on tcpdump • Unix-based • Performs traffic analysis • Primary focus... header header length length 8-bit TOS 16-bit IP identification number 8-bit time to live (TTL) 31 16-bit total length 16-bit IP datagram total length 3-bit flags 8-bit protocol 13-bit fragment offset 13-bit fragment offset length 16-bit header checksum 32-bit source IP address 32-bit destination IP address 4-bit header length – multiply by 4 to convert to bytes 16-bit total IP datagram length – already... events of interest via web browser • Requires a savvy analyst to interpret output • Freeware available from www.nswc.navy.mil 15 Shadow is a Unix based intrusion detection system It has a sensor and analysis component The sensor component collects network traffic and the analysis component fetches that traffic and analyzes it Both the sensor and analysis host process data in an hourly timeframe The entire . 1 1 Network Traffic Analysis Using tcpdump Judy Novak Judy Novak Johns Hopkins University Applied Physics Laboratory jhnovak@ix.netcom.com Introduction to. objectives of this course are to introduce you to the fundamentals and benefits of using tcpdump as a tool to analyze your network traffic. We’ll start with