Introduction to TCP/IP networks

[jggimi]

Chapter 0: Introduction to the Introduction

You can spend many years learning about the intricacies of computer networks. This tutorial is only the simplest of introductions. The intended audience is for people who are unfamiliar with computer networks in general, and TCP/IP networks specifically.

Reading this will only give you a very basic, conceptual introduction. That it all. But it should help you to reach a point where you are able to ask informed questions when an issue or problem arises. You will have a basic understanding of your own private network, and how it relates to and and communicates with other networks, including the Internet.

This is a tutorial on concepts, not on tools or techniques. IPv6 is not shown in any examples.

----

Chapter 1: Network Layers and the 7-layer OSI model
Chapter 2: Layer 1: Physical layer: hubs/switches/cables/connections
Chapter 3: Layer 2: Data link layer - Ethernets and Alphabets
Chapter 4: Layer 3: Network layer: TCP/IP and its protocols
Chapter 5: Layer 7: Applications
Chapter 6: Internet Protocol: the IP in TCP/IP. IP addresses, nets, subnets, and routing
Chapter 7: The Big Three - ICMP, UDP, TCP
Chapter 8: Session states
Chapter 9: Network Address Translation
Chapter 10: Putting it all together: Internets

[jggimi]

The International Organization for Standardization (ISO) has a model for describing networks, the Open System Interconnection (OSI) model. The model dates from the 1970s, and is still in use today primarily because it describes our current networking environments very well:

• Each layer is involved only with its own traffic, or in interfaces to the layer below or above.
• Anything used within one layer is independent of any other layer. With this model we can switch hardware or lower level protocols and not have to change our applications.

There are seven layers in the OSI model: physical, data link, network, transport, session, presentation, and application. Five of these map directly to TCP/IP.



The two layers that do not map directly to TCP/IP networking -- session and presentation -- fall into either the responsibility of the lower layer (transport) or the upper layer (application).

1. The physical layer is hardware: coaxial cables, unshielded twisted pair wiring, fiber optic lines, hubs, switches, exchanges, and wireless transmitters and receivers of all kinds. The physical layer is responsible for moving bits of information -- ones and zeros -- between two points, or between multiple points.
2. The data link layer organizes the bits of information into frames of data that are transmitted and received, and is responsible for communicating over the physical link. To do this, the data link uses agreed protocols -- as applicable to the the physical layer. There are many different data link protocols, such as Ethernet, ATM, DSL, and DOCSIS, just to name a few.
3. The network layer manages communication between systems, and organizing the frames of data of layer 2 into useful messages. In TCP/IP, this also includes the routing of messages between different networks, such as between your network and mine.
4. The transport layer conducts flow control and error handling. In TCP/IP, this may be the responsibility of the application, or the responsibility of the networking software within the OS -- the "TCP/IP protocol stack" -- or, this may a shared responsibility between the two.
5. The session layer tracks the state of any communications traffic -- for example: whether both sides have agreed to begin communication, if communication is in progress, or if communication has completed. In TCP/IP, this is either the responsibility of the application, or of the protocol stack. There is no shared responsibility.
6. The presentation layer is an abstraction -- such as XML -- used between different applications for data interfaces. If used in TCP/IP it is considered entirely the responsibility of the applications.
7. The application layer is where network applications communicate with each other. For example, this is the layer where your browser and a webserver communicate with each other using the HTTP protocol.

The layer model has helped shape the independence of our networking components. Our browsers don't need to know where the web servers are, or how to route HTTP requests to them. They don't need to know if copper wires, fiber optics, or satellites are part of the communication network. Nor do the web servers need to know how the browsers are connected. Each link in the chain need only know about its own connections and how to reach the next link in the chain.

[jggimi]

There are a wide variety of physical infrastructure, for two types of network:

1. Local Area Network (LAN) - equipment designed for high speed communication over relatively short distances, often within a building, or part of a building. The communication media may be shielded coaxial cable, unshielded twisted pair wiring, fiber optic cabling, or radio.
2. Wide Area Network (WAN) - equipment designed for long distance communication, from across a city to across the planet. The communication media are entirely under control of telecommunications carriers and are commonly carried by fiber and satellite connections. Local service connections may be of a wide variety of methods, from low speed voice telephone lines to high speed fiber connections, or radio.

Simply put, your ISP provides you with WAN services. Your LAN interconnects your local computers and network attached peripheral devices such as printers. The device that connects your WAN to your LAN is a gateway device. It may be provided by your ISP, or you may have acquired it separately. It may be called a gateway, or a router, or a modem, or have some other name ... but the one thing they all have in common is they interconnect between a WAN and a LAN.

LANs may use radio or different types of cabling. But they have common types of equipment:

• Bridge - a device which interconnects two different types of physical media on the same LAN.
• Router - a device which interconnects separate LANs, or LANs and WANs, permitting the movement of information between them.
• Switch - a device which interconnects individual point to point wiring to allow any-to-any communication.
• Hub - interconnects wiring to broadcast data from one connection to all. Significantly less efficient than a switch, as all incoming data is broadcast to all connected end points.
• Repeater - a device which rebroadcasts signals, lengthening the distance a LAN can reach. Most commonly deployed with WiFi infrastructures.

You'll commonly find combination devices. The typical Small Office / Home Office router is 3-in-1: combining a WiFi bridge with a twisted pair switch and a router to a WAN connection. And the WiFi bridge is also a hub, as radio traffic is shared among all wireless devices.

If the router connects directly with the WAN (such as DSL), the router is the gateway device. Instead, the router may have a small point-to-point LAN with an external gateway device.

A computer running BSD may be a router or a bridge. All it needs is two or more Network Interface Controllers (NICs).

[jggimi]

WANs use a variety of protocols to manage traffic between links, depending on the requirements of the physical layer and its topology, and as this is between your service provider and the gateway device will not be addressed in this introduction.

LANs, too, may use different protocols. With only a few exceptions, LANs use a link layer protocol called Ethernet. Wired Ethernet is IEEE standard 802.3, and wireless Ethernet is "WiFi", IEEE standard 802.11.

Key things to know about Ethernet:

• Every NIC is manufactured with a unique serial number - a six byte value. This is known as the Media Access Control (MAC) address and it is used by the devices on the LAN to communicate directly with each another. This field is known by other names as well, such as Ethernet Hardware Address (EHA) or sometimes "physical address".
• Data moves between NICs in Ethernet frames. Frames are blocks of data that use the MAC addresses of the sender and recipient. The recipient may be a single device, or, the recipient may be every device on the LAN -- the message may be a broadcast.
• IP addresses are not used in Ethernet frame addressing... only MAC addresses are used. IP addresses are part of the IP protocol; mapping between MAC addresses and IP addresses is discussed briefly in Chapter 6.
• Ethernet permits frames to be seen by NICs that are not the intended recipients. This occurs naturally with topologies that broadcast all frames, such as WiFi, wired hubs, or coaxial cable LANs.

In normal operation, NICs will ignore traffic destined for other MAC addresses than their own address. However, they can listen in. Your BSD's tcpdump(8) program will do this. Large Ethernet switch devices include monitoring ports so that support staff may listen in to any traffic. Monitoring traffic on a LAN is easy. The implication should be clear: Wired Ethernet traffic is not necessarily private. Any packet that can reach a NIC is permitted to be observed. And encrypted WiFi bridged to a Wired LAN is only encrypted over radio, not over the wired portion of the LAN.

---

A device on an Ethernet LAN may send a message to another device on the LAN, or to all devices. But the frames go no further. This layer of networking is limited to a single LAN.

[jggimi]

The "Internet Protocol Suite" consists of more than 130 different protocols. Your /etc/protocols file lists them, by number. Some are very obscure, and some are very commonly used. This introduction takes a look at four of them:

IP, the Internet Protocol (#0)
ICMP, Internet Control Message Protocol (#1)
TCP, the Transmission Control Protocol (#6)
UDP, the User Datagram Protocol (#17)

The network layer model works elegantly here, as TCP/IP's protocols operate in layers. Here is a diagram on TCP/IP from Wikipedia which shows how the a block of information -- a packet -- that uses UDP is transmitted, layer within layer. An Ethernet frame contains an IP packet, which contains a UDP packet, which contains the application data. Each layer is separate, and contains its own control information and the underlying payload for the next layer:

[jggimi]

We jump now to the last layer -- the protocols which manage communications at the application layer -- HTTP was mentioned above in Chapter 1, and there are other protocols at this layer you may be familiar with: such as FTP, SSH, and P2P protocols like BitTorrent.

The network model permits the applications to operate separately from protocols in other layers. This approach permits some of the protocols at this layer to choose between multiple lower level protocols as needed, or as the admin chooses. As an example of an automatic protocol switch, the Domain Name System protocol uses UDP for requests and responses that are relatively short, and uses TCP for longer length queries or for database maintenance actions between and among Domain Name Servers.

The separation of protocols into layers impacts application design. For example, the HTTP protocol uses TCP as an underlying transport mechanism. HTTP is stateless, yet HTTP takes advantage of TCP's state management and automated error handling.

[jggimi]

If you knew nothing else about TCP/IP networking before reading this introduction, you probably knew about IP addresses. You knew that every computer on a TCP/IP network has one. This is the layer that provides us with routing. With nothing more than a pair of IP addresses we can interconnect two LANs and route packets between them.

This layer is still part of our TCP/IP network, even if we only have an isolated LAN, such as two computers connected via a single cable. Because the IP address numbers are how we address all our communication in TCP/IP.

Internet Domain names like www.bsdforums.org are not part of TCP/IP addressing. Domain names are converted to IP addresses with requests (over your TCP/IP network) to a nearby Domain Name Server. Your BSD computer has the IP addresses of one or more Domain Name Servers in your /etc/resolv.conf file... put there by you or by a nearby Dynamic Host Configuration Protocol (DHCP) server at network startup.

The devices on the LAN make and keep maps between IP addresses and MAC addresses as they communicate, via an Address Resolution Protocol. It's fairly simple. The first time device 10.0.0.1 needs to send a message to IP address 10.0.0.2, it sends a broadcast packet to the LAN, asking, "Who is 10.0.0.2?" -- and it gets a reply, stating, "I am 10.0.0.2 - I am MAC address 12:34:56:78:9a:bc." On BSD systems you can see the results with the arp(8) command, or with the route(8) command's show option.

When you need packets to leave the LAN and travel to other networks, you use a router. As described in Chapter 2 of this introduction, a router is a computer with multiple network interfaces. One NIC on the LAN, and one or more NICs on other networks.

IP addresses use a definition -- a subnet -- to define the IP address range for a LAN. IP addresses within a LAN's subnet are on the LAN and can be directly addressed via the data link layer. Addresses outside of the LAN subnet are elsewhere, and must be routed. For my computer to route any packets outside the LAN, my computer needs to know which of the many devices on my LAN is the router. That information is stored along with the network(s) reachable by the router in a table in my computer's RAM -- called a routing table. Routes are fairly simple. To route a packet anywhere, my computer only needs to know:

• Its own IP address and the destination IP address
• The range of IP addresses on the local LAN.
• The address of the router on the LAN that will route the traffic to its destination.

Most small LANs only have a single router. If that's the case for me, then my computer only has a single entry in the routing table, a default route, which states, "Traffic to any non-LAN address goes to the router at <IP address>."

Subnets are defined by netmasks -- the number of bits defining the address range of a subnet. Your ifconfig(8) command will display NICs IP addresses and netmasks. There are a variety of notations to show how many bits make up a subnet. "255.255.255.0" and "/24" are two ways to describe the same netmask. IPv4 addresses are 32 bits, and both notations define the left-most 24 bits as masked out, and not part of the subnet, with the right most 8 bits available for addresses. An 8-bit subnet (256 numbers, 254 usable addresses) is very common because it is easy for human-readable IP address notations. But it is not required. Other length netmasks may be used.

Netmasks are also used in routing tables ... when there are multiple routers on a LAN. For instance, there might be 2 entries in the table, for two routers, one of which is for the internet, and one of which is for any address that begins with 10.x.x.x:

1. Route packets destined for the 10/8 network to the router at 192.168.1.1
2. My default route for all other packets is my router at 192.168.1.2

Assuming an em0 NIC, no DCHP server, and no configuration files at all, here are commands to configure this example on OpenBSD. Other BSDs will be the same or similar.

# ifconfig em0 192.168.1.3/24
# route add 10/8 192.168.1.1
# route add default 192.168.1.2

[jggimi]

ICMP is the protocol used for status, error, and information messages between networks. You have used ICMP if you have used ping(8), or perhaps traceroute(8). These are the common administrative tools. ICMP is also used by automated network management software in larger networks and across the Internet, as ICMP counts the number of networks in use along any route, as well as response times. It is also a helpful diagnostic tool when errors occur along a route path -- ICMP error information packets, such as "destination unreachable" that are returned by a router to an application may be very helpful, and may improve application performance by avoiding an application timeout.

UDP is a simple transport mechanism. It is stateless. It has no error recovery. Packets may arrive at the destination in the wrong order (if they traveled along different routes) and packets may be dropped along the way due to network congestion or error, and never arrive at all. Its main advantage is that it is fast. Either the application need not worry about error recovery or packet ordering, or the application must take on these responsibilities.

TCP is the most common transport protocol for applications, because it includes session management and error recovery. Packets are sent with sequence numbers so that they can be reordered if they arrive out of sequence, and there are acknowledgements of received packets so that missing packets that were dropped are resent. All automatically. TCP guarantees delivery, as long as there is a functional path between endpoints and functioning systems at both ends. TCP is slower than UDP, due to these additional services.

Both TCP and UDP share the concept of ports. The protocol headers contain 16 bits for a source port number, and 16 bits for a destination port number. Receiving applications "listen" for traffic assigned to specific ports. This allows devices to conduct many simultaneous UDP and/or TCP conversations at the same time, among different applications.

[jggimi]

Stateless protocols do not keep track of conversation status, they do not manage sessions. Protocols that track state, such as TCP, are known as Stateful protocols. Endpoints may be in any of ten states at any time, covering three general communication phases: a) establishing a session, b) data transfer, and c) ending a session.

TCP requires three packets to be sent and received -- a "3-way handshake" to establish a session. Once established, the session remains as a communication channel until an endpoint initiates a session teardown sequence, which may take as many as 4 packets to complete. Along with port numbers and sequence numbers, checksums are added to the packets to permit error recovery.

State management has overhead, but the overhead provides value: error detection and recovery, packet reordering (when the arrivals are out-of-order), confirmation of receipt, and flow controls to manage network congestion or latency effects.

Other protocols may manage state. For example, HTTP tracks state with application specific content called a "cookie".

[jggimi]

Network Address Translation (NAT) is very useful.

• It allows us to have private networks that can participate on the Internet.
• It provides a simple firewall; unsolicited packets are dropped.
• It greatly expands the number of devices that can reach the Internet.
• It permits organizations in partnership to link their private networks together without remapping their IP addresses to manage IP address conflicts or subnet overlaps.

NAT is a function which occurs within a router device. Individual network devices need no configuration or awareness to participate in NAT. That's because all a NAT router does is:

1. receive a packet for forwarding
2. depending on packet direction, change either the source or destination IP address
3. send the altered packet

----

An example:

• My private network uses the 10.0.0.0/24 subnet.
• My NAT router may connect this subnet to the Internet at address 1.2.3.4.

If I have a connection from my workstation at 10.0.0.5 with a server on the Internet, the NAT router translates my workstation address from 10.0.0.5 to 1.2.3.4 when it sends my traffic outbound, and it translates the inbound packets destined to my workstation from 1.2.3.4 to 10.0.0.5 before sending them to the workstation.

• There are lots of workstations on my subnet. How did the NAT router "know" to change the incoming packet destination address from 1.2.3.4 to 10.0.0.5?

NAT routers monitor state. This is known as stateful tracking. NAT routers may also translate the source port numbers, to simplify stateful tracking, as is shown in this diagram from Microsoft:



TCP states are easy to monitor. Stateless protocols (like UDP) are tracked using timers.

---

NOTE: NAT routers can't route unsolicited traffic. If a packet arrives at my example router at IP address 1.2.3.4, and there is no assocated state for this traffic ... the NAT router cannot forward the packet. This is why when you have a NAT router and you wish to offer a network service, you must pre-configure port forwarding.

If I tell the router to forward incoming TCP packets with destination port 80 to my web server at 10.0.0.9, when an incoming packet arrives with no pre-established state, and it is for destination port 80, the router will translate 1.2.3.4 to 10.0.0.9 and forward the packet to my web server.

NAT cannot route uninvited inbound traffic. Those packets are dropped, and either reflect an error via ICMP or not, depending on router configuration.

-----

Through a NAT router, many devices on a LAN may share a single IP address on the Internet. This has allowed the Internet to permit more devices to connect to it than the available IP addresses would allow. A number of address ranges have been defined for private networks. These are address ranges that are only used privately -- never on the Internet directly. To route packets in and out of networks that use these ranges, NAT must be used. These network ranges are used over and over in millions of private networks:

• 10.0.0.0 - 10.255.255.255
• 172.16.0.0 - 172.31.255.255
• 192.168.0.0 - 192.168.255.255

You and I might both have devices at address 192.168.1.10 on our private networks. They can communicate with each other, as long as both of us are using NAT routers as our gateways to the Internet. Our private addresses are translated to Internet addresses when our packets transit our routers.

[jggimi]

Thank you for taking the time to read this short introduction to networking. To obtain it:

1. Your browser extracted the www.daemonforums.org domain name from the URL for this page.
2. It requested the IP address for the domain name from your DNS server, using UDP.
3. The UDP packet became a payload of an IP packet.
4. The IP packet became a payload of an Ethernet frame.
5. The Ethernet frame was placed on the LAN with the destination the MAC address of your router, to forward to the DNS server at your ISP.
6. Your router then extracted the IP packet from the Ethernet frame and placed it in a frame for the next link: perhaps another Ethernet network, or a PPP network, or a DSL network. It translated your private network address in the IP packet and source port in the UDP packet as it did so.
7. When the router received the reply, the IP packet was extracted from that network frame and was placed on an Ethernet frame with the destination of your computer's MAC address, after translating the inbound IP address and port accordingly.
8. On arrival at your computer, the IP packet was extracted from the Ethernet frame, the UDP packet from the IP packet, the DNS response from the UDP packet.
9. Your browser initiated a TCP session with the website here, using the IP address just received. This caused a 3-way handshake to establish a TCP session. Your computer sent a TCP packet, the web server replied with a TCP packet, and your computer sent a second TCP packet. Each of these was encapsulated in an IP packet, and then encapsulated in an Ethernet frame as with the UDP DNS request above.
10. Your browser then submitted an HTTP request over the established TCP session. The HTTP request includes identifying information, so that the web server here knows if you are logged in or a guest.
11. The webserver responded with this page, via a series of TCP packets.
12. The TCP session was terminated, with a variable size handshake.

This procedure to submit an HTTP request and receive a web page back is pretty complex, isn't it? Yes. And I've simplified it. There may have been packet fragmentation, packets resent, packets reordered. And of course, there were multiple HTTP requests for graphic images, with repetition of these steps for each request.

After all, this is only a simple introduction. It is not intended to provide answers; instead, its to help you articulate better questions.

[jggimi]

Ladies and Gentlemen, I have completed this short introduction to TCP/IP.

If you have a question about a TCP/IP protocol or the application of these concepts to your situation, please post your question in its own thread. This is a conceptual introduction to networking, it is not a FAQ and it is not a Q&A thread.

Should you have a question about a technology, application, or protocol not discussed here, please post your question in its own thread. Examples of this might include IPSec, Ethernet collision detection, firewalls, security, traffic shaping, SSH, FTP, PXE, fragmentation, Quality of Service bits ... RFCs and standards ... Microsoft Networking protocols, WiFi SSIDs....

Should you have a clarifying question about the presentation of these concepts, you may post here. Suggestions for corrections to the text will be graciously received either in public or by PM, though they may not be accepted.

Thank you.

[daemonfowl]

I'd like to thank my teacher jggimi for the great support he offers to newcomers and wish him Happy Long Life & Happy Hacking ..
Thanks very much !!

[vermaden]

Great guide jggimi, thanks for taking time to write this all up in such clean and lean manner.

[drm]

Thank you very much! I created a pdf version out of the forum posts, if anyone is interested.

[jggimi]

Thank you, drm, nice job! Welcome!

[vermaden]

Originally Posted by drm

Thank you very much! I created a pdf version out of the forum posts, if anyone is interested.

Could You post the LaTeX 'source' for this document somewhere? Looks quite nice

[drm]

Sure, here it is Just basic latex, it could most likely be improved. Enjoy!

Edit: I added the microtype package and updated the pdf version above.

[vermaden]

Thanks a lot!