Unit 4 Networking

TCP Port numbers

20 FTP -- Data 21 FTP -- Control 22 SSH Remote Login Protocol 23 Telnet 25 Simple Mail Transfer Protocol (SMTP) 546 DHCP Client 547 DHCP Server 3306 TCP UDP MySQL database system

Configuring Routing

As explained earlier, routers pass traffic from one network to another.

Packets and encapsulation

Data travels on a network in the form of packets, bursts of data with a maximum length imposed by the link layer. Each packet consists of a header and a payload. The header tells where the packet came from and where it's going. It can also include checksums, protocol-specific information, or other handling instructions. The payload is the data to be transferred. As a packet travels down the protocol stack (from TCP or UDP transport to IP to Ethernet to the physical wire) in preparation for being sent, each protocol adds its own header information. Each protocol's finished packet becomes the payload part of the packet generated by the next protocol. This nesting is known as encapsulation. On the receiving machine, the encapsulation is reversed as the packet travels back up the protocol stack.

ipv6 support

ELB http://docs.aws.amazon.com/ElasticLoadBalancing/latest/DeveloperGuide/using-elb-ipv6.html

Troubleshooting web server using logs

One of the primary ways you will troubleshoot web server problems is via the logs. Each request that goes to a web server gets logged in a standard format that may look a bit odd at first, but each line packs a lot of useful information for troubleshooting. Both Apache and Nginx store their logs in custom directories under /var/log/apache2 (or apache or httpd depending on your distribution) and /var/log/nginx, respectively. Both web servers default to similar log formats and both store request logs under access.log and errors under errors.log. Get webserver statistics Both Apache and Nginx provide a server status page. In the case of Apache, it requires that you enable a built-in module named status. How modules are enabled varies depending on your distribution; for example, on an Ubuntu server, you would type a2enmod status.

SSH Options

OpenSSH options are controlled through the /etc/ssh/sshd_config file. 1. Disable Root Login (PermitRootLogin) 2. Allow Only Specific Users or Groups (AllowUsers AllowGroups) 3. Deny Specific Users or Groups (DenyUsers DenyGroups) 4. Change SSHD Port Number (Port) 5. Change Login Grace Time (LoginGraceTime) 6. Restrict the Interface (IP Address) to Login (ListenAddress) 7. Disconnect SSH when no activity (ClientAliveInterval)

Path MTU discovery

Path MTU Discovery (PMTUD) is a standardized technique in computer networking for determining the maximum transmission unit (MTU) size on the network path between two Internet Protocol (IP) hosts, usually with the goal of avoiding IP fragmentation.

Spanning Tree Protocol

The Spanning Tree Protocol (STP) is a network protocol that ensures a loop-free topology for any bridged Ethernet local area network.

TCP vs UDP

http://www.cyberciti.biz/faq/key-differences-between-tcp-and-udp-protocols/

Router

A router is a device that forwards data packets along networks. A router is connected to at least two networks, commonly two LANs or WANs or a LAN and its ISP's network. Routers are located at gateways, the places where two or more networks connect. Routers are small physical devices that join multiple networks together. Technically, a router is a Layer 3 gateway device, meaning that it connects two or more networks and that the router operates at the network layer of the OSI model.

ARP: The Address Resolution Protocol

Address Resolution Protocol (ARP) is a protocol for mapping an Internet Protocol address (IP address) to a physical machine address that is recognized in the local network. Although IP addresses are hardware-independent, hardware addresses must still be used to actually transport data across a network's link layer.13 ARP, the Address Resolution Protocol, discovers the hardware address associated with a particular IP address. If host A wants to send a packet to host B on the same Ethernet, it uses ARP to discover B's hardware address. If B is not on the same network as A, host A uses the routing system to determine the next-hop router along the route to B and then uses ARP to find that router's hardware address. Since ARP uses broadcast packets, which cannot cross networks,14 it can only be used to find the hardware addresses of machines directly connected to the sending host's local network. Every machine maintains a table in memory called the ARP cache, which contains the results of recent ARP queries. Under normal circumstances, many of the addresses a host needs are discovered soon after booting, so ARP does not account for a lot of network traffic. ARP works by broadcasting a packet of the form "Does anyone know the hardware address for 128.138.116.4?" The machine being searched for recognizes its own IP address and replies, "Yes, that's the IP address assigned to one of my network interfaces, and the corresponding Ethernet address is 8:0:20:0:fb:6a." The arp command examines and manipulates the kernel's ARP cache, adds or deletes entries, and flushes or shows the table. arp -a displays the contents of the ARP cache; output formats vary.

VPC

Amazon Virtual Private Cloud (Amazon VPC) enables you to define a virtual network in your own logically isolated area within the AWS cloud, known as a virtual private cloud (VPC). You can launch your AWS resources, such as instances, into your VPC. Your VPC closely resembles a traditional network that you might operate in your own data center, with the benefits of using AWS's scalable infrastructure. You can configure your VPC; you can select its IP address range, create subnets, and configure route tables, network gateways, and security settings. You can connect instances in your VPC to the Internet. You can connect your VPC to your own corporate data center, making the AWS cloud an extension of your data center. To protect the resources in each subnet, you can use multiple layers of security, including security groups and network access control lists. Benefits of Using a VPC By launching your instances into a VPC instead of EC2-Classic, you gain the ability to: - Assign static private IP addresses to your instances that persist across starts and stops - Assign multiple IP addresses to your instances - Define network interfaces, and attach one or more network interfaces to your instances - Change security group membership for your instances while they're running - Control the outbound traffic from your instances (egress filtering) in addition to controlling the inbound traffic to them (ingress filtering) - Add an additional layer of access control to your instances in the form of network access control lists (ACL) - Run your instances on single-tenant hardware

Bandwidth/Throughput

Bandwidth is the maximum amount of data that can travel through a 'channel'. Throughput is how much data actually does travel through the 'channel' successfully. This can be limited by a ton of different things including latency, and what protocol you are using.

Can a broadcast traverse on a router?

Broadcast information reaches every machine on the same network and is useful for exchanging service announcements and others on the local network. Gateway routers do not pass broadcast traffic. Although a router can pass a broadcast messages between subnetworks, routers are usually configured to block broadcasts from traversing their interfaces.

Common web server problems

Configuration Problems One common and relatively simple problem to identify in a web server is a configuration problem. Since web servers need to be reloaded to take on changes in their configuration, it can be tempting to make many changes without reloading the web server; however, if you do so, you are likely to find out during a server maintenance (or when you need to restart the server to load new SSL certificates) that there's some syntax error in your configuration files and your server will refuse to start. Both Apache and Nginx validate their configuration files when you start, restart, or reload the service, so that's one way to find configuration errors—unfortunately, it also means that in the case of a problem, the server is down while you fix the errors. Fortunately, both web servers provide means to test configuration syntax and highlight any syntax errors while the server is still running. In the case of Apache, the command is apache2ctl configtest. When there is a syntax error, this command will identify the file and line number of the error so it's easy to find Nginx also provides a syntax check by running nginx -t: Permissions Problems A common headache, especially for new web server administrators, is permission problems on the web server. Although both Apache and Nginx's initial processes run as root, all subprocesses that actually do the work of serving content run as a user with more restricted permissions—usually a user like www-data or apache. If you are, for instance, uploading web pages as a different user, you may initially run into permissions problems until you make sure that each file you want to serve is readable by the www-data or apache user. $ curl http://localhost The output from the web page tells us the HTTP error even without having to tell curl to display it: a 403 Forbidden error. At this point we turn to the apache or nginx error logs. This error log lets us know that Nginx attempted to open /var/www/nginx-default/index.html, but permission was denied. Sluggish or Unavailable Web Server High Load One of the first things I check when a server is sluggish or temporarily unavailable is its load. Go through your web server logs and attempt to identify which pages are being accessed during this high load period; then attempt to load them yourself (possibly on a test server if your main server is overloaded) to gauge how much CPU various dynamic pages consume. If the load seems RAM-bound and you notice you are using more and more swap storage and may even completely run out of RAM, then you may be facing the dreaded web server swap death spiral. If your load is I/O bound, and the web server has a database back-end on the same machine, you might simply be saturating your disk I/O with database requests.

Connecting to a Network with a Static IP Address

Connecting to a Network with a Static IP Address If a network lacks a DHCP server, you must provide basic network configuration options manually. You can set these options using either configuration files or interactive commands.

Understanding Network Packets

Modern networks operate on discrete chunks of data known as packets. Suppose you want to send a 100KiB file from one computer to another. Rather than send the file in one burst of data, your computer breaks it down into smaller chunks. The system might send 100 packets of 1KiB each, for instance. This way, if there's an error sending one packet, the computer can resend just that one packet rather than the entire file. (Many network protocols include error-detection procedures.) When the recipient system receives packets, it must hold on to them and reassemble them in the correct order to re-create the complete data stream. It's not uncommon for packets to be delayed or even lost in transmission, so error-recovery procedures are critical for protocols that handle large transfers. Some types of error recovery are handled transparently by the networking hardware. There are several types of packets, and they can be stored within each other. For instance, Ethernet includes its own packet type (known as a frame)

TCP 3 way handshake

The TCP three-way handshake in Transmission Control Protocol (also called the TCP-handshake; three message handshake and/or SYN-SYN-ACK) is the method used by TCP set up a TCP/IP connection over an Internet Protocol based network. TCP's three way handshaking technique is often referred to as "SYN-SYN-ACK" (or more accurately SYN, SYN-ACK, ACK) because there are three messages transmitted by TCP to negotiate and start a TCP session between two computers. Host A sends a TCP SYNchronize packet to Host B Host B receives A's SYN Host B sends a SYNchronize-ACKnowledgement Host A receives B's SYN-ACK Host A sends ACKnowledge Host B receives ACK. TCP socket connection is ESTABLISHED. - See more at: http://www.inetdaemon.com/tutorials/internet/tcp/3-way_handshake.shtml#sthash.fre109ep.dpuf

privileged ports and unprivileged ports

The former have numbers less than 1024. Unix and Linux systems restrict access to privileged ports to root. The idea is that a client can connect to a privileged port and be confident that the server running on that port was configured by the system administrator and can therefore be trusted. Unfortunately, on today's Internet, this trust would be unjustified based solely on the port number, so this distinction isn't very useful. Port numbers greater than 1024 may be accessed by ordinary users.

Subnetting

The process of dividing a network into smaller network sections is called subnetting. 8 bit in a byte CIDR: Classless Inter-Domain Routing. Think of it as a replacement for a Netmask. The CIDR Value is equivalent to the number of on bits in a 32 bit address going left to right. For example: the CIDR value of 24 means the first 24 bits are turned on and the last 8 bits are turned off: 11111111.11111111.11111111.00000000. (See RFC's: 1519, 1817, 4632). Network Address (or Network ID): This is the address that identifies the subnet of a host. Broadcast Address: An IP Address that allows information to be sent to all machines on a given subnet rather than a specific machine. (See RFCs: 826, 919, 922, 947, 1027, 1770, 3021). 192.168.106.1/29 Number of Usable Addresses: 6 Network Address: 192.168.106.0 Broadcast Address: 192.168.106.7 First Host Address: 192.168.106.1 Last Host Address: 192.168.106.6

Network mask

(also known as the subnet mask or netmask) is a number that identifies the portion of the IP address that's a network address and the part that's a computer address. A subnet mask is used to find the network address of an IP addresses. It is used to divide the IP address into network and host addresses.

Route command examples

1. Display Existing Routes $ route 2. Adding a Default Gateway $ route add default gw 192.168.1.10 3. List Kernel's Routing Cache Information Kernel maintains the routing cache information to route the packets faster. We can list the kernel's routing cache information by using the -C flag. $ route -Cn 4. Reject Routing to a Particular Host or Network Sometimes we may want to reject routing the packets to a particular host/network. To do that, add the following entry. $ route add -host 192.168.1.51 reject

HTTP Status Codes

1xx Informational Codes Status codes that start with 1 denote some sort of informational response. 2xx Successful Codes A status code that starts with 2 denotes a successful request. Ideally, if you are troubleshooting a web server, this is the kind of response you want to see. 3xx Redirection Codes When the status code starts with 3, it denotes some sort of redirection message from the server to the client. Administrators often use this sort of response when they have moved content from one URL to another, to move you from one domain to another, or even to redirect you to use HTTPS instead of HTTP. • 300 Multiple Choices The reply from the server will contain multiple resources the client can choose to redirect to. • 301 Moved Permanently This code is used when a resource will never again be available at the previous URI and has been moved to the new resource. Administrators use this code when they want to indicate that a client should point all future requests to the new URI. 4xx Client Error Codes When you are troubleshooting server problems, you are likely to run into some error codes that start with 4. This status code family deals with errors the server believes are on the client side. The most common of these errors is 404, which is returned when a client requests a page that doesn't exist. Here are some common error codes in this class: • 400 Bad Request This code is used in response to bad syntax on the client side. • 401 Unauthorized This request requires authentication from the user, so the client should repeat the request with proper authentication. • 403 Forbidden Unlike a 401, this request is not allowed from the user and the client should not attempt to repeat the request with authentication. This error code is often indicative of permissions errors. • 404 Not Found The server couldn't find the page the client requested. This error code often comes up when the user had a typo in their request, when the request is for a page that has moved without a redirection put in place, or when the file used to exist but has been deleted. • 408 Request Timeout The client took too long to produce a request. You may see this when you are experimenting with web server troubleshooting over telnet. 5xx Server Error Codes Like 4xx status codes, 5xx status codes deal with errors, only in this case the error is likely on the server side. If you are the web server administrator and you see these kinds of error codes, you will want to dig into your web server error logs for more information on the causes of the errors. Here are some example codes from this class: • 500 Internal Server Error The server received some internal error when it was processing the request. You may see this when you have a bug in a CGI or PHP script that causes the file to error out when it is run. • 501 Not Implemented The server doesn't support the feature the client is requesting. • 502 Bad Gateway The error shows up when the server is configured as a gateway or proxy device and it received an invalid response when contacting its upstream server. • 503 Service Unavailable The server is temporarily unavailable to serve the client's request, often due to the server being overloaded or to some sort of maintenance being performed on the server. • 504 Gateway Timeout The server did not receive a response in time from some upstream server it needed to fulfill the request. The upstream server could be HTTP, but it could also result from a timeout in an upstream FTP, LDAP, or even DNS server.

Firewall

A firewall is a system designed to prevent unauthorized access to or from a private network. Firewalls can be implemented in both hardware and software, or a combination of both. Firewalls are frequently used to prevent unauthorized Internet users from accessing private networks connected to the Internet, especially intranets. All messages entering or leaving the intranet pass through the firewall, which examines each message and blocks those that do not meet the specified security criteria.

Jumbo Frame

A jumbo frame is an Ethernet frame with a payload greater than the standard maximum transmission unit (MTU) of 1,500 bytes. Conventionally, jumbo frames can carry up to 9,000 bytes of payload, but variations exist and some care must be taken when using the term. Most modern networking equipment is capable of handling larger frames but must be explicitly configured to do so. Frames which take advantage of this ability are known as 'jumbo frames', and 9000 bytes is a popular choice for the MTU. Jumbo frames are large IP frames used in high-performance networks to increase performance over long distances. Jumbo frames is the term used most often to mean 9000 bytes for Gigabit Ethernet but can refer to anything over the IP MTU (Maximum Transmission Unit) which is 1500 bytes on an Ethernet.

EC2 Instance IP addressing

A private IP address is an IP address that's not reachable over the Internet. You can use private IP addresses for communication between instances in the same network (EC2-Classic or a VPC). For more information about the standards and specifications of private IP addresses, go to RFC 1918. When you launch an instance, we allocate a private IP address for the instance using DHCP. Each instance is also given an internal DNS hostname that resolves to the private IP address of the instance; for example, ip-10-251-50-12.ec2.internal. You can use the internal DNS hostname for communication between instances in the same network, but we can't resolve the DNS hostname outside the network that the instance is in. Elastic IP Addresses in EC2-Classic By default, we assign each instance in EC2-Classic two IP addresses at launch: a private IP address and a public IP address that is mapped to the private IP address through network address translation (NAT). Elastic IP Addresses in a VPC We assign each instance in a default VPC two IP addresses at launch: a private IP address and a public IP address that is mapped to the private IP address through network address translation (NAT). We assign each instance in a nondefault VPC only a private IP address, unless you specifically request a public IP address during launch, or you modify the subnet's public IP address attribute.

Protocol stack

A set of network protocol layers that work together. A protocol stack is a set of software that converts and encapsulates data between layers of abstraction. For instance, the stack can take the commands of email transfer protocols, and the email messages that are transferred, and package them into packets. Another layer of the stack can take these packets and repackage them into Ethernet frames. There are several layers to any protocol stack, and they interact in highly specified ways.

Stateful vs stateless

A stateful connection is one in which some information about a connection between two systems is retained for future use. In some cases, the connection is kept open even though the two systems might not be transmitting information (i.e., the connection itself retains state). In contrast, a stateless connection is one in which no information is retained by either sender or receiver. The sender transmits a packet to the receiver and does not expect an acknowledgment of receipt. The recipient receives the packet without any prior connection setup.

IP address

A unique string of numbers separated by periods that identifies each computer using the Internet Protocol to communicate over a network. IPv4 uses 32 binary bits to create a single unique address on the network. An IPv4 address is expressed by four numbers separated by dots. Each number is the decimal (base-10) representation for an eight-digit binary (base-2) number, also called an octet. For example: 216.27.61.137 IPv6 uses 128 binary bits to create a single unique address on the network. An IPv6 address is expressed by eight groups of hexadecimal (base-16) numbers separated by colons.

Ephemeral port

An ephemeral port is a short-lived endpoint that is created by the operating system when a program requests any available user port. The operating system selects the port number from a predefined range, typically between 1024 and 65535, and releases the port after the related TCP connection terminates. standard port a port is a software construct serving as a communications endpoint in a computer's host operating system. A port is always associated with an IP address of a host and the protocol type of the communication. It completes the destination or origination address of a communications session. A port is identified for each address and protocol by a 16-bit number, commonly known as the port number.

switches vs hubs

As a general rule, switches are superior to hubs. Hubs mirror all traffic to all computers, whereas switches are smart enough to send packets only to the intended destination. Switches also allow full-duplex transmission, in which both parties can send data at the same time (like two people talking on a telephone). Hubs permit only half-duplex transmission, in which the two computers must take turns (like two people using walkie-talkies). The result is that switches let two pairs of computers engage in full-speed data transfers with each other; with a hub, these two transfers would interfere with each other. A hub is a device that connects PCs together. In general, what is called a hub in todays market is a "dumb" device. In a hub, when one PC sends data onto the wire, the hub simply forwards the packets to all the other devices connected to it. Each device is responsible for determining which packets are destined for it and ignoring the others. Current "hubs" typically share bandwidth between all the ports. In the days of coaxial networking, hubs were often called "bridges". Because they forward every packet that they receive, they do nothing to streamline the traffic on your local network. A switch is a little smarter than a hub, in that it records the IP and MAC addresses in a table of all the devices connected to it. Thus, when a packet is put onto the wire by one device, the switch reads the destination address information to determine if the destination device is connected to it. If it is, the switch forwards the packet ONLY to the destination device, sparing the other devices connected to it from having to read and deal with the traffic (making your network more efficient). If the switch does not recognize the destination device, then the switch sends the packet to everything connected to it, thereby requiring the devices to decide for themselves whether or not the packet is for them. In general, switches provide each device connected to them with dedicated bandwidth.

DNS record types

A—An IPv4 address. AAAA—An IPv6 address. CNAME—A canonical name (mapping one hostname onto another hostname). DNSKEY—An encryption key used by DNSSEC (a cryptographically secure enhancement to the domain name system that is in the process of being phased in) when verifying the authenticity of a DNS reply. MX (mail exchanger)—The mail server (or servers) that should accept mail on behalf of the specified domain. NS—The name server delegation for a particular record (indicating that a request for that record should be answered by another server). PTR—A pointer to a canonical name. Similar to a CNAME record except that resolving typically stops at this point, and the client must then resolve the resulting CNAME, if desired. This is primarily used for reverse DNS lookups (because the goal is to get a name from an IP address, not to get the IP address back again). It is also used by DNS Service Discovery to store the human-readable name for a service. SOA (start of authority)—Used primarily to indicate how long clients should cache the results and which other servers are authoritative for the domain. SRV—Contains the hostname and port for a provided service. This record type is used by DNS Service Discovery. TXT—Contains a series of informational attributes used by DNS Service Discovery.

Hostname

Computers work with numbers, so it's not surprising that TCP/IP uses numbers as computer addresses. People, though, work better with names. For this reason, TCP/IP includes a way to link names for computers (known as hostnames) to IP addresses.

DHCP/DORA

DHCP: Dynamic Host Configuration protocol DORA: Discover Offer Request Acknowledge Dynamic Host Configuration Protocol (DHCP) is a network protocol that enables a server to automatically assign an IP address to a computer from a defined range of numbers (i.e., a scope) configured for a given network. DHCP assigns an IP address when a system is started, for example: 1. A user turns on a computer with a DHCP client. The client computer sends a broadcast request (called a DISCOVER or DHCPDISCOVER), looking for a DHCP server to answer. The router directs the DISCOVER packet to the correct DHCP server. 2. The server receives the DISCOVER packet. Based on availability and usage policies set on the server, the server determines an appropriate address (if any) to give to the client. The server then temporarily reserves that address for the client and sends back to the client an OFFER (or DHCPOFFER) packet, with that address information. The server also configures the client's DNS servers, WINS servers, NTP servers, and sometimes other services as well. 3. The client sends a REQUEST (or DHCPREQUEST) packet, letting the server know that it intends to use the address. 4. The server sends an ACK (or DHCPACK) packet, confirming that the client has a been given a lease on the address for a server-specified period of time. During DORA process, if client takes too long, and request on the client times out, then server will send a Negative-acknowledgement. Clients must report back to the DHCP server periodically to renew their leases. If a lease is not renewed, it eventually expires. The DHCP server is then free to assign the address (or whatever was being leased) to a different client. The lease period is configurable, but it's usually quite long (hours or days).

DNS

DNS - domain name system is one of the most important parts of the internet. - DNS is used to translate an actual name into numbers. How does a Domain name server work? In a web browser, you say you enter www.example.com. When you type in www.example.com, you are actually looking for www.example.com(dot). There is a dot at the end of the domain name. You are actually going to the page www.example.com. That end dot represents the root of the internet name space. When you first search www.example.com., your browser and OS will first determine if they know the IP address already, it could be configured in your computer, or it could be in memory(cache). Browser asks the OS, and they both dont know where www.example.com. is. Next, OS is configured to ask the resolving name server for IP addresses it does not know. The resolving name server is the workhorse of the DNS lookup, it is either configured automatically or manually within your os. your OS asks or queries the resolving name server for www.example.com., the resolving name server may or may not have this in memory. If it does not, the resolving name server will go to the root name server. If root name server does not know, it knows where the TLD name servers are located. Resolving name servers will put this information from the root name servers in its cache, and then goes directly to the .com TLD name servers. Resolving name server queries www.example.com, the TLD respond they do not know. However, they do know where to find the example.com name servers. The next set of name servers are called authoritative name servers. Resolving name servers take the response from the TLD name servers and stores it in cache. It then queries the example.com name server. The authoritative name server successfully responds and provides the IP address for the domain. Resolving name server takes this information from the authoritative name server and puts it in the cache, and gives the reply to the OS. OS then gives it to the browser, then the browser makes the connection to the IP address requesting www.example.com. The process seems lengthy, but its a very quick and easy process. One lookup contains resolving name server, root name server, TLD name server, and Authoritative name server.

Unable to resolve hostname

DNS Client Troubleshooting The first place to troubleshoot DNS problems is on your local host. You will find that even if the problem is on the DNS server side, you can trace down the cause of many DNS server issues from any client with nslookup and dig installed. Both the nslookup and dig tools can be used to troubleshoot DNS issues, but for basic testing, start with nslookup. For this troubleshooting step, we'll borrow a scenario from the Chapter 5 where the client has an IP address of 10.1.1.7 and we have a server named web1 that has an IP address of 10.1.2.5 that we want to resolve. Here is an example of a successful nslookup request that resolves web1: $ nslookup web1 Server: 10.1.1.3 Address: 10.1.1.3#53 Name: web1.example.net Address: 10.1.2.5 In this example DNS is working. The web1 host expands into web1.example.net and resolves to the address 10.1.2.5. One of the first things to confirm, of course, is that this IP matches the IP that web1 is supposed to have! If web1 has the wrong IP address, then you can move down to the DNS server troubleshooting section of this chapter to find out why. =========== Possible issues: - No Name Server Configured or Inaccessible Name Server In either case you will need to inspect /etc/resolv.conf and see if any name servers are configured there. If you don't see any IP addresses configured there, you will need to add a name server to the file. You now need to start troubleshooting your connection with your name server, starting off with the ping command. If you can't ping the name server and its IP address is in the same subnet (in this case 10.1.1.3 is within my subnet), the name server itself could be completely down. A good way to confirm this would be to run an nslookup directly against your configured name server by adding its IP address to the command line: $ nslookup web1 10.1.1.3 Server: 10.1.1.3 Address: 10.1.1.3#53 Name: web1.example.net Address: 10.1.2.5 If you can't ping the name server and its IP address is in a different subnet, then either the DNS server is down or you have some kind of networking problem Missing Search Path or Name Server Problem It is also possible that you will get the following error for your nslookup command: $ nslookup web1 Server: 10.1.1.3 Address: 10.1.1.3#53 ** server can't find web1: NXDOMAIN Here you see that the server did respond, since it gave a response server can't find web1. This could mean two different things. One, it could mean that web1's domain name is not in your DNS search path. This is set in /etc/resolv.conf in the line that begins with search. A good way to test this is to perform the same nslookup command, only use the fully qualified domain name (in this case web1.example.net). If it does resolve, then either always use the fully qualified domain name, or if you want to be able to use just the hostname, add the domain name to the search path in /etc/resolv.conf. If even the fully qualified domain name doesn't resolve, then the problem is on the name server.

Troubleshoot slow networks

DNS Issues Although DNS is blamed more often than it should be for network problems, when DNS does have an issue, it can often result in poor network performance. For instance, if you have two DNS servers configured for a domain and the first one you try goes down, your DNS requests will wait 30 seconds before they time out and go to the secondary DNS server. Although this will definitely be noticeable when you run tools like dig or nslookup, DNS issues can cause apparent network slowdowns in some unexpected ways; this is because so many services rely on DNS to resolve hostnames to IP addresses. Such issues can even affect your network troubleshooting tools. Find the Network Slowdown with traceroute When your network connection seems slow between your server and a host on a different network, sometimes it can be difficult to track down where the real slowdown is. Especially in situations where the slowdown is in latency (the time it takes to get a response) and not overall bandwidth, it's a situation traceroute was made for. traceroute was mentioned earlier in the chapter as a way to test overall connectivity between you and a server on a remote network, but traceroute is also useful when you need to diagnose where a network slowdown might be. Since traceroute outputs the reply times for every hop between you and another machine, you can trace down servers that might be on a different continent or gateways that might be overloaded and causing network slowdowns. Find What Is Using Your Bandwidth with iftop Sometimes your network is slow not because of some problem on a remote server or router, but just because something on the system is using up all the available bandwidth. It can be tricky to identify what process is using up all the bandwidth, but there are some tools you can use to help identify the culprit.

DNS resolver non-recursive/recurisve

DNS resolvers See also: resolv.conf The client side of the DNS is called a DNS resolver. It is responsible for initiating and sequencing the queries that ultimately lead to a full resolution (translation) of the resource sought, e.g., translation of a domain name into an IP address. A DNS query may be either a non-recursive query or a recursive query: A non-recursive query is one in which the DNS server provides a record for a domain for which it is authoritative itself, or it provides a partial result without querying other servers. A recursive query is one for which the DNS server will fully answer the query (or give an error) by querying other name servers as needed. DNS servers are not required to support recursive queries.

Broadcasting data

Earlier, I mentioned broadcasts. A broadcast is a type of network transmission that's sent to all the computers on a local network, or occasionally all of the computers on a remote network. Because the broadcast address for a network is determined by the IP address and netmask, you can convert between the broadcast address and netmask, given one of these and a computer's IP address.

Linux NAT in Four Steps using iptables

First you need to tell your kernel that you want to allow IP forwarding. echo 1 > /proc/sys/net/ipv4/ip_forward Then you'll need to configure iptables to forward the packets from your internal network, on /dev/eth1, to your external network on /dev/eth0. You will need to edit /etc/sysctl.conf and change the line that says net.ipv4.ip_forward = 0 to net.ipv4.ip_forward = 1. Notice how this is similar to step number one? This essentially tells your kernel to do step one on boot. In order for your system to save the iptables rules we setup in step two you have to configure iptables correctly. You will need to edit /etc/sysconfig/iptables-config and make sure IPTABLES_MODULES_UNLOAD, IPTABLES_SAVE_ON_STOP, and IPTABLES_SAVE_ON_RESTART are all set to 'yes'. Steps can be found here -- http://www.revsys.com/writings/quicktips/nat.html Or simply using Amazon NAT AMI, and follow instructions in the aws docs.

Whats in a IP Header/Footer

Header: Information that is placed before the actual data. The header normally contains a small number of bytes of control information, which is used to communicate important facts about the data that the message contains and how it is to be interpreted and used. It serves as the communication and control link between protocol elements on different devices. Data: The actual data to be transmitted, often called the payload of the message (metaphorically borrowing a term from the space industry!) Most messages contain some data of one form or another, but some actually contain none: they are used only for control and communication purposes. For example, these may be used to set up or terminate a logical connection before data is sent. Footer: Information that is placed after the data. There is no real difference between the header and the footer, as both generally contain control fields. The term trailer is also sometimes used Datagram Format: A base header 20 bytes (5 "longwords") in length, with the option for expanded header options, followed by data. Header: Word 1 - Version - 4 bits Header Length (in longwords) - 4 bits Type of Service / Differentiated Services Code Point (DSCP) - 8 bits Datagram Length (in bytes) - 16 bits Word 2 - ID Number - 16 bits Fragmentation Flags - 3 bits Fragmentation Offset - 13 bits Word 3 - Time to Live - 8 bits Transport Protocol - 8 bits Header Checksum - 16 bits Word 4 - Source IP Address - 32 bits Word 5 - Destination IP Address - 32 bits Variable length fields - Options Padding more information -- http://www.thegeekstuff.com/2012/03/ip-protocol-header/

What is an IP address

IP address is short for Internet Protocol (IP) address. An IP address is an identifier for a computer or device on a TCP/IP network. Networks using the TCP/IP protocol route messages based on the IP address of the destination. The Format of an IP Address The format of an IP address is a 32-bit numeric address written as four numbers separated by periods. Each number can be zero to 255.

IPv4 Addresses Classes and Reserved Ranges

IPv4 Addresses Classes and Reserved Ranges IP addresses are typically made of two separate components. The first part of the address is used to identify the network that the address is a part of. The part that comes afterwards is used to specify a specific host within that network. Where the network specification ends and the host specification begins depends on how the network is configured. We will discuss this more thoroughly momentarily. IPv4 addresses were traditionally divided into five different "classes", named A through E, meant to differentiate segments of the available addressable IPv4 space. These are defined by the first four bits of each address. You can identify what class an IP address belongs to by looking at these bits. Class A 0--- : If the first bit of an IPv4 address is "0", this means that the address is part of class A. This means that any address from 0.0.0.0 to 127.255.255.255 is in class A. Class B 10-- : Class B includes any address from 128.0.0.0 to 191.255.255.255. This represents the addresses that have a "1" for their first bit, but don't have a "1" for their second bit. Class C 110- : Class C is defined as the addresses ranging from 192.0.0.0 to 223.255.255.255. This represents all of the addresses with a "1" for their first two bits, but without a "1" for their third bit.

Configuring with a Static IP Address

If a network lacks a DHCP server, you must provide basic network configuration options manually. You can set these options using interactive commands, as described shortly; but to set them in the long term, you adjust a configuration file such as /etc/sysconfig/network-scripts/ifcfg-name or /etc/network/interfaces. Listing 8.1 shows a typical ifcfg-name file, configured to use a static IP address. (Note that this file's exact location and name may vary from one distribution to another.) Listing 8.1: A sample network configuration file DEVICE="p2p1" BOOTPROTO="static" IPADDR="192.168.29.39" NETMASK="255.255.255.0" NETWORK="192.168.29.0" BROADCAST="192.168.29.255" GATEWAY="192.168.29.1" ONBOOT="yes"

DNS settings

In order for Linux to use DNS to translate between IP addresses and hostnames, you must specify at least one DNS server in the /etc/resolv.conf file. Precede the IP address of the DNS server by the keyword nameserver, as in nameserver 192.168.29.1. You can include up to three nameserver lines in this file. Adjusting this file is all you need to do to set the name server addresses; you don't have to do anything else to make the setting permanent.

NAT

Instances that you launch into a private subnet in a virtual private cloud (VPC) can't communicate with the Internet. You can optionally use a network address translation (NAT) instance in a public subnet in your VPC to enable instances in the private subnet to initiate outbound traffic to the Internet, but prevent the instances from receiving inbound traffic initiated by someone on the Internet. The main route table sends the traffic from the instances in the private subnet to the NAT instance in the public subnet. The NAT instance sends the traffic to the Internet gateway for the VPC. The traffic is attributed to the Elastic IP address of the NAT instance. The NAT instance specifies a high port number for the response; if a response comes back, the NAT instance sends it to an instance in the private subnet based on the port number for the response.

Diagnosing Network Issues with MTR

MTR is a powerful network diagnostic tool that enables administrators to diagnose and isolate networking errors and provide helpful reports of network status to upstream providers. MTR represents an evolution of the traceroute command by providing a greater data sample, as if augmenting traceroute with ping output. Networking diagnostic tools including ping, traceroute, and mtr use "ICMP" packets to test contention and traffic between two points on the Internet. Installing MTR yum update yum install mtr For example, to test the route and connection quality of traffic to the destination host example.com, run the following command from the desired source host: $mtr -rwc 100 example.com Analyzing MTR Reports Verifying Packet Loss When analyzing MTR output, you are looking for two things: loss and latency. First, let's talk about loss. If you see a percentage of loss at any particular hop, that may be an indication that there is a problem with that particular router. However, it is common practice among some service providers to rate limit the ICMP traffic that MTR uses. This can give the illusion of packet loss when there is in fact no loss. Understanding Network Latency In addition to helping you asses packet loss, MTR will also help you asses the latency of a connection between your host and the target host. By virtue of physical constraints, latency always increases with the number of hops in a route. However, the increases should be consistent and linear. Unfortunately, latency is often relative and very dependent on the quality of both host's connections and their physical distance. How To Use MTR A dynamic alternative to the traceroute program is mtr. Combining the functionality of ping and traceroute, mtr allows you to constantly poll a remote server and see how the latency and performance changes over time.

Configuring with DHCP

One of the easiest ways to configure a computer to use a TCP/IP network is to use DHCP, which enables one computer on a network to manage the settings for many other computers. It works like this: When a computer running a DHCP client boots up, it sends a broadcast in search of a DHCP server. The server replies (using nothing but the client's hardware address) with the configuration information the client needs to enable it to communicate with other computers on the network—most important, the client's IP address and netmask and the network's gateway and DNS server addresses. The DHCP server may also give the client a hostname and provide various other details about the network. The client then configures itself with these parameters. The IP address isn't assigned permanently; it's referred to as a DHCP lease, and if it's not renewed, the DHCP server may give the lease to another computer. Therefore, from time to time the client checks back with the DHCP server to renew its lease.

Connecting to a Network with DHCP

One of the easiest ways to configure a computer to use a TCP/IP network is to use the Dynamic Host Configuration Protocol (DHCP), which enables one computer on a network to manage the settings for many other computers. Three DHCP clients are in common use on Linux: pump, dhclient, and dhcpcd (not to be confused with the DHCP server, dhcpd). Ideally, the DHCP client runs at system bootup. This can be handled by its own SysV startup script or by an Upstart configuration file The system often uses a line in a configuration file to determine whether to run a DHCP client. For instance, Red Hat and Fedora set this option in a file called /etc/sysconfig/network-scripts/ifcfg-eth0 (this filename may differ if you use something other than a single Ethernet interface). The line in question looks like this: BOOTPROTO=dhcp If the BOOTPROTO variable is set to something else, changing it as shown here will configure the system to use DHCP

Web server troubleshooting steps

One of the first web server problems to troubleshoot is a web server that's completely unavailable. Is the Remote Port Open? So you can route to the machine, but you can't access the web server on port 80. The next test is to see whether the port is even open. There are a number of different ways to do this. For one, you could try telnet: $ telnet 10.1.2.5 80 Instead of telnet, I prefer to use nmap to test ports because it can often detect firewalls. If nmap isn't installed, use your package manager to install the nmap package. To test web1, you would type the following: $ nmap -p 80 10.1.2.5 Starting Nmap 4.62 ( http://nmap.org ) at 2009-02-05 18:49 PST Interesting ports on web1 (10.1.2.5): PORT STATE SERVICE 80/tcp filtered http Here it reported it as filtered. What this tells us is that there is some firewall in the way that is dropping the packets to the floor. Test for Listening Ports Once you are satisfied that the problem is not on you network, you should log in to the web server and test whether port 80 is listening. The netstat -lnp command will list all ports that are listening along with the process that has the port open. $ sudo netstat -lnp | grep :80 tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 919/apache Firewall Rules If the process is running and listening on port 80, it's possible that the server has some sort of firewall in place. Use the iptables command to list all of your firewall rules. If your firewall is disabled, your output would look like this: $ sudo /sbin/iptables -L Test Web Servers with Curl Curl is a relatively straightforward command-line tool that can speak the HTTP and HTTPS protocols, curl <domain> curl -I <domain>, look for status codes

DHCP Options

Overview of DHCP Options Sets The Dynamic Host Configuration Protocol (DHCP) provides a standard for passing configuration information to hosts on a TCP/IP network. The options field of a DHCP message contains the configuration parameters. Some of those parameters are the domain name, domain name server, and the netbios-node-type. DHCP options sets are associated with your AWS account so that you can use them across all of your virtual private clouds (VPC). The Amazon EC2 instances you launch into a nondefault VPC are private by default; they're not assigned a public IP address unless you specifically assign one during launch, or you modify the subnet's public IP address attribute. By default, all instances in a nondefault VPC receive an unresolvable host name that AWS assigns (for example, ip-10-0-0-202). You can assign your own domain name to your instances, and use up to four of your own DNS servers. To do that, you must specify a special set of DHCP options to use with the VPC Amazon DNS Server When you create a VPC, we automatically create a set of DHCP options and associate them with the VPC. This set includes two options: domain-name-servers=AmazonProvidedDNS, and domain-name=domain-name-for-your-region. AmazonProvidedDNS is an Amazon DNS server, and this option enables DNS for instances that need to communicate over the VPC's Internet gateway. The string AmazonProvidedDNS maps to a DNS server running on a reserved IP address at the base of the VPC network range "plus two". For example, the DNS Server on a 10.0.0.0/16 network is located at 10.0.0.2.

Ping

Ping as a network test utility and approaches to network diagnostics utilizing ping Ping is one of many tools. The results of a ping check should not be used exclusively as an indicator of a network problem, as other factors in a networked compute environment can affect the results produced by a ping test. Ping's needed protocols In order for ping to function there are two ICMP types that must be open at both ends and corresponding to the role of the test host (sending or receiving), if these ICMP types at a minimum are not allowed or are not correctly specified for the sending and receiving sides, ping will fail with 100% packet loss. Ping, by default, uses the ICMP protocols echo send (ICMP type 0) and echo reply (ICMP type 8). In order for ping to function the at the minimum these ICMP types must be open at both ends as follows; the sending client, that is performing the ping function must be able to send echo request, and receive echo replies, while the remote end must be able to receive echo requests and send echo replies. If either end is not able to send or receive their required ICMP types the ping command will fail with 100% packet loss. ICMP is your friend when testing There are additional routing and network control responses that are returned with other ICMP types, therefore for best results during network testing it is useful to simply temporarily open all ICMP types on both ends to allow for other control packets to be "seen" during testing and packet captures. Therefore any examining of packet latency and/or loss realized through the use of ping and similar tools must be considered for inherent de-prioritization due to the fact that network ping and probe type activities are not considered "mission critical" traffic, thus such results should not be used directly as identification of a networking fault, but only as one tool in conjunction with the use of other tools and methods to assist in determining overall network transit path health. What is ping for The most basic use of ping is to determine if basic network connectivity to a remote host exists. === Using ping to determine optimal MTU threshold and/or packet fragmentation issues. A mismatch in MTU sizing can causing packet fragmentation or blocked packets at points that don't allow or handle fragmentation, which can affect overall network throughput and/or connectivity. Ping can be used to determine if there is an MTU (Maximum Transmission Unit, aka, packet data payload size) problem occurring in the routing. To use ping in this manner we must use the option to set the ping packet payload size, and set the "Do Not Fragment" flag, then cycle through a series of packet sizes, noting which sizes are being blocked vs. passed, as well as which sizes demonstrate the least latency.

PAT NAT

Port Address Translation (PAT), is an extension to network address translation (NAT) that permits multiple devices on a local area network (LAN) to be mapped to a single public IP address. The goal of PAT is to conserve IP addresses. Network Address Translation (NAT) is the process where a network device, usually a firewall, assigns a public address to a computer (or group of computers) inside a private network. The main use of NAT is to limit the number of public IP addresses an organization or company must use, for both economy and security purposes.

Public and Private IP Addresses

Public and Private IP Addresses In order to maintain uniqueness within global namespace, the IP addresses are publicly registered with the Network Information Center (NIC) to avoid address conflicts. Devices that need to be publicly identified such as web or mail servers must have a globally unique IP address, and they are assigned a public IP address. Devices that do not require public access may be assigned a private IP address, and make it uniquely identifiable within one organization.

Server A can't talk to Server B

Server A Can't Talk to Server B Probably the most common network troubleshooting scenario involves one server being unable to communicate with another server on the network. This section will use an example in which a server named dev1 can't access the web service (port 80) on a second server named web1 In this example, you would find another server on the same network as dev1, such as dev2, and try to access web1. If dev2 also can't access web1, then you know the problem is more likely on web1, or on the network between dev1, dev2, and web1. If dev2 can access web1, then you know the problem is more likely on dev1. To start, let's assume that dev2 can access web1, so we will focus our troubleshooting on dev1. Is it plugged in? The first troubleshooting steps to perform are on the client. You first want to verify that your client's connection to the network is healthy. To do this you can use the ethtool program (installed via the ethtool package) to verify that your link is up $ sudo ethtool eth0 Is the Interface Up? Once you have established that you are physically connected to the network, the next step is to confirm that the network interface is configured correctly on your host. The best way to check this is to run the ifconfig command with your interface as an argument. So to test eth0's settings, you would run $ sudo ifconfig eth0 Is It on the Local Network? Once you see that the interface is up, the next step is to see if a default gateway has been set and whether you can access it. The route command will display your current routing table, including your default gateway $ sudo route -n The line you are interested in is the last line, which starts with default. If you don't see a default gateway configured here, and the host you want to reach is on a different subnet (say, web1, which is on 10.1.2.5), that is the likely cause of your problem. Once you have identified the gateway, use the ping command to confirm that you can communicate with the gateway: $ ping -c 5 10.1.1.1 Is DNS Working? Once you have confirmed that you can speak to the gateway, the next thing to test is whether DNS functions. Both the nslookup and dig tools can be used to troubleshoot DNS issues, but since you need to perform only basic testing at this point, just use nslookup to see if you can resolve web1 into an IP: $ nslookup web1 If you get the error, "No Name Server Configured or Inaccessible Name Server" If you see the following error, it could mean either that you have no name servers configured for your host or they are inaccessible: $ nslookup web1 ;; connection timed out; no servers could be reached In either case you will need to inspect /etc/resolv.conf and see if any name servers are configured there. If you don't see any IP addresses configured there, you will need to add a name server to the file. Otherwise, if you see something like the following, you need to start troubleshooting your connection with your name server, starting off with ping: search example.net nameserver 10.1.1.3 Look in /etc/resolv.conf Can I Route to the Remote Host? After you have ruled out DNS issues and see that web1 is resolved into its IP 10.1.2.5, you must test whether you can route to the remote host. Assuming ICMP is enabled on your network. Do a traceroute on the remote host, also do a telnet on port 80 to see if it is even listening. Test the Remote Host Locally - Test listening ports sudo netstat -lnp | grep :80 Firewall rules

MTU (Maximum transmission unit)

Short for Maximum Transmission Unit, the largest physical packet size, measured in bytes, that a network can transmit. Any messages larger than the MTU are divided into smaller packets before being sent. Ideally, you want the MTU to be the same as the smallest MTU of all the networks between your machine and a message's final destination. Otherwise, if your messages are larger than one of the intervening MTUs, they will get broken up (fragmented), which slows down transmission speeds. A larger MTU brings greater efficiency because each network packet carries more user data while protocol overheads, such as headers or underlying per-packet delays, remain fixed; the resulting higher efficiency means an improvement in bulk protocol throughput. A larger MTU also means processing of fewer packets for the same amount of data. In some systems, per-packet-processing can be a critical performance limitation. However, this gain is not without a downside. Large packets occupy a slow link for more time than a smaller packet, causing greater delays to subsequent packets, and increasing lag and minimum latency. For example, a 1500-byte packet, the largest allowed by Ethernet at the network layer (and hence over most of the Internet), ties up a 14.4k modem for about one second. Large packets are also problematic in the presence of communications errors. Corruption of a single bit in a packet requires that the entire packet be retransmitted. Overview: In simple terms, when you send information from one place on the internet (e.g. a web server) to another (e.g. your computer) the data is broken up in to packets. The sender breaks the overall data in to small chunks, and sends them over the internet. The internet only handles packets. The packets arrive at the other end, and they are put back together to make the whole of the original data. This could be a web page, email, image, or whatever. If a packet gets dropped, the sender works this out and resends it. That way you don't get gaps in what you receive. The process is managed by a protocol called TCP (Transmission Control Protocol). The packets get to the other by a protocol called IP (Internet Protocol). There are other protocols that work over the internet but they all come down to sending packets. Ethernet allows 1,500 byte packets to be carried. Internet providers use much faster links such as gigabit and 10 gigabit and these often allow bigger packets up to around 9,000 bytes. Some links on the internet are set up specially for certain traffic and have links that support packet sizes like 1,548 bytes. The links, like Ethernet, limit the size of packets that can be sent. The size of packet that can get from one place to another without any difficulties depends on the smallest link in the chain. On the internet as a whole this is generally 1,500 bytes. But this depends where the packet goes and what links are used. The smallest link allowed is 576 bytes which used to be common for modems on dialup-internet. The maximum size of packet you can send on a link is called the MTU (Maximum Transmission Unit). The maximum size you can receive is the MRU (Maximum Receive Unit). The terms MRU and MTU often get used interchangeably for obvious reasons. What happens if a packet is too big? Consider a packet being sent that is 1,500 bytes. It is passed from router to router until one that has a link which is less than 1,500 bytes. This creates a problem as it cannot send the packet on to the next router via that link. There are two options:- (A) Don't send the packet and send an error message back saying it could not be sent (B) Break the packet up in to smaller bits (called fragments) which will fit, and send these on to the next router. The choice depends on the packet. For IPv6 packets you have to take option (A) and send an error. For IPv4 packets the packet has a flag called DF (Don't Fragment). If that is set you have to take option (A) and send an error. If not, then you take option (B) and fragment the packet. If an error is sent back then the sending computer can try again sending smaller packets this time. If the packet is broken in to fragments then they still arrive at the destination and can be put back together by the receiving end. Either way the data gets through. Is there another option? In most cases links support 1,500 bytes, so there is not problem.

Static Routes and the Default Gateway

Static routes are for traffic that must not, or should not, go through the default gateway. Routing is often handled by devices on the network dedicated to routing (although any device can be configured to perform routing). Therefore, it is often not necessary to configure static routes on Red Hat Enterprise Linux servers or clients. Exceptions include traffic that must pass through an encrypted VPN tunnel or traffic that should take a specific route for reasons of cost or security. The default gateway is for any and all traffic which is not destined for the local network and for which no preferred route is specified in the routing table. The default gateway is traditionally a dedicated network router. Configuring Static Routes Using the Command Line If static routes are required, they can be added to the routing table by means of the ip route add command and removed using the ip route del command. Global default gateway configuration is stored in the /etc/sysconfig/network file. This file specifies gateway and host information for all network interfaces

Network performance tuning

TCP tuning Like most modern OSes, Linux now does a good job of auto-tuning the TCP buffers, but the default maximum Linux TCP buffer sizes are still too small. /etc/sysctl.conf

TCP/IP networking roadmap

TCP/IP is a protocol "suite," a set of network protocols designed to work smoothly together. It includes several components, each defined by a standards-track RFC or series of RFCs: • IP, the Internet Protocol, which routes data packets from one machine to another (RFC791) • ICMP, the Internet Control Message Protocol, which provides several kinds of low-level support for IP, including error messages, routing assistance, and debugging help (RFC792) • ARP, the Address Resolution Protocol, which translates IP addresses to hardware addresses (RFC826)2 • UDP, the User Datagram Protocol, which provides unverified, one-way data delivery (RFC768) • TCP, the Transmission Control Protocol, which implements reliable, full duplex, flow-controlled, error-corrected conversations (RFC793)

Network troubleshooting

Testing Basic Connectivity The most basic network test is the ping command, which sends a simple ICMP packet to the system you name (via IP address or hostname) and waits for a reply. In Linux, ping continues sending packets once every second or so until you interrupt it with a Ctrl+C keystroke. Tracing a Route A step up from ping is the traceroute command, which sends a series of three test packets to each computer between your system and a specified target system. Using traceroute, you can localize problems in network connectivity. Highly variable times and missing times can indicate a router that's overloaded or that has an unreliable link to the previous system on the list. If you see a dramatic jump in times, it typically means that the physical distance between two routers is great. This is common in intercontinental links. Such jumps don't necessarily signify a problem unless the two systems are close enough that a huge jump isn't expected. What can you do with the traceroute output? Most immediately, traceroute is helpful in determining whether a problem in network connectivity exists in a network for which you're responsible. Look at each hop, see the latency, and packet loss. TCP traceroute is better as ICMP is deprioritized in our network. Checking network status Interface Information Pass netstat the --interface or -i parameter to obtain information about your network interfaces similar to what ifconfig returns. All Connections The --all or -a option is used in conjunction with others. It causes netstat to display information about the ports that server programs open to listen for network connections, in addition to already-open connections. Examining Raw Network Traffic One advanced network troubleshooting tool is tcpdump. This utility is a packet sniffer, which is a program that can intercept network packets and log them or display them on the screen. Packet sniffers can be useful diagnostic tools because they enable you to verify that a computer is actually receiving data from other computers. They also enable you to examine the data in its raw form, which can be useful if you understand enough of the protocol's implementation details to spot problems.

Resolving hostnames

The Domain Name System (DNS) is a distributed database of computers that converts between IP addresses and hostnames. Every domain must maintain at least two DNS servers that can either provide the names for every computer within the domain or redirect a DNS query to another DNS server that can better handle the request. Therefore, looking up a hostname involves querying a series of DNS servers, each of which redirects the search until the server that's responsible for the hostname is found. Sometimes, you need to look up DNS information manually. You might do this if you know the IP address of a server through non-DNS means and suspect your DNS configuration is delivering the wrong address or to check whether a DNS server is working. Several programs can be helpful in performing such checks: nslookup This program performs DNS lookups (on individual computers by default) and returns the results. It also sports an interactive mode in which you can perform a series of queries. This program is officially deprecated, meaning that it's no longer being maintained and will eventually be dropped from its parent package (bind-utils or bind-tools on most distributions). Thus, you should get in the habit of using host or dig instead of nslookup. host This program serves as a replacement for the simpler uses of nslookup, but it lacks an interactive mode, and of course many details of its operation differ. In the simplest case, you can type host target.name, where target.name is the hostname or IP address you want to look up. You can add various options that tweak the program's basic operation; consult host's man page for details. dig This program performs more complex DNS lookups than host. Although you can use it to find the IP address for a single hostname (or a hostname for a single IP address), it's more flexible than host. whois You can look up information on a domain as a whole with this command. For instance, typing whois sybex.com reveals who owns the sybex.com domain, who to contact in case of problems, and so on. You may want to use this command with -H, which omits the lengthy legal disclaimers that many domain registries insist on delivering along with whois information. Check the man page for whois for information on additional options.

The Coming of IPv6

The IP portion of TCP/IP has been at version 4 for many years. A major upgrade to this is underway, however, and it goes by the name IPv6, for IP version 6. Its most important improvements over IPv4 include the following: - IPv4 supports a theoretical maximum of about 4 billion addresses. - IPv6 makes multicasting—the simultaneous transmission of data from one computer to multiple recipients and others Internet Protocol version 6 (IPv6) is the latest version of the Internet Protocol (IP), developed by the Internet Engineering Task Force (IETF) to deal with the long-anticipated problem of IPv4 address exhaustion. IPv6 is intended to replace IPv4. IPv6 uses a 128-bit address, allowing 2128, or approximately 3.4×1038 addresses, or more than 7.9×1028 times as many as IPv4, which uses 32-bit addresses and provides approximately 4.3 billion addresses.

Traceroute

The Linux traceroute commoand is used to show the network path(s) traversed by packets from one host to another. Due to variations in implimentation and how ICMP protocols are handled, the results of a traceroute test can be misinterpretted as showing problems at an interim hop, where no problem exists. If the cause of a failed connection lies in the network, this tool can help locate it. Also useful for troubleshooting routing issues related to slow performance of services such as CloudFront. - Linux traceroute uses UDP by default. For TCP connectivity issues, specify -T to use TCP. Beware: Windows tracert uses ICMP by default. -Just because traceroute reports high latency doesn't mean there's a problem. When the TTL of an IP packet expires in-transit, the router must do additional processing to send the "time exceeded in-transit" ICMP message back to the sender. Not only is ICMP traffic de-prioritized on the EC2 network (as it is on many networks of the Internet), but routers on the EC2 network also rate-limit packets destined for the router's CPU. This means traceroute might report high latency even though nothing is wrong. -Just because traceroute reports an asterisk (no response received within specified time limit) doesn't mean there's a problem. See above. -A path can vary (which is also why some hops list more than one node); this by itself isn't abnormal. -A traceroute performed in the opposite direction might not look the same; this by itself isn't abnormal Simple Traceroute: $ sudo traceroute <target host> Traceroute using TCP Protocle (not UDP/ICMP) $ sudo traceroute -T <target host> Traceroute using TCP protocole to specified service port $ sudo traceroute -T -p <port> Here is what each field means in traceroute output: hop_number: A sequential count of the number of degrees of separation the host is from your computer. Traffic from hosts with higher numbers have to go through more computers to get routed. host_name: This field contains the result of a reverse DNS lookup on the host's IP address, if available. If no information is returned from the reverse DNS query, the IP address itself is given. IP_address: This field contains the IP address for this network hop. packetroundtrip_times: The remainder of the line gives the round-trip times for a packet to the host and back again. By default, three packets are sent to each host and each attempt is appended to the end of the line.

MSS (Maximum Segment Size)

The maximum segment size (MSS) is the largest amount of data, specified in bytes, that a computer or communications device can handle in a single, unfragmented piece. For optimum communications, the number of bytes in the data segment and the header must add up to less than the number of bytes in the maximum transmission unit (MTU).

MTU instance types

The maximum transmission unit (MTU) for an instance depends on its instance type. The following instance types provide 9001 MTU (jumbo frames): CC2, C3, C4, R3, CG1, CR1, G2, HS1, HI1, I2, T2, and M3. The other instance types provide 1500 MTU (Ethernet v2 frames)

Configuring hostname

The most basic tool for setting your hostname locally is called, appropriately enough, hostname. Type the command alone to see what your hostname is, or type it with a new name to set the system's hostname to that name: # hostname nessus.example.com Many Linux distributions look in the /etc/hostname or /etc/HOSTNAME file for a hostname to set at boot time. Thus, if you want to set your hostname permanently, you should look for these files, and if one is present, you should edit it.

TCP Window

The throughput of a communication is limited by two windows: the congestion window and the receive window. The former tries not to exceed the capacity of the network (congestion control) and the latter tries not to exceed the capacity of the receiver to process data (flow control). The receiver may be overwhelmed by data if for example it is very busy (such as a Web server). Each TCP segment contains the current value of the receive window. If for example a sender receives an ack which acknowledges byte 4000 and specifies a receive window of 10000 (bytes), the sender will not send packets after byte 14000, even if the congestion window allows it.

Media Access Control (MAC) addresses

What Is a MAC Address? The MAC address is a unique value associated with a network adapter. MAC addresses are also known as hardware addresses or physical addresses. They uniquely identify an adapter on a LAN. MAC addresses are 12-digit hexadecimal numbers (48 bits in length). By convention, MAC addresses are usually written in one of the following two formats: MM:MM:MM:SS:SS:SS MM-MM-MM-SS-SS-SS The first half of a MAC address contains the ID number of the adapter manufacturer. These IDs are regulated by an Internet standards body. The second half of a MAC address represents the serial number assigned to the adapter by the manufacturer. In the example, ----- Why MAC Addresses? Recall that TCP/IP and other mainstream networking architectures generally adopt the OSI model. In this model, network functionality is subdivided into layers. MAC addresses function at the data link layer (layer 2 in the OSI model). They allow computers to uniquely identify themselves on a network at this relatively low level. MAC vs. IP Addressing Whereas MAC addressing works at the data link layer, IP addressing functions at the network layer (layer 3). It's a slight oversimplification, but one can think of IP addressing as supporting the software implementation and MAC addresses as supporting the hardware implementation of the network stack. The MAC address generally remains fixed and follows the network device, but the IP address changes as the network device moves from one network to another. IP networks maintain a mapping between the IP address of a device and its MAC address. This mapping is known as the ARP cache or ARP table. ARP, the Address Resolution Protocol, supports the logic for obtaining this mapping and keeping the cache up to date. DHCP also usually relies on MAC addresses to manage the unique assignment of IP addresses to devices. ---- All devices on the same network subnet have different MAC addresses. MAC addresses are very useful in diagnosing network issues, such as problems with IP addresses. MAC addresses are useful for network diagnosis because they never change, as opposed to a dynamic IP address, which can change from time to time. For a network administrator, that makes a MAC address a more reliable way to identify senders and receivers of data on the network.

VLAN

What does Virtual Local Area Network (VLAN) mean? A virtual local area network (VLAN) is a logical group of workstations, servers and network devices that appear to be on the same LAN despite their geographical distribution. A VLAN allows a network of computers and users to communicate in a simulated environment as if they exist in a single LAN and are sharing a single broadcast and multicast domain. VLANs are implemented to achieve scalability, security and ease of network management and can quickly adapt to change in network requirements and relocation of workstations and server nodes. Higher-end switches allow the functionality and implementation of VLANs. The purpose of implementing a VLAN is to improve the performance of a network or apply appropriate security features. other definition below: Short for virtual LAN, a network of computers that behave as if they are connected to the same wire even though they may actually be physically located on different segments of a LAN. VLANs are configured through software rather than hardware, which makes them extremely flexible. One of the biggest advantages of VLANs is that when a computer is physically moved to another location, it can stay on the same VLAN without any hardware reconfiguration. advantages: Decreasing the latency and traffic load on the network and the network devices, offering increased performance disadvantages: High risk of virus issues because one infected system may spread a virus through the whole logical network

What is loopback IP address?

What is loopback IP address? The loopback IP address is the address used to access itself. The IPv4 designated 127.0.0.1 as the loopback address with the 255.0.0.0 subnet mask. A loopback interface is also known as a virtual IP, which does not associate with hardware interface. On Linux systems, the loopback interface is commonly called lo or lo0. The corresponding hostname for this interface is called localhost. The loopback address is used to test network software without physically installing a Network Interface Card (NIC), and without having to physically connect the machine to a TCP/IP network. A good example of this is to access the web server running on itself by using http://127.0.0.1 or http://localhost.

TCP/IP Protocol Types

Within TCP/IP, several different protocols exist. Each of these protocols can be classified as falling on one of the four layers of the TCP/IP stack, as shown in Figure 8.2. The most important of the internet- and transport-layer protocols are the building blocks for the application-layer protocols with which you interact more directly. These important internet- and transport-layer protocols include the following: - IP The Internet Protocol (IP) is the core protocol in TCP/IP networking. Referring to Figure 8.2, IP is an internet-layer (aka a network-layer or layer 2) protocol. IP provides a "best effort" method for transferring packets between computers—that is, the packets aren't guaranteed to reach their destination. Packets may also arrive out of order or corrupted. Other components of the TCP/IP stack must deal with these issues and have their own ways of doing so. IP is also the portion of TCP/IP with which IP addresses are associated. (The Real World Scenario sidebar "The Coming of IPv6" describes a change in the IP portion of TCP/IP that's underway.) - ICMP The Internet Control Message Protocol (ICMP) is a simple protocol for communicating data. ICMP is most often used to send error messages between computers—for instance, to signal that a requested service isn't available. This is often done by modifying an IP packet and returning it to its sender, which means that ICMP is technically an internet-layer protocol, although it relies upon IP. In most cases, you won't use programs that generate ICMP packets on demand; they're created behind the scenes as you use other protocols. One exception is the ping program, which is described in more detail in "Testing Basic Connectivity." - UDP The User Datagram Protocol (UDP) is the simplest of the common transport-layer (aka layer 3) TCP/IP protocols. It doesn't provide sophisticated procedures to correct for out-of-order packets, guarantee delivery, or otherwise improve the limitations of IP. This fact can be a problem, but it also means that UDP can be faster than more-sophisticated tools that provide such improvements to IP. Common application-layer protocols that are built atop UDP include the Domain Name System (DNS), the Network File System (NFS), and many streaming-media protocols. - TCP The Transmission Control Protocol (TCP) may be the most widely used transport-layer protocol in the TCP/IP stack. Unlike UDP, TCP creates full connections with error checking and correction as well as other features. These features simplify the creation of network protocols that must exchange large amounts of data, but the features come at a cost: TCP imposes a small performance penalty. Most of the application-layer protocols with which you may already be familiar, including the Simple Mail Transfer Protocol (SMTP), the Hypertext Transfer Protocol (HTTP), and the File Transfer Protocol (FTP), are built atop TCP.

OSI model

http://www.webopedia.com/quick_ref/OSI_Layers.asp Layer 7, Application: The Application layer provides services to the software through which the user requests network services. Your computer application software is not on the Application layer. This layer isn't about applications and doesn't contain any applications. In other words, programs such as Microsoft Word or Corel are not at this layer, but browsers, FTP clients, and mail clients are. Layer 6, Presentation: This layer is concerned with data representation and code formatting. Layer 5, Session: The Session layer establishes, maintains, and manages the communication session between computers. Layer 4, Transport: The functions defined in this layer provide for the reliable transmission of data segments, as well as the disassembly and assembly of the data before and after transmission. Layer 3, Network: This is the layer on which routing takes place, and, as a result, is perhaps the most important OSI layer to study for the CCNA test. The Network layer defines the processes used to route data across the network and the structure and use of logical addressing. Layer 2, Data Link: As its name suggests, this layer is concerned with the linkages and mechanisms used to move data about the network, including the topology, such as Ethernet or Token Ring, and deals with the ways in which data is reliably transmitted. Ethernet, Token Ring, and Frame Relay are all examples of Data Link layer or Layer 2 protocols. Layer 1, Physical: The Physical layer's name says it all. This layer defines the electrical and physical specifications for the networking media that carry the data bits across a network.

Basic Linux networking tools (Good read)

https://w.amazon.com/index.php/AmazonWebServices/SalesSupport/DeveloperSupport/Internal/BasicTools#Linux_Tools

EC2 Linux Networking & Connectivity

https://w.amazon.com/index.php/AmazonWebServices/SalesSupport/DeveloperSupport/Linux_Playbook#EC2_Linux_Networking_.26_Connectivity

10Base-T or 100Base-T

is a twisted pair Ethernet standard. Most modern Ethernet hardware uses twisted-pair cabling, which consists of pairs of wires twisted around each other to minimize interference. Such varieties of Ethernet are identified by a -T suffix to the Ethernet variety name, as in 10Base-T or 100Base-T. The numbers denote the speed of the protocol in megabits per second (Mbps). In the late 1990s, 100Base-T took over from 10Base-T as the standard in office and even home networks. More recently, 1000Base-T and Ethernet variants that use optical cabling and that are capable of 1000Mbps speeds (that is, gigabit Ethernet) have become the standard, with 10-gigabit Ethernet the new emerging standard.

TCP/UDP Header size

specifies the size of the TCP header in 32-bit words. The minimum size header is 5 words and the maximum is 15 words thus giving the minimum size of 20 bytes and maximum of 60 bytes, allowing for up to 40 bytes of options in the header.