5.5 - General Network Troubleshooting

Low optical link budget

• Amount of loss suffered by all components along a fiber transmission path • Attenuation - Can be caused by a dirty connector • Always check with a light meter and compare results to the documentation for the network device

Troubleshooting DNS issues

• Check IP configuration - Is the DNS IP address correct? • Use nslookup or dig to test - Does resolution work? • Try a different DNS server - Google is 8.8.8.8; Quad9 is 9.9.9.9

Incorrect VLAN configurations

• Check VLAS assignments on the switch - What VLANs have been assigned to which individual interfaces • Check trunk settings - Make sure that you're including all the VLANs that need to are able to traverse that trunk

Troubleshooting duplicate IP addresses

• Check you IP address configurations • Ping an IP address before static addressing - Does it respond • Determine the IP address - Ping the address, check your ARP table; Find the MAC address in your switch MAC table • Capture the DHCP process - What DHCP servers are responding

Troubleshooting IP configurations

• Check your documentation - Confirm IP address, subnet mask, gateway, DNS settings are correct • Monitor the traffic with protocol analyzer or other monitoring tool • Check devices around you - Confirm the configurations of other devices • Traceroute and ping - Which devices can you and can't you communicate to.

Duplicate IP addresses

• Common with static address assignments • Could also happen with DHCP if using in tandem with static addressing or multiple servers • Rogue DHCP servers could also cause it • Will cause intermittent connectivity as the two addresses "fight" with each other • Usually blocked by the OS - Will check on startup and will disable itself to avoid creation of a duplicate IP address

IP configuration issues

• Communicate to local IP addresses, but no outside subnets • Or may not have IP communication at all - Local or remote • May also be intermittent connectivity or they're able to communicate to some subnets, but not others

Licensed feature issues

• Features are often individually licensed - Requires some payment • Some ports and features may become unavailable - A license key unlocks functionality • This can cause problems during an upgrade or configuration update • If rolling out some configuration changes, examine what is licensed on end user devices and make sure testing matches that licensing

Rogue DHCP server

• IP address assigned by a non-authorized server • Clients can be assigned an invalid or duplicate IP address, causing intermittent connectivity, or no connectivity at all • Enable DHCP snooping on your switch • Identify authorized DHCP servers in Active Directory • Identify and disable the rogue and renew the leases

Incorrect firewall settings

• If there are configurations that limit certain applications, port number, protocols - Applications won't work properly • Check host-based firewall settings as well - What is admin allowing or not allowing to be sent/received from those devices • Confirm the network based firewall config - Check policy list and logs • Take a packet capture - The traffic may never make it to the network or dropped by the OS

Half-duplex Ethernet

• If two devices communicate simultaneously, you have a collision

IGMP snooping

• Internet Group Management Group Protocol - Used by routers and switches to intelligently forward multicast traffic • Switches can watch (snoops) for these messages and then uses that info to make intelligent decisions on where to forward the multicast traffic

Duplicate MAC addresses

• May be an on-path attack • MAC addresses are designed to be unique • Could also simply be a mistake - Locally administered MAC address or a manufacturing error • Can cause intermittent connectivity • Confirm with a packet capture, should see ARP contention • Use the ARP command from another computer to confirm the MAC address matches the IP

Multicast flooding

• Multicast is used for one-to-many traffic flows such as live video feeds • There's no multicast destination address in the switch forwarding table so all multicast traffic is sent to every switch port - Both devices that involved in the communication and devices not involved • Each device receives the multicast traffic - Consumes unnecessary resources on the remote device; Uses bandwidth and switch processing time

Hardware failure

• No response - Application doesn't respond • Confirm connectivity - ping • Run a traceroute - See if you're being filtered • Check the server - Lights? Error Messages? Fire?

Collisions

• Normal on a half-duplex • Heavy utilization can cause excessive collisions • Most Ethernet connections are full-duplex so if there are collisions, there is a problem • Interface configuration issues - Duplex mismatch • Hardware issue - Could indicate a bad NIC or bad driver

Switching loops

• Possible if Spanning Tree Protocol is not configured

Missing route

• Route to the next destination network does not exist so the packet will be dropped • ICMP host unreachable message will be sent to the same route address - Source device will be informed of the error • Check your routes in both directions

Certificate issues

• Security alert and invalid certificates • Look at the certificate details - Click on the lock icon • May be expired or the wrong domain name • May not be properly signed (untrusted certificate authority) • Correct time and date is important

Exhausted DHCP scope

• Shortage of available IP addresses • Client receives an APIPA address and will only be able to communicate on the local subnet • Check DHCP server and add more IP addresses if possible • IP address management (IPAM) may help - Will monitor and report on IP addresses shortages • Lower the lease time - Especially if lots of people come and go from the office

NTP issues

• Some cryptography is very time sensitive • Kerberos communication uses time stamps to determine how old a ticket might be • If the timestamps are wrong on either server or client, devices simply can't login to the network • Configure NTP on all devices and confirm configurations

Troubleshooting broadcast storms

• Take a packet capture to identify the source • Research the process that's broadcasting • Separate the network into smaller broadcast domains

Broadcast storms

• The forwarding of a frame repeatedly - hundreds or thousands of times - on the same links, consuming significant parts of the links' capacities • Will cause network utilization to go to a near-maximum capacity and the CPU utilization of the switches will jump

Network performance issues

• The network is slow • Never one single performance value to look at, and is really a combination of many different metrics • Any one thing performing poorly can cause the entire device to have poor performance • Must monitor all of them to find the slowest one - Can be quite a challenge

Asymmetric routes

• Traffic that follows one path on egress (the way out) and a different path ingress (the way in), or vice versa • Can be difficult to troubleshoot if accidently created • Firewalls may drop sessions - An unexpected traffic flow is dropped by default • Traceroute can help identify • Configure firewalls to take into account the asymmetric route • Or, if it's a mistake, go to routers on inbound and outbound side to make sure they're forwarding to the right IP

DNS issues

• Web browsing doesn't work • Ping works, browser doesn't - Not a communication problem • May prevent the access of company shared drives, which are typically accessible through fully qualified domain names • Applications often use names rather than IP addresses so they may not communicate either

Routing loops

• When router A thinks the next best hop is router B, and router B thinks the next best hop is router A, creating a loop until TTL=0 and the packet is discarded • Very easy to misconfigure if managing a statically routed network • A traceroute will clearly show where this is happening • Check the routing tables in each L3 device and modify as needed