Chapter 19: Network Troubleshooting

Step 4: Establish a plan of action to resolve the problem and identify potential effects

Identity some possible changes, follow through and test your solution to see if you really solved the problem. Ask the user to try doing whatever it was they couldn't do when they called you in the first place. If it works, problem solved. If not, try the operation yourself

incorrect Interface/Interface misconfiguration

If a host is plugged into a misconfigured switch port, or if it's plugged into the wrong switch port that's configured for the wrong VLAN, the host won't function correctly. Make sure the speed, duplex, and correct Ethernet cable is used. Get any of that wrong and either you'll get interface errors on the host and switch port or, worse, things just won't work at all!

If the theory is not confirmed, reestablish a new theory or escalate

If you find that the suggested theory is not the cause of the issue, then you should move on to test any other theories you may have developed. In the event you have exhausted all theories you may have developed, it is advisable to escalate the issue to a more senior technician or, when it involves a system with which you are unfamiliar, the system owner or manager.

Open impedance mismatch (echo)

Impedance is the measure of resistance within the transmission medium. Impedance is measured in ohms (Ω). All cables must have the same impedance rating. The impedance rating for the cable must match the impedance of the transmitting device. Impedance is mostly a factor in coaxial cables used for networking. Be sure to choose cable with the correct rating (50 or 75 ohm) based on the network type. Never mix cables with different ratings. When signals move from a cable with one impedance rating to a cable with another rating, some of the signal is reflected back to the transmitter, distorting the signal. With video (cable TV), impedance mismatch is manifested as ghosting of the image. Cable distance does not affect the impedance of the cable.

Expired IP Address

In almost all cases, when DHCP is used to allocate IP configurations to devices, the configuration is supplied to the DHCP client is on a temporary basis. The lease period is configurable, and when the lease period and a grace period transpire, the lease is expired. The effect of an expired lease is the next time that client computer starts, it must enter the initilization state and obtain new TCP/IP configuration information from a DHCP server. There is nothing, however, to prevent the client from obtaining a new lease for the same IP address

Port Duplex Mismatch

In order for two devices to connect effectively, the duplex settings have to match on both sides of the connection. If one side of a connection is set to full andthe other is set to halt, they're mismatched

incorrect time

Incorrect time on a device can be the cause of several issues. First, in a Windows environment using Active Directory, a clock skew of more than 5 minutes between a client and server will prevent communication between the two. Second, when certificates are in use, proper time synchronization is critical for successful operation. Finally, when system logs are sent to a central server such as a Syslog server, proper time synchronization is critical to understand the order of events.

Bad SFP/GBIC - cable or transceiver

It is possible that you have a bad SFP transceiver or cable. Try replacing the cable and transceiver to see if they are the issue.

simultaneous wired/wireless connections

It's not unusual to find that a laptop today will have both a wired and wireless connection at the same time. Typically this doesn't create a problem, but don't think you get more bandwidth or better results because of it. It's possible that the configurations can cause a problem, although that's rare today. For instance, if each provides a DNS server with a different address, it can cause name resolution issues, or even default gateway issues. Most of the time, it just causes confusion in your laptop, which will make it work harder to determine the correct DNS or default gateway address to use. And it's possible for the laptop to give up and stop communicating completely! Because of this, you need to remind the user to turn off their wireless when they take it into their office and connect it to their dock.

Environmental factors

It's vital to understand your environmental factors when designing and deploying your wireless network. Do you have concrete walls, window film, or metal studs in the walls? All of these will cause a degradation of Db or power level and result in connectivity issues. Again-plan your wireless network carefully!

Attenuation

Loss of power in a signal as it travels from the sending device to the receiving device

bend radius limitation

Make sure you understand [THIS] of each type of fiber and that you don't exceed the specifications when installing fiber in your rack

bent pins

Many of the connectors you will encounter have small pins on the end that must go into specific holes on the interface to which they plug. If these pins get bent, either they won't go into the correct hole or they won't go into a hole period. When this occurs, the cable either will not work at all or will not work correctly. Taking care not to bend these fragile pins when working with these cable types will prevent this issue from occurring.

Step 3: Test the theory to determine the cause

Once you've gathered information and established a plausable theory, you've got to determine the next steps to resolve your problem. If you can't confirm your theory during this Step, you must formulate a new theory or escalate the problem

Hardware problems

Problems related to the motherboard, CPU, memory, hard disk, and other components on a motherboard. Solutions include: - changing hardware settings - updating device drivers - replacing dead hardware

Software problems

Programming errors, installation errors, unauthorized changes

untrusted SSL certificate

Reception of an untrusted SSL certificate error message can be for several reasons. The first reason, "The Security certificate presented by this website was not issued by a trusted certificate authority," means the CA that issued the certificate is not trusted by the local machine. This will occur if the certificate of the CA that issued the certificate is not found in the Trusted Root Certification Authorities Folder on the local machine. The second reason this might occur is that the certificate is not valid. It may be that the certificate was presented before the validity period begins, or it may have expired, meaning the validity period is over. The third reason is that the name on the certificate does not match the name listed on the certificate.

Missing routes

Routers must have routes either configured or learned to function. There are a number of issues that can prevent a router from learning the routes that it needs. To determine if a router has the route to the network in question, execute the show ip route command and view the routing table. This can save a lot of additional troubleshooting if you can narrow the problem to a missing route.

unresponsive Service

Services can fail for several reasons. Many services depend on other services for their operation. Therefore, the failure of one service sometimes causes a domino effect, taking down other services that depend on it. You can use the Services applet in the Control Panel to indentity these dependencies as well as start and stop services. To indentity the services upon which a particular service depends, use the Dependencies tab on the Services applet.

Absorption

Some materials will absorb a signal and reduce its strength. While there is not much that can be done about this, this behavior should be noted during a site survey, and measures such as additional APs or additional antenna types may be called for.

Incorrect VLAN

Switches can have multiple VLANs each, and they can be connected to other switches using trunk links. As you now know, VLANs are often used to represent departments or the occupations of a group of users. This makes the configurations of security policies and network access lists much easier to manage and control. On the other hand, if a port is accidentally assigned to the wrong VLAN in a switch, it's as if that client was magically transported to another place in the network. If that happens, the security policies that should apply to the client won't anymore, and other policies will be applied to the client that never should have been. The correct VLAN port assignment of a client is as important as air; when I'm troubleshooting a single-host problem, this is the first place I look.

Incorrect IP address

Symptoms: No connectivity between devices Causes: Either the source or destination device has an incorrect IP address. Resolution: Use Ping cmd to determine if there is connectivity between devices. Check IP addresses and empty the ARP cache on both computers.

Wrong subnet mask

Symptoms: No connectivity between devices Causes: Either the source or destination device has an incorrect subnet mask. Resolution: Use the ping cmd to determine if there is connectivity b/t devices. Check the subnet mask on both devices. Change the incorrect subnet mask to a correct one and test connectivity.

Incorrect antenna placement

Symptoms: No or low signal and connectivity. Causes: The position of your antenna can negatively affect overall performance if placed incorrectly. Resolution: Alter the position of your antenna and monitor device performance.

Untested updates

Test updates before pushing them to the APs in your wireless network.

wireless channel

The band of frequency used for wireless communications. Each IEEE wireless standard specifies the channels that can be used.

default gateway

The first IP address of the device that a client computer will look for when attempting to gain access outside the local network.

SFP/GBIC - cable mismatch

The small form-factor pluggable (SFP) is a compact, hot-pluggable transceiver used for networking and other types of equipment. It interfaces a network device motherboard for a switch, router, media converter, or similar device to a fiber-optic or copper networking cable. Due to its smaller size, SFP obsolesces the formerly ubiquitous gigabit interface converter (GBIC), so SFP is sometimes referred to as a mini-GBIC. Always make sure you have the right cable for each type of connector Itype and that they are not mismatched.

Duplicate MAC address

There should never be duplicate MAC addresses in your environment. Each interface vendor is issued an organizationally unique identifier (QUI), which will match on all interfaces produced by that vendor, and then the vendor is responsible for ensuring unique MAC addresses. That means duplicate MAC addresses usually indicate a MAC spoofing attack, in which some malicious individual changes their MAC address, which can be done quite easily in the properties of the NIC.

Latency

Time it takes for a bit to travel from its sender to its receiver.

Split Pair (Cable)

UTP/STP became untwisted, crosstalk not filtering out

Wrong DNS

Verifying your network connectivity is the most basic step you need to try in this situation. Then check if the server is doing load balancing across servers? Also, don't forget to check the server's forwarders. Then, try to ping some hosts and perform troubleshooting through NSLookup. Finally, alternate server DNS can also be tried and make sure the device is virus - free by performing a quick scan and then reboot the DNS server to complete the process

Exhausted DHCP scope

When a DHCP server is implemented, it is configured with a limited number of IP addresses. When the IP addresses in a scope are exhausted, any new DHCP clients will be unable to obtain an IP address and will be unable to function on the network. DHCP servers can be set up to provide backup to another DHCP server for a scope. When this is done, it is important to ensure that while the two DHCP servers service the same scope, they do not have any duplicate IP addresses.

incorrect Pinout/TX/RX

- Check for bad cable conectors - make sure the wireing is correct on both ends by Phisically checking the cable Pinouts

Check the Super simple stuff

- Check to verify login procedures and Rights - look for link lights and collision lights - check all power, switches, cords, and adapters - look for user errors

operator error

- Equipment exceeds operator capability (EEOC) - Problem exists between chair and keyboard (PEBCAK) - ID Ten T error (ID10T)

Step 1: Identify the problem

- Gather information by questioning users - Duplicate the problem, if possible - Determine if anything has changed - indentity symptoms - approach multiple problems individually

step 2: Establish a Theory of probable cause

- Question the obvious - consider multiple approaches

Troubleshooting steps

1. Identify the problem. 2. Establish a theory of probable cause. 3. Test the theory to determine cause. 4. Establish a plan of action to resolve the problem and identify potential effects. 5. Implement the solution or escalate as necessary. 6. Verify full system functionality, and if applicable, implement preventative measures. 7. Document findings, actions, and outcomes.

the correct login procedure and rights

1. Make sure the username and password is being entered correctly 2. check that Caps Lock Key 3. Try to log in yourself from another workstation assuming that doing this doesn't violate the security Policy. If it works, go back to the user-oriented login problems, and go through them again. 4. It none of this solves the problem, check the network documentation to find whether any of the aforementioned Kinds of restrictions are in place; if so, find out whether the user has violated any of them

wrong passphrase

When a passphrase is used as an authentication method, the correct passphrase must be entered when authenticating to the AP or to the controller. When an incorrect passphrase is provided, access will be denied. This is another issue that will impair functionality.

Reflection

When a wave hits a smooth object that is larger than the wave itself, depending on the media the wave may bounce in another direction. This behavior is categorized as reflection. Reflection can be the cause of serious performance problems in a WLAN. As a wave radiates from an antenna, it broadens and disperses. If portions of this wave are reflected, new wave fronts will appear from the reflection points. If these multiple waves all reach the receiver, the multiple reflected signals cause an effect called multipath. Multipath can degrade the strength and quality of the received signal or even cause data corruption or canceled signals. APs mitigate this behavior by using multiple antennas and constantly sampling the signal to avoid a degraded signal.

Blocked TCP/UDP ports

When the ports used by common services and applications are blocked, either on the network firewall or on the personal firewall of a device, it will be impossible to make use of the service or application. One easy way to verify the open ports on a device is to execute the netstat command.

once the theory is confirmed, determine the next steps to resolve the process,

When the testing of the theory is complete, you will have determined if the suggested cause is correct. If you find you are correct, the next steps (next section) is to establish a plan of action to resolve the problem and identify potential effects.

broadcast storm

When there is an accumulation of broadcast and multicast packet traffic on the LAN coming from one or more network interfaces.

Power failure/Power anomalies

When you have power issues, whether it's a full-blown power outage or intermittent power surges, it can cause some serious issues with your network devices. Your servers and core network devices require a fully functional UPS system

SSID Mismatch

A client has the wrong SSID specified, and cannot communicate with the access point

collision light

A light on some older NICs that flickers when a network collision is detected. Usually amber in color.

Tx/Rx reverse

A problem caused by mismatched pinout standards, resulting in near end cross-talk.

fiber type mismatch

A problem created by mismatching fiber core types, such as when connecting an SMF cable to an MMF cable.

Connector mismatch

A problem created by using the wrong connectors for each type of cable end or transceiver

wavelength mismatch

A problem created when transmissions are optimized for one type of cable, such as SMF, but sent over a different type of cable, such as MMF.

routing loop

A problem that happens when a message gets stuck in a loop between a limited number of routers without ever reaching its destination.

MTU black hole

A problem that occurs when a router receives a message that is too large for the next segment's MTU. The router returns an ICMP error message to the sender, but the error message is not returned correctly. From the sender's perspective, messages are lost for no apparent reason.

NIC teaming misconfiguration

A type of link aggregation in which two or more NICs work in tandem to handle traffic to and from a single node. If they are not configured correctly, either they will operate at a severly diminished capacity or, worse, neither cart will work at all

Incorrect ACL settings

Access control lists are used to control which traffic types can enter and exit ports on the router. When mistakes are made either in the construction of the ACLs or in their application, many devices may be affected. The creation and application of these tools should only be done by those who have been trained in their syntax and in the logic ACLs use in their operation.

shorts

An error in a cable caused by low resistance.

Wrong gateway

Any device encountering the problem of wrong or incorrect gateway can communicate with any host available on its own network segment but not with some or all hosts available on the remote network. A network using more than one router and if wrong router is set the default gateway, the host can communicate only with some remote networks. The problem can be solved by setting up a DLINK to work as a switch or put the same in bridge mode that can provide the link for communication between networks

Bad cables or connectors

Are the cables properly connected to the correct port? check the NIC to see if a link light blinking. If not then a bad patch cable may be to blame.

Switching loops

Caused by network switches that are broadcasting routes to one another to try to build a map of the network

Device saturation/Bandwidth saturation

Clearly it's important to design and implement your wireless network correctly. Be sure to understand the number of hosts that will be connecting to each AP that you'll be installing. If you have too much device saturation on an AP, it will result in low available bandwidth. Just think about when you're in a hotel and how slow the wireless is. This is directly due to device/bandwidth saturation for each AP. And more APs don't always solve the problem-you need to design correctly!

Rogue DHCP Server

DHCP server that has not been authorized within Active Directory. -Hands out unauthorized IP addresses

Port Speed

Depending on the cost of a switch, it may be possible to support a mixture of speeds. Ports of 100 Mb/s, and 1 or 10 Gb/s are common (100 Gb/s is also possible).

Mismatched MTU

Ethernet LANs enforce what is called a maximum transmission unit (MTU). This is the largest size packet that is allowed across a segment. In most cases, this is 1,500 bytes. Left alone this is usually not a problem, but it is possible to set the MTU on a router interface, which means it is possible for a mismatch to be present between two router interfaces. This can cause problems with communications between the routers, resulting in the link failing to pass traffic. To check the MTU on an interface, execute the command show interface.

Distance limitations

Fiber can transmit up to 40 Kilometers—about 25 miles

bounce

For a wireless network spanning large geographical distances, you can install repeaters and reflectors to bounce a signal and boost it to cover about a mile. This can be a good thing, but if you don't tightly control signal bounce, you could end up with a much bigger network than you wanted. To determine exactly how far and wide the signal will bounce, make sure you conduct a thorough wireless site survey. However, bounce can also refer to multipath issues, where the signal reflects off objects and arrives at the client degraded because it is arriving out of phase. The solution is pretty simple. APs use two antennas, both of which sample the signal and use the strongest signal and ignore the out-of-phase signal. However, 802.11n takes advantage of multipath and can combine the out-of-phase signals to increase the distance hosts can be from the AP.

Proxy ARP

a process that uses the same ARP messages as a normal ARP, but by which a router replies instead of the host listed in the ARP request. When a router sees an ARP request that cannot reach the intended host, but for which the router knows a route to reach the host, the router acts on behalf of the host and responds to the ARP request with the router's MAC address listed in the ARP reply

Bottleneck

areas of the network where the physical infrastructure is not capable of handing the traffic

Step 6: verify full system functionality, and if applicable implement preventative measures

before you fully implement the solution to the problem, make sure you totally understand the ramifications for doing so—clearly, if it causes more problems than it fixes, you should toss it and find a different solution that does no harm.

Step 7: Document findings, actions, and outcomes

document problems and soutions so that you have the information at hand whh a similar prolem aries in the future. Be sure to include the following: - A description of the conditions surrounding the problem _ the OS version, the software version, the type of computer, and the type of NIC - whether you were able to reproduce the problem - the solutions you tried - the ultimate solution

radio frequency interference

electromagnetic interference that is in the radio frequency range. Only way around this is to use Shielded network cables like Shielded twisted pair (STP) and coaxial cable or run EMI/RFI-immune but pricy fiber-optic cable throughout your entire network

Step 5: Implement the solution or escalate as necessary

esculate the following: - switching loops - Missing routes - Routing loops - Routing problems - MTU black hole - Bad modules - Proxy Address Resolution Protocol (ARP) - Broadcast storms - NIC Teaming misconfiguration - power failure/power anomalies

collisions

happens when two devices communicate on the same physical segment at the same time.

Distance/signal strength/Power levels

if your AP doesn't seem to have enough power to provide a connectivity point for your clients, you can move it closer to them, increase the distane that the AP can transmit by changing the type of antenna it uses, or use multiple APs connected to the same switch or set of switches to solve the problem.

Incorrect host-based firewall settings

incorrect host-based firewall settings can either prevent transmissions or allow unwanted communications. Neither of these outcomes is desirable. One of the best ways to ensure that firewall settings are consistent and correct all the time is to control these settings with a group policy. When you do this, the settings will be checked and reset at every policy refresh interval.

amplified

increased strength

Signal-to-noise ratio (SNR)

is the difference in decibels between the received signal and the background noise level (noise floor). If the amplitude of the noise floor is too close to the amplitude of the received signal, data corruption will occur and result in Layer 2 retransmissions, negatively affecting both throughput and latency. An SNR of 25 dB or greater is considered good signal quality, and an SNR of 10 dB or lower is considered very poor signal quality.

bad Ports

loopback plugs can be used to test the functionality of a port

incorrect Frequency/incompatibilities

make sure that your APs are all on different chanels/frequencies to avoid potential interference problems.

wireless Standard mismatch

makes sure the standards on the AP match the standards on the client, or that they're at least backwards compatible. Be sure to understand the throughput, frequency, distance capabilities, and available channels for each standard you use.

Bad modules (SFPs, GBICs)

need to verify information on hardware of the network and will need to use show diag prompt in EXEC mode. If the information is not present in the module, reset hardware module. Router will need to be reloaded after installing the module. Check for documentation on hardware installation as some modules also need configuration from router after installation. If an error message is displayed, it is the hardware incompatible issue. Check modules to make sure that they are in support with router

Crosstalk

occurs when the signals of one wire affect the signals on an adjacent wire

Near-End Crosstalk (NEXT)

occurs when there is measured interference between two pairs in a single cable, measured on the cable end nearest the transmitter.

Far-End Crosstalk (FEXT)

occurs when there is similar interference, measured at the cable end farthest from the transmitter.

incorrect, overlapping, or mismatched channels

overapping channels cause your signal-to-noise ratio to drop because you'll get a ton of interference and signal loss

routing problems

problems concerned with routing the delivery of goods or services to a set of destinations

top-down approach

start with the user application and work your way down the layers of the OSI model. If a layer is not in good working condition, you inspect the layer below it. When you know that the current layer is not in working condition and you discover that a lower layer works, you can conclude that the problem is within the layer above the lower working layer. Once you've determined which layer is the lowest layer.with problems, you can begin identifying the cause of them from within that layer.

bottom-up approach

starts with the physical components of the network and works its way up the layers of the OSI model. If you conclude that all the elements associated with a particular layer are in good working order, move on to inspect the elements associated with the next layer up until the cause(s) of the problem is/are identified. The downside to the bottom-up approach is that it requires you to check every device, interface, and so on. In other words, regardless of the nature of the problem, the bottom-up approach starts with an exhaustive check of all the elements of each layer, starting with the physical layer and working its way up from there.

Refraction

the bending of an RF signal as it passes through a medium with a different density, thus causing the direction of the wave to change. RF refraction most commonly occurs as a result of atmospheric conditions. In long-distance outdoor wireless bridge links, this can be an issue. An RF signal may also refract through certain types of glass and other materials that are found in an indoor environment.

Interference

the combination of two or more waves that results in a single wave.

jitter

the data flow in a connection is not consistent; that is, it increases and decreases in no discernable Pattern. Results from network congestion, timing, drift, and route changes.

Latency and overcapacity

the more users that correct to a session or a session is, all user throughput goes down. if it becomes a constant problem as opposed to the occasional issue, it may be time to consider placing a second AP in the area.

incorrect encryption/security type mismatch

to ensure the configuration the tightest Security, configure your wieless networks with the highest encryption protocol that both the WAP and the clients can support.

transceiver mismatch

two transceivers must have certain settings the same or issues will occur. These Settings are the duplex and speed setting. If the speed Settings do not match, there will be no communication. If the duplex settings are incorrect, there may be functionality out the performance will be poor.

Dirty connectors

verify your connectors to make sure no dirt or dust has corrupted the cable end. Polish with a soft cloth.

check the software configuration

when checking for software problems, don't forget to check types of configurations. Software-configuration Settings love to hide in places like these and can be notoriously hard to find (especially in the Registry)

Don't overlook Physical conditions

when troubleshooting an obscure network problem, don't forget to check the physical conditions under which the network device is operating. Check for problems like these: - Excessive heat - Excessive humidity (condensation) - low humidity (leads to [ESD] problems) - EMI/RFI problems - ESD problems - Power problems - Unplugged cables

Divide and Conquer

you select [an OSI] layer, test its health, and based on the results, you can move up or down through the model from the layer you began scrutinizing.

prioritize your problems

you start this process by again asking some basic questions to determine to determine the severity of the problem being reported. example rank, from highest priority to lowest: - Total network failure (affects everyone) - Partial network failure (affects small group of users) - Small network failure (affects a small, single group of users) - Total workstation failure (single user can't work at all) - Partial workstation failure (single user can't do most tasks) - Minor issue (single user has problems that crop up now and then)