ISEC200 Chapter 4

Linux Logs

Linux loves to make log files, dedicating an entire folder to nothing but all the logs you'll ever need. In most distros, this folder is called /var/log. Log files end with a .log extension and, unlike in Windows, there is no single log viewing app. You can add various GUI log readers, such as LOGalyze or glogg. The go-to log viewer on most Linux systems, journalctl, displays all logs in a system in a single format. journalctl also takes all the common Linux terminal arguments. Figure 4-35 shows journalctl using grep commands to filter on the term "warn." Linux logs are almost as simple as Windows logs in that there's basically a single source for all your log needs: syslog. OK, that's not exactly true, as syslog is over 30 years old and has been supplanted by two improved syslog versions: rsyslog, which came out in the late 1990s and is basically just an improved syslog, and syslog-ng, which is an object-oriented version of syslog. Right now syslog is probably the most popular version of these three syslog-like tools, but deciding which tool is best is really up to the individual user, as syslog, rsyslog, and syslog-ng all have their fans and detractors. In most cases, you'll end up using the form of syslog that works best for whatever system you want to use. (See the upcoming "Centralizing Log Files" section.) The one good thing about all versions of syslog is their superb standardized format for all log entries. This format is considered the standard for all log files. Even Windows log files can manifest in this format.

SIEM

The traditional way to manage logging and monitoring in a network was to sit down at a system and look at the log files generated by the system, reviewing them for the events of interest. This is not the most efficient way to manage network monitoring and analysis activities. Almost every organization now uses security information and event management (SIEM), an enterprise-level technology and infrastructure that collects data points from every host on the network, including log files, traffic captures, SNMP messages, and so on. SIEM can collect all this data into one centralized location and correlate it for analysis to look for security and performance issues, as well as negative trends, all in real time. SIEM unifies and correlates all the real-time events from a multitude of disparate sources, including network alerts, log files, physical security logs, and so on. You could still have real-time monitoring going on, but without combining and correlating all those different event sources. A true unified SIEM system is required to do that.

Dashboard

A SIEM dashboard presents the analyzed data in a way that makes sense to those monitoring the data and informs them of incidents taking place. Most SIEM dashboards provide graphs and counters.

Server

A SIEM server stores all the data coming in from multiple collectors. A single SIEM server might be a subserver that in turn reports to a main SIEM server.

pathping

A Windows utility that combines the functionality of the tracert and ping utilities to provide deeper information about network issues along a route; similar to UNIX's mtr command. pathping first runs a traceroute, but then pings each hop 100 times. All this pinging determines latency much more accurately than tracert does. Pathping is slower than tracert.

SIEM Security Monitoring

A good SIEM dashboard provides security monitoring—tools for watching and recording what's happening on your network. Beyond that, though, you can count on a SIEM dashboard to supply the following information: • Sensor list/sensor warning If an incident is taking place at a certain point, which sensor is giving that information? The sensor list or sensor warning provides that information. • Alerts Alerts enable the SIEM dashboard to inform the person(s) monitoring of a potential incident. This can be a warning ribbon at the bottom of the screen, an audible alarm, or a log entry shown in red. • Sensitivity How sensitive is a certain setting that might detect an incident? Too high and you'll get false positives. Too low and you'll get false negatives. • Trends Certain incidents make more sense when seen as a trend as opposed to an alert. Network usage is one good example. Techs can watch usage grow on a chat and consider those implications, as opposed to just getting some alert. Anyone who owns an automobile with an oil pressure gauge instead of an idiot light knows this feeling. • Correlation A good dashboard will recognize relationships between alerts and trends and in some way inform the person(s) monitoring of that correlation. This is often presented as line graphs with multiple data fields.

System Log

A system log file records issues that directly affect a single system but aren't network functions. System log files will show reboots, executable files starting, and edited files on the system, for example.

Ping Note (ping of death)

Administrators disable ICMP—and thus, ping requests—for a variety of reasons, but most notably because people can use ping maliciously. The ping of death sends malformed packets to a computer, possibly causing a crash. Continuous ping requests—as a denial-of-service attack—can also crash systems.

Sensors/Collectors

All SIEM solutions rely on sensors/collectors to acquire data about the network. These collectors manifest as special software running on a Windows system, an SNMP connection to a switch, or perhaps a third-party tool like NXLog providing information from a gateway router. Whatever the case, these devices must work together to bring the data into a single source, a SIEM server.

chmod

All files on a Linux system, including all log files and text files, have file permissions, or modes. File permissions are read (r), write (w), and execute (x). Type ls -l to see these permissions. Examining this output from left to right, the first character determines the type of entry: a hyphen denotes a regular file and a d denotes a directory. You can see the file or directory name on the far right of each line. The first four lines show the permissions for regular files, and the last line shows the permissions for the directory named timmy. Permissions are listed in three sets of three permissions each, all in a row. The first set shows the permissions for the owner/creator; the second set defines the group permissions; the third set describes permissions for everyone else, or other (Figure 4-13). If a set includes r, w, and x, it means all three of those permissions are assigned. A hyphen in place of a letter means that specific permission is not assigned. For example, the file2.txt file in the previous ls -l output has read and write permissions assigned for the owner and group, but only read permission for other. No one has execute permission (probably because it really is just a text file!). The directory timmy, in contrast, shows read, write, and execute permissions for both the owner and the group, plus read and execute (but not write) for other. The chmod command enables you to change permissions—or change modes—for a file or directory. A common way to edit these permissions is to give each set of three permissions a single numeric value between 0 and 7, representing the combined values of read (4), write (2), and execute (1). For example, if owner is assigned 0, owner has no permissions for the file or directory; if assigned 7, owner has all three permissions. Using three numbers with chmod can set permissions for any file. Here are some examples: • chmod 777 <filename> sets permissions to rwxrwxrwx • chmod 664 <filename> sets permissions to rw-rw-r-- • chmod 440 <filename> sets permissions to r--r------

Application Log

An application may have its own log file. What appears in this application log file requires some knowledge of the application that is using the log. Probably one of the most common application logs is for a Web server. Web server software is an application to share Web pages. In this case, since we know what Web servers do, we can assume the Web log keeps track of the number of pages served per hour/minute, perhaps even a listing of the different IP addresses asking for the Web page, or maybe the number of malformed HTTPS packets.

Authentication Log

An authentication log is a special type of security log that tracks nothing other than users attempting to log onto a system. This includes tracking failed logons as well as successful logons.

Analyzer

Analyzers take data from SIEM servers and, using a myriad of tools, look at the data to try to locate signatures of something that an organization would consider an incident.

Angry IP Scanner note

Angry IP Scanner and, in fact, most scanning tools will set off your anti-malware tools. In most cases you'll need to shut off the anti-malware; or better yet, go through whatever process is needed to make these scanners exempt from the anti-malware software.

DNS Log

Any good DNS server is going to keep a log. DNS logs are application logs that keep track of things appropriate to a DNS server application. DNS logs typically include entries for activities such as the creation of new forward lookup zones, cache updates/clearing, and changes to critical settings like root server.

tcpdump

As popular as Wireshark is, especially in Windows, many Linux/UNIX techs swear by tcpdump. tcpdump predates Wireshark by almost a decade and was the only real protocol analyzer option for Linux/UNIX until Wireshark was ported over in the very late 1990s tcpdump is amazing at sniffing packets, but it can be time consuming to use as a protocol analyzer, relying on complex command-line switches for sorting and filtering. No worries! Plenty of graphical protocol analyzers, even Wireshark, run on Linux

Security Log

Both systems and applications typically include security logs that record activities that potentially impact security. Security logs might track all successful and/or unsuccessful logon attempts. They track the creation or deletion of new users and also keep track of any permission changes to resources within the system or application.

Benefits of a protocol analyzer

Collecting and analyzing data via a protocol analyzer enables you to perform wildly detailed analyses. A protocol analyzer enables security professionals to access and use data sources to support an investigation in case of an incident. The protocol analyzer output—the data source—can provide critical information. A protocol analyzer can reveal all sorts of information, providing huge benefits to the security of your network. Here are six examples: • Count all the packets coming through over a certain time period to get a strong idea as to your network utilization. • Inspect the packets for single protocols to verify they are working properly (of course, this means you must know how these protocols work in detail). • Monitor communication between a client and a server to look for problems in the communication. • Look for servers that aren't authorized on the network. • Find systems broadcasting bad data. • Find problems in authentication by watching each step of the process in detail.

File Manipulation

Commands generate readable output. If you want to preserve output from a command for future reference, you can save the command output as a text file. In Linux, pretty much all log files (we're diving into log files later in Module 4-4) are text files that you can review, search, and edit. Several Linux utilities provide this capability: cat, chmod, grep, head, tail, and logger.

DNS Tools

DNS security issues can create huge problems for IT security professionals. Techniques such as spoofing or replacing a host's DNS server settings give an attacker tremendous insight as to whom the host is communicating with. While there are multiple tools and techniques used to diagnose DNS issues, two built-in OS utilities can be very useful: nslookup and dig.

Sniffer note

Different network technologies need different sniffers. If you want to analyze a wired network, you need a wired sniffer. If you want to sniff a wireless network, you need a wireless sniffer.

Windows Log Tools

Every Windows systems contains the venerable Event Viewer as the go-to log observation tool. Event Viewer contains a number of preset logs (notice the System logs folder in Figure 4-34). You can create new logs here or in the local security policy application.

Device Logs

Every enterprise-level (and many SOHO-level) devices come with some form of logging built into them. In most cases, you just need to log onto that device (often a Web interface) to see the logs on that device in action. The challenge to using these log files is that you have to access the device every time you want to see what's happening on that system. A better solution is where multiple devices send log files to a central source that can, in turn, analyze that information. Then that centralized analysis can display what's happening on the network in a way that administrators can react to the needs of the network. To do that requires centralization of far-flung log files into a single source.

arp

Every host on a network keeps a cache of mapped IPv4-to-Ethernet addresses for the local network. The arp command enables you to observe and administer this cache. Interestingly, both the Windows version and the Linux version use almost the same switches. If you want to see the current cache, type the command as follows (Windows version): arp -a The dynamic entries in the Type column show listings under control of DHCP. Static types are fixed (the multicast addresses that start with 224 never change) or are for statically assigned IP addresses. The arp command is only for IPv4 addresses. IPv6 uses the Neighbor Discovery protocol.

Items that network scanners will inventory

Every scanner scans differently, but they try to find certain common data. Here's a short list of some of the more important items every network scanner will inventory: • Topology • MAC addresses • IP addresses • Open ports • Protocol information on open ports • Operating systems A good network scanner compiles this information and presents it in a way that enables security people to understand what's going on in the network. A network scanner enables network mapping on many levels.

Powershell

For decades, Microsoft leaned heavily and exclusively on the Command shell. Compared to UNIX/Linux, Command was (and still is) extremely primitive and Microsoft knew this. After a few (better forgotten) attempts, Microsoft introduced PowerShell back in 2006. Since then PowerShell has gone through multiple improvements, making it arguably the best combination of shell and scripting language, certainly for the Windows platform. PowerShell's built-in scripting language has a unique feature that makes it vastly more powerful than any other language when it comes to working with the Windows operating system, a direct connection to the internals of Windows. Want to see a list of local users on a system? Try this PowerShell cmdlet (pronounced "command let"): Get-LocalUser | Select * Need to see a setting for a key in the Windows registry? That's easy for PowerShell: Get-ItemProperty -Path HKCU:\Software\ScriptingGuys\Scripts -Name version). version Feel free to integrate PowerShell cmdlets into the powerful scripting language to do, well, anything you can imagine. Create GUIs, access databases . . . PowerShell can do it all. PowerShell scripts normally end with a .ps1 file extension. Don't feel like using PowerShell scripting in your PowerShell terminal? No worries. The PowerShell terminal supports other languages such as JavaScript and Python. PowerShell is a fabulous shell as well as scripting language, and if you're working on or developing on a Windows platform, you need to know how to use it. PowerShell is also cross platform (but good luck finding anyone who runs PowerShell for a living outside of Windows systems).

IP Scanner

IP scanners are any class of network scanner that concentrates on IP addresses for network discovery. The one area that IP scanners do not usually cover are protocols underneath TCP/IP. For example, a TCP/IP scanner isn't commonly going to verify the Ethernet speed/type. Some network scanners do inventory lower protocol information, but they tend to be more specialized. With that in mind, let's dive into network IP scanners. In general, we turn to this information for two reasons: baselining and monitoring.

packet analyzer (sniffer)

If something is going to analyze packets, it is first going to need, well, packets! It needs some program that can put a NIC into promiscuous mode so that the NIC grabs every packet it sees. This frame/packet grabber program needs to monitor or sniff the wired or wireless network and start grabbing the frames/packets. So, a packet analyzer is really two totally separate functions: a packet sniffer that captures the frames/packets, and the packet analyzer that replays the collected packets and analyzes them This combination of sniffer and analyzer leads to a funny name issue. Any of the following terms are perfectly interchangeable: packet analyzer, network sniffer, network analyzer, and packet sniffer. Networks generate a huge number of packets. Even in a small home network, moving a few files and watching a few videos generates hundreds of thousands of packets. A good analyzer needs two features. First, it must provide some type of interface that allows the security professional a way to inspect packets. Second, it must provide powerful sorting and filtering tools to let you look at the packets that interest you while eliminating the ones you do not want to see.

tracert/traceroute

If you want to know how packets are routed from a host to an endpoint, try the Windows tracert command. Linux can use the almost identical command called traceroute (although it's not installed by default on many distros, including Ubuntu). Either command sends packets to each connection between the host and the endpoint, checking the time to live (TTL) between those connections. The power of tracert comes by running it before there are any problems. If you look at the previous example, you see the first two hops use a private IP address. This is correct, because this network has two routers between the host and the ISP—in this case, Comcast. The tracert successfully made it through both internal routers, but couldn't make it to the ISP. In this case we know there's something wrong between the gateway and the ISP. It could be that the ISP simply blocks ICMP packets (the protocol used with tracert in Windows). You can test this by running traceroute in Linux or macOS, which use UDP rather than ICMP by default.

ip

If you're looking to do anything serious in terms of IP and Ethernet information on a Linux (not macOS) system, the cool kid is the ip command. The ip command replaces ifconfig, doing many of the same tasks, such as viewing IP information on a system, checking the status of network connections, managing routing, and starting or stopping an Ethernet interface. The syntax differs from ipconfig and ifconfig, dropping a lot of the extra non-alphanumeric characters and shortening switch names. To see all the Ethernet and IP information for a system, for example, just type ip addr to get results like You can see the loopback address (lo) for both IPv4 (127.0.0.1/8) and IPv6 (::1/128) and the IPv4 and IPv6 addresses assigned to the Ethernet port (eth0). Typing ip addr does not show the MAC address. For that you'd type ip link.

Network Log

It you have a device or a piece of software that deals with networking, then you almost certainly have some kind of network log. A network log varies by the type of device using the network. A router might have a network log that tracks the number of connections per hour on every route. A switch might record packets per seconds for VLANs. On an individual host, you might log the usage of a particular NIC.

iwconfig

Linux systems also have the iwconfig command exclusively for wireless connections.

Syslogs (System logs)

Linux systems refer to them as syslogs, or system logs.

Event logs

Logs are databases, usually distinct files that record events on systems. You'll most commonly hear them referenced as event logs, as in the case of Microsoft Windows operating systems.

Security Information and Event Management (SIEM)

More complex systems rely on advanced monitoring tools such as a security information and event management (SIEM) application that monitors log files—brought into the monitoring tools using protocols like SNMP—to provide managers powerful, easy to read and understand toolsets. These toolsets enable them to react to situations on their network in real time.

OpenSSH

Most SSH implementations today rely on the OpenSSH suite of secure networking utilities. First released in 1999 and in continuous development since then, OpenSSH offers best-in-class secure connectivity between systems. OpenSSH is hosted at https://www.openssh.com/. Check it out—maybe you can use OpenSSH to develop the next great SSH server!

Python

Most security folks consider Python to be the "go to" scripting language for anything that's cross platform, because Python works perfectly and equally on Windows, macOS, and UNIX/Linux systems. Python has been around for a very long time, is totally free, is well known and well supported, and has easy-to-find tutorials and support.

NXLog

NXLog is one example of many solutions out there that provide centralized log monitoring. NXLog is cross platform and takes advantage of darn near every and any protocol out there (including syslog and SNMP) to bring log data together. On Linux systems, NXLog reads from both a local system's syslog and NXLog's installed daemon. On Windows, NXLog runs its own agent and also uses SNMP. NXLog is a powerful logging tool, but it is not designed to provide pretty desktops or to analyze the log data. NXLog is ready and able to give this log data to those types of systems, but that means we need to start talking about SIEM.

TCPView

Need something a little more graphical in your life? There are many interesting graphical tools that enable you to access these commands graphically. Graphical tools often provide easier-to-grasp information that the command-line tools lack. Finding netstat a little frustrating? Try TCPView from Windows Sysinternals instead of netstat.

Scanner Types

Network scanners tend to fit into one of two categories: simple or powerful. Simple scanners make quick, basic scans. They require little or no actual skill and only provide essential information. Powerful network scanners use many protocols to drill deeply into a network and return an outrageous amount of information.

Nmap Basics

Nmap is run from a command line. Nmap scans the 1000 most common port numbers by default. In this case it located four open ports: 53 (my router is a DNS forwarder), 80 (the router's Web interface), and 515 and 9100 (I have a printer plugged into the network). Nmap can just as easily scan an entire network ID. For example, this Nmap command will run the same command except generate a very long output to the terminal: nmap 192.168.1.* By adding a few extra switches, -A to tell Nmap to query for OS version and -v to increase the "verbosity" (in essence telling Nmap to grab more information), you can get a lot more information. Nmap also works perfectly fine for any public-facing server. Note that not only does Nmap detect open ports, it pulls detailed information about each port, including the RSA key for the SSH server, the HTTP server version, and the title to the index page as well as NetBIOS ports.

Zenmap

Nmap outputs can be massive, so in many cases security professionals don't use this interactive mode of terminal screen output; instead, they direct output to XML files other tools can analyze, including graphical tools. Nmap provides a wonderful GUI called Zenmap that enables you to enter Nmap commands and then provides some handy analysis tools. As previously mentioned, this is just the lightest of tastes of the powerful Nmap scanner. Nmap is a critical part of many Linux security packages, including the powerful Metasploit framework.

SIEM Review Reports

No SIEM system is perfect. Review reports detail the results of examining the success rate of a SIEM system over a certain timeframe. These review reports tell the administrators of the SIEM system where they need to improve.

Dump Files

On some operating systems, a dump file is generated when an executable program crashes. These dump files record memory locations, running processes, and threads. Dump files are almost always used exclusively by the developers of the executable file that needs . . . dumping.

Security trail (audit trail/audit logs)

Operating systems also store log files in various locations and in various formats. Security personnel often refer to these logs as security or audit trail, or sometimes as audit log, because they provide information on events that are of a security interest. Security logs audit access to a particular resource or system. Audit logs can record very specific information about an event, to include user names, workstation or host names, IP addresses, MAC addresses, the resource that was accessed, and the type of action that occurred on the resource. This action could be something such as a file deletion or creation; it also could be use of elevated or administrative privileges.

PingPlotter

Or maybe replace boring old ping/traceroute with PingPlotter from Pingman Tools

Nmap (Powerful Network Scanner)

Originally written back in the mid-1990s by Gordon Lyon, Nmap established itself as the gold standard for TCP/IP network scanners early on and has retained its prominence through ongoing updates from the Nmap community. Nmap runs on most operating systems, but it runs best on Linux/UNIX systems. Having said all these nice things about Nmap, be warned: Nmap is a command-line tool with a powerful and incredibly complex number of switches and options. Nmap is complicated to master; there are many thick books and long online Web page Nmap tutorials that do a great job showing you the power of Nmap. (Also, GUI overlays that run on top of Nmap are available, such as Zenmap, discussed shortly.)

Protocol analyzers

Protocol analyzers collect and inventory the network traffic on a network. The IT industry defines protocol analyzers as any type of hardware or software that analyzes any form of communication. In IP networks, all data transmits via Ethernet frames or IP packets contained in those frames, so the better term is packet analyzer (Figure 4-24). Yet another term you'll hear many folks in the IT industry use is packet sniffer (or just sniffer).

netstat -b

Running netstat with the -b option displays the executable file making the connection.

SIEM Analysis

SIEM analysis is a massive topic, but CompTIA only wants you to consider two types of analyses, user behavior analysis and sentiment analysis. User behavior analysis, as the name implies, considers how users behave on a network. Does Maria always log onto the same machine every day? Does Mario add new users to his department within a day after they are hired? When users move outside their anticipated behaviors, that may be indicative of trouble. Sentiment analysis tries to parse actual language used to determine the meaning behind that use. The system monitors things like publications for negative sentiment to determine intentions of a threat group, for example.

SIEM Infrastructure

SIEM isn't a law or a standard. It's an integrated approach to monitoring networks that enables security professionals to react on a timely basis to incidents. SIEM systems employ certain components, discrete or combined with others, but you can count on the following: • Sensors/collectors • Server • Analyzers • Dashboard

SIEM: Log aggregation

SIEM servers can't just have data dumped on them. First, data needs to be normalized, to regularize the same type of data among multiple logs. One great example of this is log aggregation. What if one log stores all source IP address information as "SRC_IP" while another log stores the same data as "Source"? Proper log aggregation will make the data output under a single label. Another example is too much data. Does the server need to store every IP packet going through a switch if all it really needs to know is the number of packets per second?

SIEM Data Inputs

SIEM solutions grab data from many different sources. Most SIEM solutions use log collectors—tools for recording network events—and SNMP inputs, of course, but many go far beyond those sources. For example, it's not uncommon for a SIEM solution to employ agents that perform packet capture for situations where that might be needed.

Shell and script environments

Security experts understand operating system shells, the command-line interfaces (such as Command, Bash, Terminal, etc.) that enable you to do a ton of things quickly and with authority. A huge part of working at the command prompt is using scripting languages. Scripting languages enable automation of complex tasks and, with the right shells, take advantage of powerful operating system features (like updating registry settings in Windows) that would otherwise require writing actual compiled code. The funny part is that the CompTIA Security+ objectives only mention one shell—Windows' PowerShell—and only one scripting language, Python. OK, PowerShell also includes its own scripting language, but CompTIA Security+ seems to skip popular shells such as GNU Bash (a Linux shell) and scripting languages like JavaScript.

Session Initiation Protocol Traffic

Session Initial Protocol (SIP) traffic is usually a subset of VoIP traffic but exclusive to the SIP protocol. In this case, a SIP traffic log tracks where the IP address to/from is logged as well as any details about the call itself.

Simple Network Scanners

Simple scanners are easy to use and invariably have pretty graphical user interfaces (GUIs) to improve usability. The free and surprisingly powerful Angry IP Scanner from Anton Keks (https://angryip.org) does a good job using simple protocols, mainly ping, to query a single IPv4 address or an address range. If you just want to count the systems in a simple LAN and you have confidence that your internal network isn't blocking ICMP messages, Angry IP Scanner is a great tool. Angry IP Scanner is good for a single network ID and it does some basic port scanning, but that's about it. Simple scanners aren't limited to Windows. macOS users have similar tools. One of the more common and popular is IP Scanner by 10base-t Interactive (https://10base-t.com). Like Angry IP Scanner, this tool uses only the simplest protocols to query a local network. There are also several excellent simple network scanners for smart devices running Android and iOS operating systems. Given that these devices all have dedicated 802.11 connections, almost all of these simple IP network scanners tend to give excellent wireless information. One of our favorite scanners is Fing (https://www.fing.com).

Powerful Network Scanner

Simple scanners are wonderful tools if you're looking for quick scans that grab IP addresses, open ports, and maybe a Windows network name, but little else. But what if you're looking for more? What if you have multiple address ranges? What if you have a routed network and you want to install multiple sensors that all report back to a single system? In that case you'll need to turn to a powerful network scanner.

Secure Shell Protocol (SSH)

Sometimes you just can't physically be in front of a machine to open a terminal and do whatever you need to do. That's where the Secure Shell (SSH) protocol comes into play. Applications using the SSH protocol can manifest a terminal to a remote machine, assuming you have a user name and password on that remote machine (and that remote machine is running an SSH-compatible server). SSH runs on TCP port 22, and almost every operating system comes with a built-in SSH client, if not an SSH server as well. SSH servers and clients must first create an encrypted connection. The SSH protocol has several ways to do this, but one of the more common methods is for the SSH server to generate SSH keys: a traditional RSA asymmetric key pair. The server then treats one key as public and the other as private. When an SSH client attempts to access an SSH server for the first time, the server sends the public key

head/tail

Sometimes you want to see only the beginning of a text file to get a clue about what data is stored in that file. Alternatively, there are times where you just want to see the end of a text file, maybe to see if records were added. That's where head and tail are used. head displays the first ten lines of a text file. tail shows the last ten lines.

grep

Text files sometimes get large, making it challenging to find a specific bit of data in them you need. The grep command looks for search terms (strings) inside text files and returns any line of that text file containing the string you requested. • Find the word "mike" in database.txt: mike@mike-VirtualBox:~$ grep mike database.txt mike meyers 1313 mockingbird lane Houston TX The grep utility isn't limited to files. You can take any output to the terminal and then "pipe" the output through grep to filter out anything you don't want. The handy ps aux command shows all the running processes on a Linux system, but the output can be massive. Let's say you want to see all the Firefox processes on your system. The command ps aux | grep firefox tells the system to send (to pipe) the output to the grep command, not to the screen. The output runs through the grep command and then goes to the screen. The grep utility is a wonderful tool to filter output to and from files or the screen. This quick introduction is just that—an introduction and enough to get you through the exam—but anyone who works with Linux should on their own dig deeper into grep.

NetFlow/sFlow

The Cisco-based NetFlow utility provides real-time information about all the IP traffic in a system. sFlow provides similar information, but, unlike NetFlow, runs in hardware. NetFlow is a software implementation. Numerous switch and router manufacturers deploy sFlow chips/technology, so it is not limited to Cisco hardware. In concept, a flow represents the movement of an IP packet through a network. By monitoring such information, NetFlow/sFlow can pick up very quickly when patterns diverge from what would be considered normal traffic. Flow information is stored in logs or flow caches that the tools can analyze. Spawned from NetFlow version 9, Internet Protocol Flow Information Export (IPFIX) provides more flexibility in the types of information that can be combined and saved for analysis, such as mail records, HTTP URL information, SIP data, and more. IPFIX is an Internet Engineering Task Force (IETF) specification (RFC 7011), is backwardly compatible to NetFlow v9, and is widely adopted in the industry.

Simple Network Management Protocol (SNMP)

The Simple Network Management Protocol (SNMP) enables proactive monitoring of network hosts in real time. Among other things, SNMP is a bandwidth monitor, providing up-to-the-second information for network administrators. This real-time information can provide critical data sources to support investigations in the face of an incident. Devices that use SNMP have a small piece of software known as an agent installed on them, and this agent can report certain types of data back to a centralized monitoring server. This agent is configured with a Management Information Base (MIB), which is a very specific set of configuration items tailored for the device. Using the MIB, the agent can relay very specific event information to a centralized console in real time. These events are known as traps and are essentially specific events configured with certain thresholds. If a configured threshold is reached for a particular event, the trap is triggered and the notification or alert is sent to the management console. These traps and messages can let an administrator know about the device status and health. In other words, the administrator could get information on when the device is up or down, whether its hardware is functioning properly or not, if its memory or CPU use is too high, and other interesting details about the device. In some cases, these trap messages could indicate an active attack against the device or some other type of unauthorized access or misuse.

arp command note

The arp command (lowercase) displays information on aspects of the system that use the Address Resolution Protocol (ARP). The mapped cache of IP-to-Ethernet addresses, for example, is the ARP cache. arp and other command-line utilities listed here are lowercase to reflect the case-sensitivity of UNIX/Linux systems.

ARP spoofing

The arp command enables detection of ARP spoofing, when a separate system uses the arp command to broadcast another host's IP address. Given that most operating systems return an error when this happens, you might think using arp isn't helpful. But arp at least provides the MAC address for the spoofing system, which might help you determine the offending system. (A lot of modern switches employ Dynamic ARP Inspection [DAI] to deal with ARP spoofing automatically, though that's beyond the discussion here.)

cat

The cat command enables you to combine—concatenate—files. Concatenate two files to the screen with the following command: cat file1.txt file2.txt To concatenate these same two files and create a new file, use the following command: cat file1.txt file2.txt > file3.txt The cat utility also enables you to view the contents of any text file. Just type cat followed by the text file you wish to see and it appears on the screen: mike@mike-VirtualBox:~/Desktop$ cat onelinefile.txt This is the one line in this file The cat utility provides a quick, powerful, and versatile tool for working with text files from the command line. You can view log files, for example, and combine multiple logs. To search the log files, you can combine cat with grep (which is discussed shortly). cat is a go-to tool for Linux users. cat is a fine tool, but explore another tool for current systems (that's not on the exam) called less.

dig

The dig command is a Linux-based DNS querying tool that offers many advantages over nslookup. dig works with the host's DNS settings, as opposed to nslookup, which ignores the host's DNS settings and makes queries based on the variables entered at the command line. dig also works well with scripting tools. The dig tool is unique to Linux, although you can find third-party Windows dig-like tools. dig is simple to use and provides excellent information with an easy-to-use interface. Try other dig options. For example, if you want to know all the mail server (MX) servers for a domain, just add the MX switch

logger

The logger command enables you to add text to log files manually. You would do this when you want to add comments. We'll get to log files in Module 4-4.

Metadata

The metadata from applications, such as Microsoft Office 365, provides a great source, because it includes e-mail logs, mobile device records, Web traffic, and file manipulation details. Another source is logs from devices running protocols designed specifically for network monitoring, such as the Simple Network Management Protocol (SNMP).

netstat

The netstat command is the go-to tool in Windows and Linux to get any information you might need on the host system's TCP and UDP connections, status of all open and listening ports, and a few other items such as the host's routing table. Typing netstat by itself shows all active connections between a host and other hosts: You can see that the first few lines of any netstat show a number of active connections with the loopback address (the 127.x.x.x). These are used by Microsoft for several different information exchanges such as the Cortana voice recognition utility. The preceding netstat output shows the open connections on this system, mainly HTTPS connections for Web pages. In the State column on the right, ESTABLISHED identifies active connections and CLOSE_WAIT indicates connections that are closing.

Netstat Note(s)

The netstat command works perfectly well with IPv6. netstat is an incredibly powerful tool, and this short description barely touched its capabilities. For the CompTIA Security+ exam, you should experiment with netstat both in Linux and in Windows.

nslookup

The nslookup tool, built into both Windows and Linux, has one function: if you give nslookup a DNS server name or a DNS server's IP address, nslookup will query that DNS server and (assuming the DNS server is configured to respond) return incredibly detailed information about any DNS domain. For example, you can run nslookup to ask a DNS server for all the NS (name server) records for any domain This power of nslookup has been used for evil purposes. For that reason, almost no public DNS server supports nslookup anymore for anything but the simplest queries. But there are still good reasons to use nslookup. nslookup works similarly in both Linux and Windows. This is good. The authoritative DNS server is configured to protect this private server and therefore not respond to an nslookup query. nslookup can tell you if an IP address is a functioning DNS server. Run nslookup interactively by typing nslookup to get a prompt; Then type in the IP address of the DNS server you want to check; Note that it returns a DNS name. This means there is a reverse DNS zone; a very good sign, but still not proof; This could mean that the server IP address is not a functional DNS server. Or, at the very least, that the DNS service on this particular server isn't working.

ping

The ping utility enables you to query another system on a TCP/IP network to determine connectivity. ping uses Internet Control Message Protocol (ICMP) packets at OSI Layer 3, Network, for queries. It's not a perfect tool, because many systems disable ICMP traffic to avoid attacks, but ping is quick and effective enough that you'll find it very useful. When one system successfully pings another host, you automatically know that the other system is connected and is properly IP addressed. By default, ping runs four times in Windows and stops automatically. In Linux, ping runs continuously until you press ctrl-c. To get ping to run continuously in Windows, use the -t switch. The ping utility has uses beyond simply verifying connectivity. Running ping using a DNS name, for example, is a great way to verify you have a good DNS server. Look at Figure 4-3. Note that the ping is unsuccessful (shows Destination port unreachable), but the ping successfully resolves the DNS name (www.cheese.com) to an IP address (195.149.84.153).

Ping Switches

The ping utility offers many more features that IT security professionals use all the time. Here's a list of some of the more useful switches: • -a Resolve addresses to hostnames • -t Run continuously • -f Set Don't Fragment flag in packet (IPv4 only) • -4 Force using IPv4 • -6 Force using IPv6

Route

The route command enables you to display and edit a host's routing table. Inspecting a routing table enables you to find problems if packets leave your system but never get a response. In Linux you type route to see the routing table; in Windows you type route print The operating system automatically generates routing tables on individual hosts based on the network settings. So using route to display the routing table seems logical. On the other hand, you can use route to make changes to the routing table for a variety of reasons. You could set up a static route for security purposes, for example, or remove a route to a specific host to force a secondary gateway.

route note

The route command rarely comes with Linux distros. Use the apt or rpm command to install the net-tools package, which includes plenty more than route as well.

Baseline

Think of a baseline as a verification of the network assets. Security professionals run baselines to make sure of exactly what is on the network. This a great time to build an "official" inventory that defines and details every computer, server, printer, router, switch, and WAP in the network infrastructure. Once you know what's on a network, you can then occasionally reapply a network scan to verify that there are no surprises on the network. Compare recent scans to the baseline and look for differences. Any sort of baseline deviation—things that have changed—can help secure the network. In such a scenario, monitoring enables rogue system detection, such as finding rogue WAPs, unauthorized servers, and other such security breaches.

netstat -a

Typing netstat -a in Windows or Linux shows the same information as netstat alone, but adds listening ports. This is a very powerful tool for finding hidden servers or malware on a host. Look carefully at the following command. Notice that HTTP port 80 and HTTPS port 443 are listening. This instantly tells you that the host is an HTTP/HTTPS server. You must then answer the question: "Should this system be running a Web server?"

Netstat in Linux

Typing netstat in Linux gives the same information, but in a slightly different format. At the very bottom are the associated UNIX sockets. A socket is an endpoint for connections. This data is very long and not very useful.

ifconfig

Use the ifconfig command in UNIX/Linux and macOS operating systems to show the current status of the network settings for a host system. This shows a lot of configuration settings for the Ethernet connection (enp0s3), including the MAC address (HWaddr), IPv4 address (inet addr), broadcast IP address, and subnet mask. You can see the IPv6 address (inet6 addr) and many more details. The lo connection is the loopback, which is the expected 127.0.0.1. Unlike ipconfig, ifconfig goes beyond basic reporting, enabling you to configure a system. The following example sets the IP address and the subnet mask for the Ethernet NIC eth0: sudo ifconfig eth0 192.168.0.1 netmask 255.255.255.0

Ipconfig

Use the ipconfig command in Windows to show the current status of the network settings for a host system. Figure 4-4 shows sample output from ipconfig in Windows 10. You can see the various IPv6 addresses plus the IPv4 address and subnet mask. Typing ipconfig by itself also shows the default gateways.

ipconfig switches

Using switches enhances the ipconfig output; the following six switches are particularly useful. Typing ipconfig /all, for example, lists virtually every IP and Ethernet setting on the system. • /all Get exhaustive listing of virtually every IP and Ethernet setting • /release Release the DHCP IP address lease • /renew Renew the DHCP IP address lease • /flushdns Clear the host's DNS cache • /displaydns Display the host's DNS cache

VoIP and call manager logs

Voice over IP (VoIP) and call manager software solutions create logs that store information about the calls themselves. Phone numbers and duration of calls are the two most common items logged, but items from other VoIP tools such as billing might also be included.

Network Scanners

What is on your network? This is one of the most important questions a good IT security professional must answer. When you want to see what's on a network, you need to go through the process of network reconnaissance and discovery. Network scanners use protocols on a network, almost always a LAN, to determine as much information about the LAN as possible. In essence, a scanner is an inventory tool. There's no single standard or type of network scanner. Different network scanners use a variety of protocols, standards, and methods to query the hosts on the LAN. Given that virtually every network in existence uses the TCP/IP protocol stack, it might be more accurate to use the term IP scanner when describing these tools. Network scanners are everywhere. If you go into Network and Sharing Center in Windows (assuming your firewall settings aren't too restrictive), you'll see a perfectly functional, although very simple, network scanner

netcat (nc) note

Windows lacks an equivalent to netcat, but Ncat from https://nmap.org/ncat/ has very similar functionality.

Wireshark API(s)

Wireshark employs an application programming interface (API) to enable a NIC to ingest all traffic passing by, rather than the default of only traffic intended for the box. Wireshark uses the libpcap API in Linux or the WinPcap API on Windows systems.

Wireshark

Wireshark is the Grand Old Man of packet analyzers, originally developed in 1998 by Gerald Combs as Ethereal. In 2006, Ethereal was forked from the original development team and renamed Wireshark. Wireshark may be old, but it has an amazing development team that keeps this venerable tool sharp with a huge number of updates. Wireshark is not only powerful but completely free and works on all major operating systems (and quite a few not so common ones as well). Its default GUI is so common that even a few competing protocol analyzers copy it for theirs. The interface has three main panes. At the top is a list of all the frames currently captured. (Wireshark shows each Ethernet frame, a feature that few other protocol analyzers provide.) In the middle are the gritty details of the frame currently selected in the top frame. At the bottom is the raw frame data in hexadecimal. You begin the process of using Wireshark by starting a capture. Wireshark locates your network interfaces and gives you the opportunity to select which interface you want to start capturing. Once you start a capture, at some point you then stop the capture. From there you can filter and sort the capture by source or destination IP address, source or destination port number, protocol, or hundreds of other criteria. Wireshark has an expression builder to help you with complex filters. You can use a filter on the incoming packets or you can just sniff everything and filter afterward. Wireshark by default captures and analyzes packets in real time, so you can analyze as things happen. Sometimes, on the other hand, you need to record data over a period of time and then run tools for analysis for baselining or historical reference. Imagine a situation in which a tech wants to capture all HTTP traffic between 12 a.m. and 4 a.m. on a specific server. In this case, instead of running Wireshark all night, he or she will turn to a capture-only tool such as TShark. TShark is a command-line tool that works well with scripting and scheduling tools (Figure 4-29). TShark saves the capture in a format called pcap that a tech can load into Wireshark the next day for analysis.

Visitor logs

are resource-oriented and list all the different types of access that occur on a resource (such as a shared folder, file, or other object). Most systems can produce visitor logs for a specific resource, but often this must be configured on the resource itself to generate those logs. You can use visitor logs that are not system-generated. For example, you can have manually generated visitor logs for entrance into a facility or restricted area. Security experts work hard to keep up with all the different log files available and use them in the most efficient and reasonable way possible.

netcat (nc)

is a terminal program for Linux that enables you to make any type of connection and see the results from a command line. With nc, you can connect to anything on any port number or you can make your system listen on a port number. The nc command is a primitive tool. To get any good information from the connection, the user must know the protocol well enough to type in properly formed input data. Since this is a Web page, type get index.html HTTP/1.1, as this is what the Web server is expecting. The challenge and the power of nc come from the fact that it's a tool for people who know how to type in the right commands, making it great for penetration testing or, if you're evil, hacking. Imagine making a connection to a server and typing in anything you want to try to fool the server into doing something it's not supposed to do! The nc command even works as a handy scanning command. Find a server, guess on a port number, and try connecting! The nc command works perfectly with scripts. It's relatively easy to write a script that tries 1024 port numbers one after the other, automating the scanning process.

Log Management

refers to the logistics of managing all the device logs in a network. In addition to reviewing log files on a periodic basis, you'll need to consider a few other aspects of log management. First, you should decide how long to keep log files, since they take up a lot of space and may not exactly contain a lot of relevant information—unless you need to look at a specific event. Sometimes governance, including regulations and other legal requirements, may require you to keep system log files for a certain amount of time, so log management is also concerned with retention of archived log files for a defined period. Secure storage, retention, and retrieval of archived logs are required simply because log files can contain sensitive information about systems and networks, as well as other types of sensitive data. You must ensure that only personnel authorized to view log files have access to them and protect those log files from unauthorized access or modification. You must also be able to retrieve the log files from storage in the event of an investigation or legal action against your organization, as well as comply with audit requirements imposed by law. Log management applies to all of these things, and effective policy regarding managing the logs in the organization should address all these issues. Security professionals look for particular types of information on a routine basis, and thus should configure logging to provide only that pertinent information. This helps to find the relevant information and eliminate the massive amounts of unnecessary information. For event-specific information, such as a major performance issue, an outage, or even a significant security event, a lot more information may be needed during those specific circumstances only. In such a case, security professionals may temporarily configure logging to provide far more detailed information. Getting that level of detail on a routine basis, however, would likely make logs unmanageable and limit their usefulness. For example, let's say that you are monitoring a potential hacking attempt. You may change the audit level of your network devices or internal hosts to include more detailed information on processes, threads, and interactions with lower-level hardware. This level of detail can produce a vast amount of logs very quickly, which can fill up device storage and be very unwieldy to examine. Because of this, you'd want to enable this level of logging only for a short while, just long enough to get the information you need. After that, you'd want to reduce the logging level back to normal.

Syslog

syslog and its alternative forms are more than just log tools. syslog is a complete protocol for the transmission and storage of Linux logs into a single syslog server, configured by the network administrators. Once all the files are stored in a single location, we can use tools such as journalctl to monitor the entire network.

tracert note

tracert is often a great way to determine the ISP just by looking at the router names.