Chapter 8

Univariate Model

Each metric is treated as an independent random variable

Common Sensor Data Sources

- System Call Traces - Audit (Log file) Records - File Integrity Checksums - Registry Access

Analysis Approaches

- Anomaly Detection - Signature/Heuristic Detection

Sensor Modes

- Inline - Passive

Time-Series Models

Use the order and time between observed events to better classify the behavior

Attack Types Suitable for Signature Detection (NIDS)

- Application layer reconnaissance and attacks: (Buffer overflow, password guessing and malware transmission) - Transport layer reconnaissance and attacks: (Unusual packet fragmentation, scans for vulnerable ports, TCP attacks like SYN floods) - Network layer reconnaissance and attacks: (Spoofed IP addresses and illegal IP header values) - Unexpected application services: (Host running an unauthorized application service) - Policy violations: (Use of inappropriate web sites and forbidden application protocols)

Intruder Skill Levels

- Apprentice - Journeyman - Master

Machine-Learning Approaches

- Bayesian Networks - Markov Networks - Neutral Networks - Fuzzy Logic - Genetic Algorithms - Clustering and Outlier Detection

Classes of Intruders

- Cyber Criminals - Activists - State-Sponsored Organizations - Others

Attack Types Suitable for Anomaly Detection (NIDS)

- Denial-of-service (DoS) attacks - Scanning - Worms

Distributed IDS Architecture Components

- Host Agent Module - LAN Monitor Agent Module - Central Manager Module Designed to be independent of any operating system or system auditing implementation

Distributed IDS Requirements

- May need to deal with different sensor data formats (Heterogeneous environment) - One or more nodes in the network will serve as collection and analysis points for the data from the systems on the network -- Need confidentiality and integrity - Either a centralized or decentralized architecture can be used

Desirable IDS Aspects

- Run continually with minimal human supervision. - Must be able to recover from system crashes and reinitializations (Fault tolerant) - Resist subversion. The IDS must be able to monitor itself and detect if it has been modified by an attacker. - Impose minimal overhead on the system - Configurable according to the security policies of the system that is being monitored. - Able to adapt to changes in system and user behavior over time. - Able to scale to monitor a large number of hosts. - Provide graceful degradation of service in the sense that if some components of the IDS stop working for any reason, the rest of them should be affected as little as possible. - Allow dynamic reconfiguration; that is, the ability to reconfigure the IDS without having to restart it.

IDS Components

- Sensors - Analyzers - User Interface

Worm Network Warning Signs

- Some worms propagate quickly and use large amounts of bandwidth. - Worms can also be detected because they can cause hosts to communicate with each other that typically do not, and they can also cause hosts to use ports that they normally do not use. - Many worms also perform scanning

Anomaly Classification Approaches

- Statistical - Knowledge based - Machine-learning

Input Types (SIEM)

- Summary Events - DDI Events - PEP Events

Attack Steps

- Target Acquisition and Information Gathering - Initial Access - Privilege Escalation - Information Gathering or System Exploit - Maintaining Access - Covering Tracks

Typically Logged Information (NIDS)

- Timestamp (usually date and time) - Connection or session ID (typically a consecutive or unique number assigned to each TCP connection or to like groups of packets for connectionless protocols) - Event or alert type Rating (e.g., priority, severity, impact, confidence) - Network, transport, and application layer protocols - Source and destination IP addresses - Source and destination TCP or UDP ports, or ICMP types and codes - Number of bytes transmitted over the connection - Decoded payload data, such as application requests and responses - State-related information (e.g., authenticated username)

Autonomic Enterprise Security Motivations

1. IDSs deployed selectively may miss a network-based attack or may be slow to recognize that an attack is under way. The use of multiple IDSs that share information has been shown to provide greater coverage and more rapid response to attacks, especially slowly growing attacks 2. Analysis of network traffic at the host level provides an environment in which there is much less network traffic than found at a network device such as a router. 3. Host-based detectors can make use of a richer set of data, possibly using application data from the host as input into the local classifier

File Integrity Checksums

A common approach to detecting intruder activity on a system is to periodically scan critical files for changes from the desired baseline, by comparing a current cryptographic checksums for these files, with a record of known good values. - Disadvantages include the need to generate and protect the checksums using known good files, and the difficulty monitoring changing files.

Typical NIDS Sensor Placement

A common location for a NIDS sensor is just inside the external firewall This position has a number of advantages: - Sees attacks, originating from the outside world, that penetrate the network's perimeter defenses (external firewall). - Highlights problems with the network firewall policy or performance. - Sees attacks that might target the Web server or ftp server. - Even if the incoming attack is not recognized, the IDS can sometimes recognize the outgoing traffic that results from the compromised server.

IETF Intrusion Detection Working Group

A group within the IETF focused on creating standards of interoperability to facilitate the development of distributed IDS's that can function across a wide range of platforms and environments - Data formats - Exchange procedures

Security Information and Event Management (SIEM)

A method for analyzing risk in software systems. It is a centralized collection of monitoring of security and event logs from different systems. SIEM allows for the correlation of different events and early detection of attacks.

System Call Traces

A record of the sequence of systems calls by processes on a system, is widely acknowledged as the preferred data source for HIDS - Work well on Unix and Linux, but not Windows due to the extensive use of DLLs that obscure which processes use specific system calls

Security Intrusion

A security event, or a combination of multiple security events, that constitutes a security incident in which an intruder gains, or attempts to gain, access to a system (or system resource) without having authorization to do so

Intrusion Detection

A security service that monitors and analyzes system events for the purpose of finding, and providing real-time or near real- time warning of, attempts to access system resources in an unauthorized manner

Host Audit Record (HAR)

A standardized format for audit records collected by host agents in a distributed IDS for analyzation

Information Gathering or System Exploit

Actions by the attacker to access or modify information or resources on the system, or to navigate to another target system.

Maintaining Access

Actions such as the installation of backdoors or other malicious software as we discuss or through the addition of covert authentication credentials or other configuration changes to the system, to enable continued access by the attacker after the initial attack

Privilege Escalation

Actions taken on the system, typically via a local access vulnerability as discussed in to increase the privileges available to the attacker to enable their desired goals on the target system.

Wireless IDS (WIDS)

An NIDS focused exclusively on wireless

Registry Access

An approach used on Windows systems is to monitor access to the registry, given the amount of information and access to it used by programs on these systems. - This source is very Windows specific, and has recorded limited success

Host Agent Module

An audit collection module operating as a background process on a monitored system - Purpose is to collect data on security-related events on the host and transmit these to the central manager

Statistical Classification

Analysis of the observed behavior using univariate, multivariate, or time-series models of observed metrics.

Analyzer (IDS)

Analyzers receive input from one or more sensors or from other analyzers. Also responsible for determining if an intrusion has occurred. The output of this component is an indication that an intrusion has occurred. - The output may include evidence supporting the conclusion that an intrusion occurred. - The analyzer may provide guidance about what actions to take as a result of the intrusion. - The sensor inputs may also be stored for future analysis and review in a storage or database component

Stateful protocol analysis (SPA)

Anomaly detection that compares observed network traffic against predetermined universal vendor supplied profiles of benign protocol traffic. - This distinguishes it from anomaly techniques trained with organization specific traffic profiles. - SPA understands and tracks network, transport, and application protocol states to ensure they progress as expected. - A key disadvantage of SPA is the high resource use it requires

Machine-Learning Classification

Approaches automatically determine a suitable classification model from the training data using data mining techniques.

Knowledge-Based Classification

Approaches use an expert system that classifies observed behavior according to a set of rules that model legitimate behavior

Cyber Criminals

Are either individuals or members of an organized crime group with a goal of financial reward. To achieve this, their activities may include identity theft, theft of financial credentials, corporate espionage, data theft, or data ransoming. - Typically, they are young, often Eastern European, Russian, or southeast Asian hackers, who do business on the Web

Activists

Are either individuals, usually working as insiders, or members of a larger group of outsider, who are motivated by social or political causes. - Also known as Hacktivists, and their skill level is often quite low. - The aim of their attacks is often to promote and publicize their cause, typically through website defacement, denial of service attacks, or the theft and distribution of data that results in negative publicity or compromise of their targets

State-Sponsored Organization

Are groups of hackers sponsored by governments to conduct espionage or sabotage activities. - They are also known as Advanced Persistent Threats (APTs), due to the covert nature and persistence over extended periods involved with many attacks in this class.

Others (Attacker Class)

Are hackers with motivations other than those listed above, including classic hackers or crackers who are motivated by technical challenge or by peer-group esteem and reputation. - Also, given the wide availability of attack toolkits, there is a pool of "hobby hackers" using them to explore system and network security, who could potentially become recruits for other organizations or classes

Primary Benefit of HIDS

Can detect both external and internal intrusions, something that is not possible either with network-based IDSs or firewalls

Distributed/Hybrid IDS

Combines information from a number of sensors, often both host and network-based, in a central analyzer that is able to better identify and respond to intrusion activity

Sequence Time-Delay Embedding (STIDE) Algorithm

Compares observed sequences of system calls with sequences from the training phase to obtain a mismatch ratio that determines whether the sequence is normal or not

Multivariate Model

Consider correlations between metrics for better levels of discrimination

Intrusion Detection System (IDS)

Designed to ____________________ - Can be reasonably effective against known, less sophisticated attacks, such as those by activist groups or large-scale email scams.

Markov Networks

Develop a model with sets of states, some possibly hidden, interconnected by transition probabilities

DDI Events

Distributed Detection and Inference (DDI) events are alerts that are generated when the gossip traffic enables a platform to conclude that an attack is under way

IDS Drawback

Efficiency and the cost of the detection process

Bayesian Networks

Encode probabilistic relationships among observed metrics

Summary Events

Events from various sources are collected by intermediate collection points such as firewalls, IDSs, or servers that serve a specific segment of the enterprise network. These events are summarized for delivery to the central policy system.

Clustering and Outlier Detection

Group the observed data into clusters based on some similarity or distance measure, and then identify subsequent data as either belonging to a cluster or as an outlier

Master

Hackers with high-level technical skills capable of discovering brand new categories of vulnerabilities, or writing new powerful attack toolkits. - Some of the better-known classical hackers are of this level, as clearly are some of those employed by some state-sponsored organizations, as the designation APT suggests. This makes defending against these attacks of the highest difficulty.

Apprentice

Hackers with minimal technical skill who primarily use existing attack toolkits. They likely comprise the largest number of attackers, including many criminal and activist attackers. Given their use of existing known tools, these attackers are the easiest to defend against. - Also known as "script-kiddies" due to their use of existing scripts (tools).

Journeyman

Hackers with sufficient technical skills to modify and extend attack toolkits to use newly discovered, or purchased, vulnerabilities; or to focus on different target groups. They may also be able to locate new vulnerabilities to exploit that are similar to some already known. - A number of hackers with such skills are likely found in all intruder classes, adapting tools for use by others. - The changes in attack tools make identifying and defending against such attacks harder.

NIDS Sensor Between External Firewall and Internet/WAN

In this position, the sensor can monitor all network traffic, unfiltered. The advantages of this approach are as follows: - Documents number of attacks originating on the Internet that target the network. - Documents types of attacks originating on the Internet that target the network.

Typical NIDS Facility

Includes - A number of sensors to monitor packet traffic - One or more servers for NIDS management functions - One or more management consoles for the human interface Analysis may be done at the sensors, the management server(s), or both

Inline Sensor

Inserted into a network segment so that the traffic that it is monitoring must pass through the sensor. - One way to achieve an inline sensor is to combine NIDS sensor logic with another network device, such as a firewall or a LAN switch - Advantages are that no additional hardware is needed and the sensors can block an attack upon detection

False Negatives

Intruders not identified as intruders

Anomaly Detection

Involves the collection of data relating to the behavior of legitimate users over a period of time. Then current observed behavior is analyzed against normal behavior to determine with a high level of confidence whether this behavior is that of a legitimate user or alternatively that of an intruder - Able to detect unknown, zero-day attacks, as this is usually anomalous

Rule-Based Heuristic Identification

Involves the use of rules for identifying known penetrations or penetrations that would exploit known weaknesses. Rules can also be defined that identify suspicious behavior, even when the behavior is within the bounds of established patterns of usage. - Typically, the rules used in these systems are specific to the machine and operating system and best created analyzing attack tools and scripts collected

Signature Approaches

Match a large collection of known patterns of malicious data against data stored on a system or in transit over a network. - The signatures need to be large enough to minimize the false alarm rate, while still detecting a sufficiently large fraction of malicious data

Passive Sensor

Monitors a copy of network traffic; the actual traffic does not pass through the device. - From the point of view of traffic flow, the sensor is more efficient than the inline sensor, because it does not add an extra handling step that contributes to packet delay - Connects to the transmission medium by a direct physical tap containing an NIC with no IP address -- A second NIC onboard does have an IP, and uses it to communicate findings to the network

Network-based IDS (NIDS)

Monitors network traffic for particular network segments or devices and analyzes network, transport, and application protocols to identify suspicious activity.

Host-based IDS (HIDS)

Monitors the characteristics of a single host and the events occurring within that host, such as process identifiers and the system calls they make, for evidence of suspicious activity

Network-Based IDS (NIDS)

Monitors traffic at selected points on a network or interconnected set of networks. - Examines the traffic packet by packet in real time, or close to real time, to attempt to detect intrusion patterns. - May examine network, transport, and/or application-level protocol activity - Typically included in the perimeter security infrastructure - Cannot read encrypted content, hindering its ability

Audit (Log file) Records

Most modern operating systems include accounting software that collects information on user activity. - The advantage of using this information is that no additional collection software is needed. - The disadvantages are that the audit records may not contain the needed information or may not contain it in a convenient way -- Attackers may try to manipulate it as well

NIDS Sensor for Individual LANs

Network facilities at a site may include separate LANs that support user workstations and servers specific to a single department. The administrator could configure a firewall and NIDS sensor to provide additional protection for all of these networks or target the protection to critical subsystems Advantages: - Detects attacks targeting critical systems and resources. - Allows focusing of limited resources to the network assets considered of greatest value.

Scanning Attack

Occurs when an attacker probes a target network or system by sending different kinds of packets. - Using the responses received from the target, the attacker can learn many of the system's characteristics and vulnerabilities - Thus, a scanning attack acts as a target identification tool for an attacker.

LAN Monitor Agent Module

Operates in the same fashion as a host agent module except that it analyzes LAN traffic and reports the results to the central manager

File Signatures

Patterns of data found in known malicious software

PEP Events

Policy Enforcement Points (PEPs) reside on trusted, self-defending platforms and intelligent IDSs. These systems correlate distributed information, local decisions, and individual device actions to detect intrusions that may not be evident at the host level

Machine-Learning Classification Pros/Cons

Pros - Automatically develops a model using labeled normal training data and applies it to data-mined data - Flexible and adaptable - Can capture interdependencies between observed metrics Cons - Process typically requires significant time and computational resources -- Afterward though, analysis is generally efficient - Dependent on assumptions about accepted behavior - High false-alarm rate

Signature Pros/Cons

Pros - Relatively low time and resource use - Widely accepted Cons - Significant effort required to constantly identify and review new malware to create signatures able to identify it - Inability to detect zero-day attacks

Statistical Classification Pros/Cons

Pros - Relatively simple with low computation costs - Lack assumptions about behavior expected Cons - Difficulty in selecting suitable metrics to obtain a reasonable balance between false positives and false negatives - Not all behaviors can be modeled using these approaches.

Knowledge-based Classification Pros/Cons

Pros - Robust - Flexible Cons - Difficulty and time required to develop high-quality knowledge from the data - Need for human experts to assist in gaining knowledge

Intrusion Detection Message Exchange Requirements (IDMEF)

RFC 4766

Central Manager Module

Receives reports from LAN monitor and host agents and processes and correlates these reports to detect intrusion

Sensors (IDS)

Responsible for collecting data. The input for a sensor may be any part of a system that could contain evidence of an intrusion. - Types of input to a sensor include network packets, log files, and system call traces. - Sensors collect and forward this information to the analyzer.

Heursitic Rules

Rules that characterize known malicious behavior

Neural Networks

Simulate human brain operation with neurons and synapse between them, that classify observed data

NIDS Sensor Guarding Major Backbone Networks

The benefits of this placement include the following: - Monitors a large amount of a network's traffic, thus increasing the possibility of spotting attacks. - Detects unauthorized activity by authorized users within the organization's security perimeter.

Initial Access

The initial access to a target system, typically by exploiting a remote network vulnerability by guessing weak authentication credentials used in a remote service or via the installation of malware on the system using some form of social engineering or drive-by-download attack

Detection Rate

The ratio of detected to total attacks

False Alarm Rate

The ratio of incorrectly classified events to total normal usage

Base Rate Fallacy

The tendency to ignore information about general principles in favor of very specific but vivid information.

User Interface (IDS)

The user interface to an IDS enables a user to view output from the system or control the behavior of the system. - In some systems, the user interface may equate to a manager, director, or console component

Autonomic Enterprise Security (Intel)

This approach does not rely solely on perimeter defense mechanisms, such as firewalls, or on individual host-based defenses. - Instead, each end host and each network device (e.g., routers) is considered to be a potential sensor and may have the sensor software module installed. - The sensors in this distributed configuration can exchange information to corroborate the state of the network (i.e., whether an attack is under way).

Monitoring Changes to Important Files

This uses a cryptographic checksum to check for any changes from the known good baseline for the monitored files. - Typically all program binaries, scripts, and configuration files are monitored, either on each access, or on a periodic scan of the file system - Cannot detect changes made to processes once they are running

Traditional Problems

Two key problems have always confronted systems such as IDSs, firewalls, virus and worm detectors, and so on: - These tools may not recognize new threats or radical modifications of existing threats. - It is difficult to update schemes rapidly enough to deal with quickly spreading attacks. A separate problem for perimeter defenses, such as firewalls, is that the modern enterprise has loosely defined boundaries, and hosts are generally able to move in and out.

User Trespass

Unauthorized logon to a machine or - if an authorized user - acquisition of privileges or performance of actions beyond those that have been authorized. - Software trespass can take the form of a virus, worm, or Trojan horse.

Signature/Heuristic Detection

Uses a set of known malicious data patterns or attack rules that are compared with current behavior. Also known as misuse detection. Can only identify known attacks for which it has patterns or rules.

Fuzzy Logic

Uses fuzzy set theory where reasoning is approximate, and can accommodate uncertainty

Genetic Algorithms

Uses techniques inspired by evolutionary biology, including inheritance, mutation, selection and recombination, to develop classification rules.

Alert Logging (NIDS)

When a sensor detects a potential violation, it sends an alert and logs information related to the event. - The NIDS analysis module can use this information to refine intrusion detection parameters and algorithms. - The security administrator can use this information to design prevention techniques

False Positives

Where authorized users are identified as intruders

Covering Tracks

Where the attacker disables or edits audit logs to remove evidence of attack activity, and uses rootkits and other measures to hide covertly installed files or code

Target Acquisition and Information Gathering

Where the attacker identifies and characterizes the target systems using publicly available information, both technical/non- technical and the use of network exploration tools to map target resources.