P2L4 - Intrusion Detection

Algorithms - A for alarm/positive, I for intrusion

- Detection (true positive rate): P(A|I) - False negative rate: P(~A|I) - False alarm rate: P(A|~I) - True negative rate: P(~A|~I) - Bayesian detection rate:P(I|A)

Workflow of IDS

Input of network/system data, goes through data preprocessor to extract activity records for analysis. Analyzed by detection engine which uses detection models that have already been constructed and stored. If intrusion via detection rule, IDS produces alert and uses decision engine to lookup decision table to send the response/report

Rule Based Detection

Involves use of rules for identifying known penetrations, identify suspicious behavior, typically are specific

SNORT Rule Option: Non-payload

Look for non-payload data, examine packet headers

Signature Approach Advantages

Low cost in time and resource use, wide acceptance

Signature Detection Approach Characteristics

Match large collection of known patterns of malicious data, signatures large enough to minimize false alarm rate, widely used in anti-virus products, network traffic scanning proxies and network intrusion detection systems (NIDS)

Statistical or Knowledge Approach: Any action that is not classified as normal is considered to be an attack

Mentions classifications, means its not normal per set of rules so its knowledge based approach

SNORT Rule Options

Meta-data, payload, non-payload, post-detection

Intrusion Detection Approaches - Modeling and Analysis

Misuse detection (signature based) - models all represent long intrusions. Anomaly detection - models all represents normal activities. Used to analyze data and detect intrusions

IDS at subnet/set of servers

Monitors large amount of networks traffic, increases possibility of finding attacks. More detailed analysis of traffic data. Detects unauthorized activity by authorized users.

Network Based IDS (NIDS)

Monitors traffic at selected points in real time, comprised of a number of sensors and 1+ servers for management and 1+ consoles for human interface, does analysis of traffic patterns. Can analyze traffic in multiple layers of the network as well.

Defense in depth principle

Multiple layers of defense mechanisms with detection mechanisms

Passive Sensors

Network IDS common deployment strategy. Monitors a copy of network traffic - actual traffic does not pass through. More efficient. No overhead added. Sensor has network interface card to connect to network so it can communicate w backhand management server

Honeypot characteristics

No production value, no legitimate reason to access it, any access is likely an attack, if honeypot initiates outbound traffic system is likely compromised

SNORT

Open source network intrusion detection / prevention tool. Open source, small memory and processor time.

IDS Passive Monitoring

Records and analyzes data about system and network activity. If IDS sends alert and response policy dictates intervention, activities are affected

Knowledge Based Approach Advantages

Robust and flexible, easy to update and improve them

IDS just inside the external firewall

Sees attacks originating from outside, can see if firewall misses attacks, sees attacks that might target web server, sees outgoing traffic from a potentially compromised server either from DMZ or internally

SNORT Rules

Simple and very flexible rule definition language. Each rule consists of rule header and number of options

SNORT Passive Mode

Simply copies and monitors traffic, traffic does not pass through. Configured for intrusion detection. Not an inline sensor

Machine Learning - Neural networks

Simulate human brain operation with neurons and synapse between them. One of most powerful approaches

Approaches for Anomaly Detection Model

Statistical - analysis of observed behavior using univariate, multivariate, or time series models of observed behaviors knowledge based - approaches use an expert system that classifies observed behavior according to rules that model legit behavior Machine learning - auto determine suitable classification model from training data using data mining techniques

Statistical or Knowledge Approach: Any action that does not fit the normal behavior profile is considered an attack

Statistical approach - it is used based off a behavior profile

Assumptions for intrusion detection

System activities are observable, along w network and user activities. Also, activities whether normal and intrusion, have distinct evidence.

Intruder Behavior

Target acquisition and information gathering -> initial access through vulnerability exploitation -> privilege escalation -> information gather or system exploit -> maintaining access -> cover tracks

SNORT Rule Actions

Tells SNORT what to do when it finds a packet that matches the rule. alert, log, pass, activate, dynamic, drop, reject, sdrop (drop reject and sdrop only when SNORT is inline)

Compiler Backdoors

This backdoor inserts backdoors into other programs during compilation

Limitations of Anomaly Detection

Trained on legitimate data, limits the effectiveness of some techniques. High false alarm rate that might just be new or observed normal activity

T/F: Intruder can also be referred to as a hacker or cracker

True

T/F: Intruders typically use steps from a common attack methodology

True

T/F: To defeat an IDS, attackers can embed attack in packets that cause non-uniform processing by different OS (bad checksum, overlapping fragments)

True, IDS sees different traffic than end host, which may be attacked but not detected

Signature T/F: can only detect an intrusion attempt if it matches a pattern that is in the database

True, aka the definition of signature based detection system

Anomaly T/F: If malicious activity looks like normal traffic to the system, it will not detect an attack

True, anomaly detection looks at what does't look normal but this is

Anomaly T/F: Longer system in use, more it learns about network activity

True, anomaly detection needs to learn and profile what is normal, longer period makes it better

T/F: The primary purpose of an IDS is to detect intrusions, log suspicious events, and send alerts

True, basic foundations of an IDS

T/F: reduce false alarm rate while detecting as many intrusions as possible improves detection performance

True, both are ideal, reduce burden on sys admins

T/F: To defeat an IDS, attackers can send a packet that would trigger a buffer overflow in the IDS code

True, buffer overflow exploit can be used to attack a program, attacker can buffer overflow an IDS so that the attacker controls it

T/F: To defeat an IDS, attackers can send a huge amount of traffic

True, can cause DDOS and cant analyze other incoming attacks

T/F: Network based intrusion detection makes use of signature detection and anomaly detection

True, can use both approaches

Anomaly T/F: False positives can become a problem, normal usage can be mistaken for an attack

True, definition of false positive is that normal activity is mistaken as an attack. Can cause a waste of time in investigation

T/F: Applying detection models at processed event data that has higher base rate improves detection performance

True, if we can keep intrusion evidence in event data, event date is going to have a higher base rate and therefore ideas will have a higher bayesian detection rate

T/F: To defeat an IDS, attackers can send traffic that purposely matches detection rules

True, lots of alerts to be analyzed, attacker can then send real attack

T/F: A network IDS sensor monitors a copy of network traffic, the actual traffic does not pass through the device

True, network IDS typically performs passive monitoring by copying network traffic

Signature T/F: when a new virus is identified, it must be added to signature DB

True, otherwise it won't be able to detect it

T/F: Intrusion detection is based on the assumption that the behavior of the intruder differs from that of a legitimate user in ways that can be quantified

True, this is primary assumption of IDS

T/F: Running a packet sniffer on a workstation to capture usernames and passwords is an example of intrusion

True, unless packet sniffing is done w proper authorization

T/F: common location for a NIDS sensor is just inside the external firewall

True, very typical of NIDS

Inline Sensors

Type of Network IDS configuration to block an attack when detected. Inserted into network segment so that the traffic it monitors must pass through the sensor

Machine Learning Approach Characteristics

Use data mining techniques to develop a model that can classify data as normal or anomaly. Uses examples of normal data and outputs model to classify normal data.

Misuse/Signature Detection Elements

Uses set of known malicious data patterns or attack rules that are compared with current behavior, can only identify known attacks for which it has patterns or rules

honeypot deployment - internal network alongside workstations and servers

advantages - can detect misconfigured firewall, catch internal attacks disadvantages - compromised honeypot can attack other systems, firewall must adjust to allow traffic to honeypot

honeypot deployment - outside external firewall

advantages - no side effect or increased risk, reduces amount of traffic to firewall disadvantages - cannot trap internal attacks

Intrusion

attack that aims to compromise the security of an organization

Components of IDS - Architecture

audit data processor, knowledge base, decision engine, alarm generation and responses

Honeypots

decoy systems designed to lure potential attackers away from critical systems

IDS at workstations

detect attacks target critical systems and resources, allows focuses of limited resources to the network assets of greatest value

honeypot deployment - DMZ

disadvantages - DMZ not fully accessible, will not trap interesting attacks

Low interaction honeypot

emulates particular IT services or systems to provide realistic initial environment, not a full version, less realistic target, sufficient for use as a component of a distributed IDS to warn of imminent attack

Defense in depth strategies include:

encryption, detailed audit trails, strong authentication and authorization controls, active management of OS, application security

Bayesian detection rate

given IDS produces an alert, how likely is it that an intrusion actually occurs.

Evaluating IDS: Detection Rate (True Positive Rate)

given there is an intrusion how likely would the IDS correctly output an alert

Evaluating IDS: Force negative rate

how many intrusions are missed when there is an alert

SNORT Rule Option: Payload

look for data inside of the packet

Problem with NIDS

may produce larger number of alerts to be examined by sys admin. Need to prioritize w security levels

How Misuse/signature detection detects intrusion

observing events in the system, applying set of patterns or rules to the data, determining if it is intrusive or normal

SNORT Packet Decoder

possesses each capture packet to identify the protocol headers as its data link network transport and application layers

SNORT Rule Option: meta-data

provides information about the rule, but does not have any effect during detection

high interaction honeypot

real system, full OS, realistic target, requires way more resources, can learn more about attack and attackers

intrusion examples

remote root, web server defacement, guessing/crack pw, copying DBs, viewing sensitive data, impersonation to get info, unattended workstation, packet sniffers, unsecured modem access network, distributing pirated software

SNORT Rule Option: Post-detection

rule-specific triggers that happen after a rule has matched a packet.

DDoS Attack on Network IDS

send lots of traffic to IDS, exhaust its resources. Can abuse reactive nature of IDS. Can simply trigger lots of alerts and send in the attack after a lot of other alerts of the fake attacks

Signature Approach Disadvantages

significant effort to identify and review new malware to create signatures, inability to detect zero-day attacks

Statistical Approach Advantages

simplicity, low computation cost, lack of assumptions about expected behavior

How to mitigate weakness while system is learning what normal behavior is in anomaly detection?

use a firewall

Statistical Approach Characteristics

use captured sensor data, multivariate models using time of and order of the event, uses correlations

Who can write rules for SNORT

users of SNORT, SNORT community, security experts, teams, etc

Intrusion Prevention System (IPS)

A technology that monitors activity like an IDS but will automatically take proactive preventative action if it detects unacceptable activity. IPS can be much more sophisticated w detection algorithms than firewall

SNORT Detection Engine

Actual work of intrusion detection, checks each packet against set of rules. Match triggers alert specified by the rule. Discarded if no matches.

Which could be considered an anomaly to a typical network traffic: -IP address -Port address -Packet length -Flag setting

All of them. If IP address not normally accessed it is an anomaly. If port not usually access, anomaly. If packet is way too long or flags not seen normally, it is an anomaly

Honeypots are designed to do what?

Attract attackers, divert an attack, collect information about attackers, and encourage an attacker to stay long enough for administrators to respond

Good/Bad Sensor Practice: Use a shared network resource to gather NIDS data

Bad, attacker can disable the IDS or modify the alerts that sent it

Good/Bad Sensor Practice: Set IDS level to the highest sensitivity to detect every attack

Bad, large number of false alarms

Intrusion Detection Approaches - Deployment

Can deploy IDS on end host or network perimeter

Asymmetric Backdoors

Can only be used by the person who created it, even if it discovered by others, usually protected or controlled by codographic schemes

Intrusion Detection Approaches - Development and maintenance

Can use manual encoding of expert knowledge

Anomaly Detection Elements

Collection of data relating to behavior of legit users over period of time and checking current observed behavior to see if its legit user or that of intruder

How to achieve inline sensors

Combining NIDS sensor logic w. firewall or LAN switch - no additional hardware needed. Can also use stand alone inline NIDS sensor

4 Logical Components of SNORT

Decoder, Detection Engine, Log, and Alert

Machine Learning Approach Disadvantages

Dependency on assumptions about accepted behavior, high false alarm rate, high resource cost, significant time and computational resources for training.

Network IDS

Deployed at perimeter of network, or subnet, to monitor traffic going in and out of the network. Uses packet capturing tool (libpcap) to get network traffic data.

Bayesian Detection Rate Implications to IDS

Design algorithms to reduce false alarm rate, deploy IDS to appropriate point/layer w sufficiently high base rate, multiple independent detection models

Machine Learning - Markov Models

Develop a model w sets of states that are connected by transitional probabilities. Example: detecting real words in websites versus random names which are used by botnets for command and control

Knowledge Based Approach Characteristics

Developed during training to characterize data into distinct classes, relies on experts to develop set of rules.

Knowledge Based Approach Disadvantages

Difficulty and time required to develop knowledge from data, human experts must assist w the process

Statistical Approach Disadvantages

Difficulty selecting suitable metrics, not all behaviors can be modeled using these approaches.

IDS b.w external network an internet

Documents number and types of attacks originating on the internet that target the network. Sees all attempted attacks

SNORT characteristics

Easily configured by sys admins and deployed on most nodes, efficient operation, performs realtime packet capture, detects a variety of attacks and probes

IDS effectivity

Effective against known, less sophisticated attacks. Not effective against new, zero-day exploits

Machine Learning - Bayesian Networks

Encode probabilistic relationships among observed metrics. Example: how likely is event, if its low probability, it could be anomaly

What do honeypots contain?

Fabricated information, monitors and event loggers that are triggered with access, makes attacks seem successful

T/F: Activists are either individuals or members of an organized crime group w a goal of financial reward

False, activists typically have political or social cause

T/F: There is no benefit of deploying a NIDS or honeypot outside of the external firewall

False, allow us to see what attacks are coming from the internet to the enterprise network. For honeypot, attacks are trapped so it reduces traffic that firewall has to process (aka less alerts)

T/F: applying detection models at all unfiltered packet data directly improves detection performance

False, base rate at this level is very low. IDS will have low bayesian detection rate

Signature T/F: New threats can be detected immediately

False, can only detect known rules or intrusions

T/F: Those who hack into computers do so for the thrill of it or for status

False, many for financial gains

T/F: honeypot can be a workstation that a user uses for work

False, not a real system for use by real users

T/F: Signature based approaches attempt to define normal, or expected behavior, whereas anomaly approaches attempt to define proper behavior

False, signature based approach is used to represent known intrusion patterns, not normal behavior

Components of IDS - Algorithmic

Features - capture intrusion evidences, models - piece evidences together

Firewall or IDS: Tries to stop intrusion from happening

Firewall

Firewall or IDS:Limits access b.w networks to prevent intrusion

Firewall

Firewall V Network IDS

Firewall - Active filtering, fail-close Network IDS - Passive monitoring, fail-open

Architecture of Network IDS

First apply filters to the packet. Second, event engines to analyze them into security related events, failed logins, etc. Finally detection models to the security related event data. Volume of data therefore decreased, can get higher bayesian detection rate

Machine Learning Approach Advantages

Flexible, handle small changes, adaptable, ability to capture inter-dependencies or deeper connections b.w observed metrics

Evaluating IDS: False alarm rate or false positive rate

Given no intrusion, how likely is IDS to falsely output an alert

Good/Bad Sensor Practice: NIDS sensors are not turnkey solutions, system admins must interpret alerts

Good, NIDS can produce false positives, so sys admins must interpret the alerts and take appropriate actions

Good/Bad Sensor Practice: Monitor both outbound and inbound traffic

Good, traffic in both directions ought to be monitored

Machine Learning - Clustering and outlier detection

Group observed data into clusters based on similarity or distance measure, identify subsequently observed data as belonging to a cluster or an outlier

Object Code Backdoors

Hard to detect, modifies machine code

Firewall or IDS: Tries to evaluate an intrusion after it has happened

IDS

Firewall or IDS: Watches for instructions that start within the system

IDS

Host IDS

IDS deployed in an end host. Traces system calls

Eluding Network IDS

IDS on Unix, end host on windows, may not process packets same way. attacker can exploit evidences in fragments and checksums and hope IDS doesn't catch it and the host gets it