P2L4 - Intrusion Detection
Algorithms - A for alarm/positive, I for intrusion
- Detection (true positive rate): P(A|I) - False negative rate: P(~A|I) - False alarm rate: P(A|~I) - True negative rate: P(~A|~I) - Bayesian detection rate:P(I|A)
Workflow of IDS
Input of network/system data, goes through data preprocessor to extract activity records for analysis. Analyzed by detection engine which uses detection models that have already been constructed and stored. If intrusion via detection rule, IDS produces alert and uses decision engine to lookup decision table to send the response/report
Rule Based Detection
Involves use of rules for identifying known penetrations, identify suspicious behavior, typically are specific
SNORT Rule Option: Non-payload
Look for non-payload data, examine packet headers
Signature Approach Advantages
Low cost in time and resource use, wide acceptance
Signature Detection Approach Characteristics
Match large collection of known patterns of malicious data, signatures large enough to minimize false alarm rate, widely used in anti-virus products, network traffic scanning proxies and network intrusion detection systems (NIDS)
Statistical or Knowledge Approach: Any action that is not classified as normal is considered to be an attack
Mentions classifications, means its not normal per set of rules so its knowledge based approach
SNORT Rule Options
Meta-data, payload, non-payload, post-detection
Intrusion Detection Approaches - Modeling and Analysis
Misuse detection (signature based) - models all represent long intrusions. Anomaly detection - models all represents normal activities. Used to analyze data and detect intrusions
IDS at subnet/set of servers
Monitors large amount of networks traffic, increases possibility of finding attacks. More detailed analysis of traffic data. Detects unauthorized activity by authorized users.
Network Based IDS (NIDS)
Monitors traffic at selected points in real time, comprised of a number of sensors and 1+ servers for management and 1+ consoles for human interface, does analysis of traffic patterns. Can analyze traffic in multiple layers of the network as well.
Defense in depth principle
Multiple layers of defense mechanisms with detection mechanisms
Passive Sensors
Network IDS common deployment strategy. Monitors a copy of network traffic - actual traffic does not pass through. More efficient. No overhead added. Sensor has network interface card to connect to network so it can communicate w backhand management server
Honeypot characteristics
No production value, no legitimate reason to access it, any access is likely an attack, if honeypot initiates outbound traffic system is likely compromised
SNORT
Open source network intrusion detection / prevention tool. Open source, small memory and processor time.
IDS Passive Monitoring
Records and analyzes data about system and network activity. If IDS sends alert and response policy dictates intervention, activities are affected
Knowledge Based Approach Advantages
Robust and flexible, easy to update and improve them
IDS just inside the external firewall
Sees attacks originating from outside, can see if firewall misses attacks, sees attacks that might target web server, sees outgoing traffic from a potentially compromised server either from DMZ or internally
SNORT Rules
Simple and very flexible rule definition language. Each rule consists of rule header and number of options
SNORT Passive Mode
Simply copies and monitors traffic, traffic does not pass through. Configured for intrusion detection. Not an inline sensor
Machine Learning - Neural networks
Simulate human brain operation with neurons and synapse between them. One of most powerful approaches
Approaches for Anomaly Detection Model
Statistical - analysis of observed behavior using univariate, multivariate, or time series models of observed behaviors knowledge based - approaches use an expert system that classifies observed behavior according to rules that model legit behavior Machine learning - auto determine suitable classification model from training data using data mining techniques
Statistical or Knowledge Approach: Any action that does not fit the normal behavior profile is considered an attack
Statistical approach - it is used based off a behavior profile
Assumptions for intrusion detection
System activities are observable, along w network and user activities. Also, activities whether normal and intrusion, have distinct evidence.
Intruder Behavior
Target acquisition and information gathering -> initial access through vulnerability exploitation -> privilege escalation -> information gather or system exploit -> maintaining access -> cover tracks
SNORT Rule Actions
Tells SNORT what to do when it finds a packet that matches the rule. alert, log, pass, activate, dynamic, drop, reject, sdrop (drop reject and sdrop only when SNORT is inline)
Compiler Backdoors
This backdoor inserts backdoors into other programs during compilation
Limitations of Anomaly Detection
Trained on legitimate data, limits the effectiveness of some techniques. High false alarm rate that might just be new or observed normal activity
T/F: Intruder can also be referred to as a hacker or cracker
True
T/F: Intruders typically use steps from a common attack methodology
True
T/F: To defeat an IDS, attackers can embed attack in packets that cause non-uniform processing by different OS (bad checksum, overlapping fragments)
True, IDS sees different traffic than end host, which may be attacked but not detected
Signature T/F: can only detect an intrusion attempt if it matches a pattern that is in the database
True, aka the definition of signature based detection system
Anomaly T/F: If malicious activity looks like normal traffic to the system, it will not detect an attack
True, anomaly detection looks at what does't look normal but this is
Anomaly T/F: Longer system in use, more it learns about network activity
True, anomaly detection needs to learn and profile what is normal, longer period makes it better
T/F: The primary purpose of an IDS is to detect intrusions, log suspicious events, and send alerts
True, basic foundations of an IDS
T/F: reduce false alarm rate while detecting as many intrusions as possible improves detection performance
True, both are ideal, reduce burden on sys admins
T/F: To defeat an IDS, attackers can send a packet that would trigger a buffer overflow in the IDS code
True, buffer overflow exploit can be used to attack a program, attacker can buffer overflow an IDS so that the attacker controls it
T/F: To defeat an IDS, attackers can send a huge amount of traffic
True, can cause DDOS and cant analyze other incoming attacks
T/F: Network based intrusion detection makes use of signature detection and anomaly detection
True, can use both approaches
Anomaly T/F: False positives can become a problem, normal usage can be mistaken for an attack
True, definition of false positive is that normal activity is mistaken as an attack. Can cause a waste of time in investigation
T/F: Applying detection models at processed event data that has higher base rate improves detection performance
True, if we can keep intrusion evidence in event data, event date is going to have a higher base rate and therefore ideas will have a higher bayesian detection rate
T/F: To defeat an IDS, attackers can send traffic that purposely matches detection rules
True, lots of alerts to be analyzed, attacker can then send real attack
T/F: A network IDS sensor monitors a copy of network traffic, the actual traffic does not pass through the device
True, network IDS typically performs passive monitoring by copying network traffic
Signature T/F: when a new virus is identified, it must be added to signature DB
True, otherwise it won't be able to detect it
T/F: Intrusion detection is based on the assumption that the behavior of the intruder differs from that of a legitimate user in ways that can be quantified
True, this is primary assumption of IDS
T/F: Running a packet sniffer on a workstation to capture usernames and passwords is an example of intrusion
True, unless packet sniffing is done w proper authorization
T/F: common location for a NIDS sensor is just inside the external firewall
True, very typical of NIDS
Inline Sensors
Type of Network IDS configuration to block an attack when detected. Inserted into network segment so that the traffic it monitors must pass through the sensor
Machine Learning Approach Characteristics
Use data mining techniques to develop a model that can classify data as normal or anomaly. Uses examples of normal data and outputs model to classify normal data.
Misuse/Signature Detection Elements
Uses set of known malicious data patterns or attack rules that are compared with current behavior, can only identify known attacks for which it has patterns or rules
honeypot deployment - internal network alongside workstations and servers
advantages - can detect misconfigured firewall, catch internal attacks disadvantages - compromised honeypot can attack other systems, firewall must adjust to allow traffic to honeypot
honeypot deployment - outside external firewall
advantages - no side effect or increased risk, reduces amount of traffic to firewall disadvantages - cannot trap internal attacks
Intrusion
attack that aims to compromise the security of an organization
Components of IDS - Architecture
audit data processor, knowledge base, decision engine, alarm generation and responses
Honeypots
decoy systems designed to lure potential attackers away from critical systems
IDS at workstations
detect attacks target critical systems and resources, allows focuses of limited resources to the network assets of greatest value
honeypot deployment - DMZ
disadvantages - DMZ not fully accessible, will not trap interesting attacks
Low interaction honeypot
emulates particular IT services or systems to provide realistic initial environment, not a full version, less realistic target, sufficient for use as a component of a distributed IDS to warn of imminent attack
Defense in depth strategies include:
encryption, detailed audit trails, strong authentication and authorization controls, active management of OS, application security
Bayesian detection rate
given IDS produces an alert, how likely is it that an intrusion actually occurs.
Evaluating IDS: Detection Rate (True Positive Rate)
given there is an intrusion how likely would the IDS correctly output an alert
Evaluating IDS: Force negative rate
how many intrusions are missed when there is an alert
SNORT Rule Option: Payload
look for data inside of the packet
Problem with NIDS
may produce larger number of alerts to be examined by sys admin. Need to prioritize w security levels
How Misuse/signature detection detects intrusion
observing events in the system, applying set of patterns or rules to the data, determining if it is intrusive or normal
SNORT Packet Decoder
possesses each capture packet to identify the protocol headers as its data link network transport and application layers
SNORT Rule Option: meta-data
provides information about the rule, but does not have any effect during detection
high interaction honeypot
real system, full OS, realistic target, requires way more resources, can learn more about attack and attackers
intrusion examples
remote root, web server defacement, guessing/crack pw, copying DBs, viewing sensitive data, impersonation to get info, unattended workstation, packet sniffers, unsecured modem access network, distributing pirated software
SNORT Rule Option: Post-detection
rule-specific triggers that happen after a rule has matched a packet.
DDoS Attack on Network IDS
send lots of traffic to IDS, exhaust its resources. Can abuse reactive nature of IDS. Can simply trigger lots of alerts and send in the attack after a lot of other alerts of the fake attacks
Signature Approach Disadvantages
significant effort to identify and review new malware to create signatures, inability to detect zero-day attacks
Statistical Approach Advantages
simplicity, low computation cost, lack of assumptions about expected behavior
How to mitigate weakness while system is learning what normal behavior is in anomaly detection?
use a firewall
Statistical Approach Characteristics
use captured sensor data, multivariate models using time of and order of the event, uses correlations
Who can write rules for SNORT
users of SNORT, SNORT community, security experts, teams, etc
Intrusion Prevention System (IPS)
A technology that monitors activity like an IDS but will automatically take proactive preventative action if it detects unacceptable activity. IPS can be much more sophisticated w detection algorithms than firewall
SNORT Detection Engine
Actual work of intrusion detection, checks each packet against set of rules. Match triggers alert specified by the rule. Discarded if no matches.
Which could be considered an anomaly to a typical network traffic: -IP address -Port address -Packet length -Flag setting
All of them. If IP address not normally accessed it is an anomaly. If port not usually access, anomaly. If packet is way too long or flags not seen normally, it is an anomaly
Honeypots are designed to do what?
Attract attackers, divert an attack, collect information about attackers, and encourage an attacker to stay long enough for administrators to respond
Good/Bad Sensor Practice: Use a shared network resource to gather NIDS data
Bad, attacker can disable the IDS or modify the alerts that sent it
Good/Bad Sensor Practice: Set IDS level to the highest sensitivity to detect every attack
Bad, large number of false alarms
Intrusion Detection Approaches - Deployment
Can deploy IDS on end host or network perimeter
Asymmetric Backdoors
Can only be used by the person who created it, even if it discovered by others, usually protected or controlled by codographic schemes
Intrusion Detection Approaches - Development and maintenance
Can use manual encoding of expert knowledge
Anomaly Detection Elements
Collection of data relating to behavior of legit users over period of time and checking current observed behavior to see if its legit user or that of intruder
How to achieve inline sensors
Combining NIDS sensor logic w. firewall or LAN switch - no additional hardware needed. Can also use stand alone inline NIDS sensor
4 Logical Components of SNORT
Decoder, Detection Engine, Log, and Alert
Machine Learning Approach Disadvantages
Dependency on assumptions about accepted behavior, high false alarm rate, high resource cost, significant time and computational resources for training.
Network IDS
Deployed at perimeter of network, or subnet, to monitor traffic going in and out of the network. Uses packet capturing tool (libpcap) to get network traffic data.
Bayesian Detection Rate Implications to IDS
Design algorithms to reduce false alarm rate, deploy IDS to appropriate point/layer w sufficiently high base rate, multiple independent detection models
Machine Learning - Markov Models
Develop a model w sets of states that are connected by transitional probabilities. Example: detecting real words in websites versus random names which are used by botnets for command and control
Knowledge Based Approach Characteristics
Developed during training to characterize data into distinct classes, relies on experts to develop set of rules.
Knowledge Based Approach Disadvantages
Difficulty and time required to develop knowledge from data, human experts must assist w the process
Statistical Approach Disadvantages
Difficulty selecting suitable metrics, not all behaviors can be modeled using these approaches.
IDS b.w external network an internet
Documents number and types of attacks originating on the internet that target the network. Sees all attempted attacks
SNORT characteristics
Easily configured by sys admins and deployed on most nodes, efficient operation, performs realtime packet capture, detects a variety of attacks and probes
IDS effectivity
Effective against known, less sophisticated attacks. Not effective against new, zero-day exploits
Machine Learning - Bayesian Networks
Encode probabilistic relationships among observed metrics. Example: how likely is event, if its low probability, it could be anomaly
What do honeypots contain?
Fabricated information, monitors and event loggers that are triggered with access, makes attacks seem successful
T/F: Activists are either individuals or members of an organized crime group w a goal of financial reward
False, activists typically have political or social cause
T/F: There is no benefit of deploying a NIDS or honeypot outside of the external firewall
False, allow us to see what attacks are coming from the internet to the enterprise network. For honeypot, attacks are trapped so it reduces traffic that firewall has to process (aka less alerts)
T/F: applying detection models at all unfiltered packet data directly improves detection performance
False, base rate at this level is very low. IDS will have low bayesian detection rate
Signature T/F: New threats can be detected immediately
False, can only detect known rules or intrusions
T/F: Those who hack into computers do so for the thrill of it or for status
False, many for financial gains
T/F: honeypot can be a workstation that a user uses for work
False, not a real system for use by real users
T/F: Signature based approaches attempt to define normal, or expected behavior, whereas anomaly approaches attempt to define proper behavior
False, signature based approach is used to represent known intrusion patterns, not normal behavior
Components of IDS - Algorithmic
Features - capture intrusion evidences, models - piece evidences together
Firewall or IDS: Tries to stop intrusion from happening
Firewall
Firewall or IDS:Limits access b.w networks to prevent intrusion
Firewall
Firewall V Network IDS
Firewall - Active filtering, fail-close Network IDS - Passive monitoring, fail-open
Architecture of Network IDS
First apply filters to the packet. Second, event engines to analyze them into security related events, failed logins, etc. Finally detection models to the security related event data. Volume of data therefore decreased, can get higher bayesian detection rate
Machine Learning Approach Advantages
Flexible, handle small changes, adaptable, ability to capture inter-dependencies or deeper connections b.w observed metrics
Evaluating IDS: False alarm rate or false positive rate
Given no intrusion, how likely is IDS to falsely output an alert
Good/Bad Sensor Practice: NIDS sensors are not turnkey solutions, system admins must interpret alerts
Good, NIDS can produce false positives, so sys admins must interpret the alerts and take appropriate actions
Good/Bad Sensor Practice: Monitor both outbound and inbound traffic
Good, traffic in both directions ought to be monitored
Machine Learning - Clustering and outlier detection
Group observed data into clusters based on similarity or distance measure, identify subsequently observed data as belonging to a cluster or an outlier
Object Code Backdoors
Hard to detect, modifies machine code
Firewall or IDS: Tries to evaluate an intrusion after it has happened
IDS
Firewall or IDS: Watches for instructions that start within the system
IDS
Host IDS
IDS deployed in an end host. Traces system calls
Eluding Network IDS
IDS on Unix, end host on windows, may not process packets same way. attacker can exploit evidences in fragments and checksums and hope IDS doesn't catch it and the host gets it