Data Mining Exam 3
Apriori Algorithm
-Initially, every item is considered as a candidate 1-itemset (let k=1) -Their supports are counted; anything below minsup is discarded -Candidate (k+1)-itemsets are generated from the frequent k-itemsets -Their supports are counted; anything below minsup is discarded -Repeat until no additional frequent itemsets are found Then, use Apriorti rule generation algorithm to come up with tules
Candidate Generation: Fk-1 x Fk-1 Method (Apriori-gen)
-Items in each frequent itemset must be sorted (in lexographical order) -Merges pairs of (k-1)-itemsets if all but their last item are the same
Candidate Generation: Fk-1 x F1 Method
-Items in each frequent itemset must be sorted (in lexographical order) -To generrate k-itemsets, extend each frequent (k-1)-itemset with a frequent 1-itemset that is lexographically larger than the items already in the (k-1)-itemset Benefit: avoids generating duplicates
Fk-1 x F1 Method: Pruning
-Need to check generated candidates for infrequent itemsets, and prune them -Can use a heuristic: For every frequent k-itemset, every item in the set must be contained in at least k-1 of the frequent (k-1)-itemsets Ex: {Beer, Diapers, Bread} is only a frequent itemset if Beer, Diapers, and Bread all show up in 2 of the frequent 2-itemsets. IF Beer is only in 1 of the frequent 2-itemsets, Beer will not be in a frequent 3-itemset.
Fk-1 x Fk-1 Method (Apriori-gen): Pruning
-Need to check generated candidates for infrequent itemsets, and prune them -If any (k-1)-subset of the candidate is not a frequent (k-1)-itemset, the candidate is pruned Ex: {Bread, Diapers, Milk} is only a frequent 3-itemset if {Bread, Diapers}, {Bread, Milk}, and {Diapers, Milk} are all frequent 2-itemsets.
Association Rule Mining Steps
1. Frequent itemset generation: find all the itemsets that satisfy minsup 2. Rule generation: find all the strong rules in the frequent itemsets that satisfy minconf
For GSP, how do you generate candidate k+1-sequences from frequent k-sequences?
1. Generate -1 and -last items and then find any -1 items that match -last items 2. Merge them to get the new item 3. Do example?!
FP-Growth algorithm
1. Make an initial scan of the data to get support counts for each individual item. Discard infrequent items, Sort frequent items in decreasing order 2. Make a second scan of the data to construct the FP-tree. Each transaction is added to the tree with its frequent items sorted by decreasing support 3. Find the frequent itemsets from the FP-tree
What are some ways of storing support count for 2-itemsets in memory?
2D array: arr[i][j] = count Hashmap: "triples method" {i,j} --> count 1D triangular array: (i-1)(n-i/2) + j - i
closed frequent itemsets
A frequent itemset is closed if it has no superset with the same support count
negative association rule
A rule extracted from a negative itemset that meets minsup and minconf
What is the difference between a sequence and an itemset?
A sequence contains ordered elements/events/itemsets. Elements/events/itemsets contain unordered items.
What is a k-sequence?
A sequence is a k-sequence f it contains k items aka k = |A1| + |A2| + · · · + |An|.
What does it mean for a transaction to contain itemset X?
A transaction contains itemset X if X is a subset of the transaction Ex: t2 contains {Bread, Diapers} but not {Bread, Milk}
Association Rule
An implication expression of the form X --> Y, where X and Y are itemsets Ex: {Milk, Diaper} --> {Beer} do NOT mean causality... more like co-occurrence
negatively correlated pattern
An itemset (or association rule) where s(X U Y) < s(X)s(Y)
For GSP, how do you generate candidate 2-sequences from frequent 1-sequences?
Consider both cases of transactions: 1. Bought separately (i.e. AB) Have to consider entire matrix because with separate transactions, order of the transactions matters 2. And bought together (ie. (AB)) Consider only half of matrix because within a single transaction, order doesn't matter
Mining Streams
Data is coming in as you mine, so frequent itemsets are changing quickly so you need to keep up! Basically chunking Accumulate a certain amount (i.e. number of transactions) --> mine those Let the more transactions coming in build up and ignore them Check the first batch's frequent itemset and compare with second batch's and see if the itemsets are still frequent (or remove after certain amount of iterations) and add any new itemsets Constantly build up chunks, mine them, and update your frequent item sets
What are some advantages and disadvantages to using a hashmap for storing support count?
Disadvantages: hashing is expensive, store 3 pairs of numbers per count rather than 1 Advantages: Doesn't waste space on support counts of 0; doesn't get added to hashmap
Goal of association rule
Find all rules having support ≥ minsup threshold confidence ≥ minconf threshold Bonus question: why is looking at support not enough? (21.6)
Support
Fraction of transactions that contain both X and Y s(X --> Y) = (𝜎(𝑋 ∪ 𝑌))/𝑁 where 𝑁 is total number of transactions
apriori principle
If an itemset is frequent, then all of its subsets must also be frequent If an itemset is infrequent, then all of its supersets must also be infrequent
Given s = < {7} {3 8} {9} {4 5 6} {8} >, why is r2 = < {3} {4} {5} {8} > NOT a subsequence of s?
If two elements are grouped initially, the have to be grouped in the subsequence as well if they're included
What is the difference subsequence vs. substring?
In a subsequence, you can have gaps between the elements within the original sequence as long as the order is maintained Substring is basically a continuous subsequence Ex: s= ACTGAACG r1 = CGAAG is a subsequence of s r2 = CTGA is a substring of s
Apriori Rule Generation Algorithm
Initially, all high-confidence rules that have only one item in the consequent are generated and tested against minconf The high-confidence rules that are found are then used to generate the next round of candidate rules by merging consequents Bonus question: How does this relate to the anti-monotone property of confidence? Ex: ABD --> C ACD --> B Consequents are merged to make: AD --> BC
closed frequent itemset algorithm
Keep a list of closed frequent itemsets Each time a frequent itemset is generated, perform subset and superset check
maximal frequent itemset algorithm
Keep a list of maximal frequent itemsets Each time a frequent itemset is generated, perform subset and superset check
Which time series constraints can cause the apriori principle to be violated?
Maxgap
What is the difference of performing a subset check for maximal sets vs. closed sets?
Maximal: Is the freq itemset just found a subset of anything in the maximal list? If so, it is not maximal; end. Else, add it to the maximal list and do a superset check. Closed: Is the freq itemset just found a subset of anything in the maximal list? If so, is its support higher than the superset in the list? If no, it is not closed; end. If yes, add it to the closed list and do a superset check.
What is the difference of performing a superset check for maximal sets vs. closed sets?
Maximal: Is the freq itemset just found a superset of anything in the maximal list? If so, remove items already in the maximal list that are subsets of this freq itemset, as they are no longer maximal. Closed: Is the freq itemset just found a superset of anything in the closed list? If so, does the subset in the list have the same or higher support? If subset's support is the same, remove the subset from the closed list. If subset's support is higher, it remains in the list.
Confidence**
Measures how often items in Y appear in transactions that contain X c(X --> Y) = (𝜎(𝑋 ∪𝑌))/(𝜎(𝑋)) and the higher the score, the more it validates that X --> Y
Advantages of FP-Growth?
Only scan the database twice Avoids large memory requirements, repeated scans of the database
What is the formula for lift (x --> y)?
P(X, Y)/(P(X)P(Y)) = (Nf11)/(f11 + f01)(f11+f10)
four ways of dealing with scaling. explain each
PCY Algorithm, sampling, SON Algorithm, mining streams
What does it mean for a rule to be more general that another rule given that they have the same consequent?
The first rule's LHS is a subset of the second rule's LHS.
What does it mean for a rule to be redundant with another rule given that they have the same consequent?
The second rule is more general and it has the same support
What does it mean for a rule to be more specific that another rule given that they have the same consequent?
The second rule's LHS is a subset of the first rule's LHS.
Sampling
Use a sample dataset that will work with the available memory size Must be truly random Can produce false negatives (frequent in total but not in sample) Can produce false positives (frequent in sample but not in sample) But we can eliminate false positives (cross check with total data) and mitigate false negatives (reduce minsup)
prefix search tree
When order didn't matter, it turned into a lattice, but now order does matter (i.e. AG is not the same as GA) Levels = k Can still prune using support counts
What does a lift less than 1 mean? (e.g. lift = .84)
a negative correlation
negative item
absent item
Lift **
also known as the surprise or interest the ratio of the observed frequency of co-occurrence to the expected frequency
maximal frequent itemsets
an itemset that has has no frequent supersets
negative itemset
an itemset that meets minsup and contains at least one negative item
Frequent Itemset
an itemset whose support is equal to or greater than some minsup threshold
What property does confidence of rules generated from the SAME set have?
anti-monotone property with respect to items in the consequent Ex: Frequent Itemset = {A,B,C,D}: c(ABC --> D) >= c(AB --> CD) >= c(A --> BCD)
Itemset
any collection of 0 or more items Ex: {Beer, Diapers, Eggs} is an itemset
infrequent pattern
any pattern that does not meet minsup
CDIST
distinct occurrences with no event-timestamp overlap allowed
CDIST_O
distinct occurrences with possibility of event-timestamp overlap
FP-trees
frequent pattern tree •Each node in the tree is a single item •Each node stores the support count for the itemset comprising of the items on the path from the root to that node •For each transaction in the dataset, the itemset is inserted into the FP-tree, incrementing the count of all nodes along its common prefix path, and creating new nodes beyond the common prefix •Items are sorted in decreasing support order, so most frequent items are at the top of the tree
K-Itemset
if an itemset contains k items, it is called a k-itemset Ex: {Beer, Diapers, Eggs} is a 3-itemset
When is a rule productive?
if improvement is greater than 0
rule improvement
imp(X --> Y) = c(X --> Y) - maxW ⊂𝑋 {c(W --> Y)}
support ratio
min[𝑠(𝑖1), 𝑠(𝑖2), ..., 𝑠(𝑖𝑘)]/(max[𝑠(𝑖1), 𝑠(𝑖2), ..., 𝑠(𝑖𝑘)])
CMINWIN
number of minimal windows of occurrence
COBJ
one occurrence per object
CWIN
one occurrence per sliding window
Time-series data
order and time matters
Sequential data
order matters but no notion of time Ex: DNA
cross-support pattern**
rules that relate low frequency items to high frequency items created from an itemset whose support ratio is under a user-specified threshold.
anti-monotone property of support
support of an itemset never exceeds the support of its subsets
contingency table
table used to examine the relationship between categorical variables [ ]Y !Y X !X
What does a lift of 1 represent
that X and Y are statistically independent. P(X,Y) = P(X)P(Y)
Maxgap
the maximum allowed time span between two elements of the sequence
window size
the maximum allowed time span of any one element of the sequence
Maxspan
the maximum allowed time span of the entire sequence
Mingap
the minimum allowed time span between two elements of the sequence i.e. How much time will you let pass between two elements of a sequence before you just consider them one transaction?
Transaction Width
the number of items in a transaction
Support count [𝝈(𝑿)]
the number of transactions that contain this itemset
GSP (Generalized Sequential Pattern) Algorithm
•Given the set of frequent sequences at level k, it generates all candidate k+1-sequences •Prunes based on the apriori principle (the antimonotonic property of support: all subsequences of a frequent sequence must be frequent) Same algorithm as apriori but this time order matters (i.e. AB is different from BA)
SON algorithm **
•Improves upon sampling •Divide the dataset up into chunks - however large your memory can handle •Process each chunk as a sample (i.e. find frequent itemsets for this chunk) •Once all chunks are processed, take the union of all frequent itemsets- these are the candidate sets •Compare all candidate itemsets against full dataset to get true frequent itemsets But we can eliminate false positives (cross check with total data) and mitigate false negatives (reduce minsup)
PCY algorithm
•In the first pass of Apriori (support counting 1-itemsets), there is typically memory to spare •Use this extra space for an array •Hash pairs of items into this array and keep total sum of support count for each item •Use this array when constructing candidate 2-itemsets •Make pairs {i,j} such that: 1) i and j are frequent items, and 2) {i,j} hashes to a frequent "bucket" If bucket's hash doesn't meet minsup, then prune everything that hashes into that bucket Pre-pruning
