Clustering (k-means)

k-Means clustering

partional, exclusive, and complete clustering approach assigns all n items in a dataset to one of k clusters in order to minimize similarities between clusters and maximize similarity within a cluster

clustering

unsupervised machine learning approach used in partitioning unlabeled data into subgroups (clusters) based on similarity

similarity

"alikeness of instances" sometimes expressed as distance function

strengths for k-Means clustering

- simple, non-statistical principles - very flexible and malleable algorithm - wide set of real world applications

weaknesses of k-Means Clustering

- simplistic algorithm - relies on chance - sometimes requires domain knowledge to choose k - not ideal for non-spherical clusters - works with numeric data only

k-Means Clustering Algorithm

1) pick k random cluster centers (*centroids*) 2) assign each item to the nearest cluster center (by distance metric) 3) move each centroid to the mean of the assigned items in that cluster 4) repeat 2 and 3 until convergence achieved

Types of Clustering

Boundary: *Paritional* (boundaries are separate) vs. *Hierarchical* (start with one and increase cluster until boundary includes all) Items: *Exclusive* (every item only belongs to 1 cluster) vs. *Overlapping* (can belong to one or more clusters) Inclusion: *Complete* (every item in data set must belong to a cluster) vs. *Partial* (put items in a cluster if they fit)

WCSS

Within Cluster Sum of Squares SUM ( SUM ( distance from point to its centroid ) ^2 for each cluster ) ) as k --> n, WCSS --> 0

convergence

achieved when the change in cluster assignment is less than a chosen threshold, or non-existent

association vs. clustering

ass: identify patterns clu: learn how to group things

choosing the right "k"

business person is best to pick k- usually you know how many segments you'd want to have rule of thumb: k = sqrt(n/2) or, use: elbow method, average silhouette method, gap statistic

random initialization trap

different initial cluster centers may lead to different cluster results

distance measures

euclidean: we use. sqrt(sum((xi-yi)^2))) also: manhattan (sum(abs(xi-yi))) or pearson, spearman, kendall

elbow method

graph k vs. WCSS, choose k where the elbow point is

good clustering

high intra-class similarity low inter-class similarity

challenges with k-means clustering

sensitive to the initially chosen centroids how to choose value for k?

k-means++

the initialization approach used to overcome the R.I. trap first, choose random data point, calculate distance to that point from all others choose the next point as the one farthest from the first, but still central to all other points keep iterating (R does this for us)