Factorial Analysis vs. Principal Component Analysis (PCA)

Principle Components Analysis (PCA) statistical equation

Includes common factors only. No unique factor. X1 = a11F1 + a12F2 + ... + a1mFm

What are factor loadings?

It is the "a" in the statistical equation: X1 = a11F1 + a12F2 + ... + a1mFm + a1U1

What does it mean to have a large factor loading?

It means that there is a great variance shared by the variable and the factor.

What does factor loadings tell us?

They tell us the magnitude and direction of the association between a variable and a given factor.

"I suspect that these factors should hang together. Is it true?"

Use Confirmatory Factor Analysis (CFA) to test. Test analysis will tell if the statement is true or not, with a 95% confidence interval.

In Factor Analysis (FA), patterns of correlations among variables are identified and used as __________?

indicative of underlying theory

Rule of thumb for "big-enough" factor loading.

loadings > 0.4

Difference between Factor Analysis and Principal Component Analysis

- Principal Component Analysis (PCA) does not necessarily contribute to theory building. - Factor Analysis is more theoretical in nature.

Exploratory Factor Analysis (EFA)

- Summarize data by grouping correlated variables together; - Investigate sets of variables related to theoretical construct; - Usually done at the beginning of research

Confirmatory Factor Analysis (CFA)

- Used to test model(s) of factor structure. - Used when factor structure is known or at least theorized. - Used to test and refine theories. - Used to test generalization of factor structure into new data. - Tested through Structure Equation Model (SEM) methods. * Assessing model fit - "Does the data fit into the model I specified?"

What is the principle behind eigenvalues (or extraction solutions)?

- Each subsequent uncorrelated factor accounts for as much of the variance of the observed variables as possible. - Factors are extracted in order of eigenvalue, from largest to smallest.

What do eigenvalues tell? What are they used for?

- Eigenvalues are used to determine how many factors to extract. - The most common approach: retain all factors with eigenvalues greater than 1 - Used in combination with results of a scree plot

How is the statistical equation different between Factor Analysis (FA) and Principle Components Analysis (PCA)?

- Factor Analysis (FA) includes a UNIQUE factor - Principle Components Analysis (PCA) does not include a unique factor.

What are eigenvalues?

- It tells the amount of variance accounted for by each factor. - It is the sum of squared loadings of a given factor.

What is a scree plot?

A graph that shows the number of eigenvalues (Y-axis) and number of independent factors (X-axis). - The second factor has a eigenvalue that is less than the first factor, and so forth. - The best number of factor solution is one factor short of the elbow.

What is rotation in Factor Analysis (FA)?

After creating the first factor, the initial pattern is often adjusted (rotated) so that each individual variable has substantial loadings on as few factors as possible (preferably only one). This adjustment is called rotation to simple structure, and seeks to provide a more interpretable outcome.

How do eigenvalues change across different unrelated factors?

All variables typically have the most substantial loadings in the first factor, meaning the highest eigenvalue. Eigenvalue then decreases across subsequent factors.

Commonality between Factor Analysis and Principal Component Analysis

Common objective: Data reduction - Both distills many variables into a few sets of factors

What is commonality in Factor Analysis?

Commonality is the proportion of variance in a variable that is accounted for by a set of common factors. * Only when factors are uncorrelated!!

Process of Factor Analysis and Principle Components Analysis - How are they done theoretically?

Comparing CORRELATIONS among variables and then group them together in order to: - Maximize correlations among variables within a factor/component. - Minimize correlations between factors/components. Then, patterns of correlation are identified and used as: - descriptors and aggregates (PCA), or - indicative of underlying theory (FA)

Types of Factor Analysis

Exploratory Comfirmatory

Factor Analysis (FA) statistical equation

Includes common factors (aka. latent variables) and a unique factor. X1 = a11F1 + a12F2 + ... + a1mFm + a1U1 X1 - observed variable #1 a1 - coefficient (aka. factor loading) of X1 F1, F2, Fm - factor 1, factor 2, factor m U1 - unique factor of X1

What are the common statistical procedures to do Exploratory Factor Analysis (EFA)?

Principle Axes Approach Principle Factors Approach

What is it when patterns of correlations among variables are identified and used as descriptors and aggregates?

Principle Components Analysis (PCA)

Descriptors and aggregates is to Indicative of underlying theory as _______ is to ________?

Principle Components Analysis (PCA) is to Factor Analysis (FA)

How to calculate commonality in Factor Analysis?

Sum of (squared loading) of a variable.