Factor Analysis

SESSION 13
BRM
Example: Variables
V1: It is important to buy a toothpaste that prevents cavities
V2: I like a toothpaste that gives shiny teeth
V3: A toothpaste should strengthen your gums
V4: I prefer a toothpaste that freshens breath
V5: Prevention of tooth decay is not an important benefit offered by a toothpaste
V6: The most important consideration in buying a toothpaste is attractive teeth
Introduction
Uni-variate vs. Multi-variate technique – Based on no. of dependent
variables
Factor analysis-
◦ Multivariate technique
◦ A data reduction method
◦ Helps to reduce a large number of variables to a few manageable factors, thus
reducing complexity
◦ These factors explain most part of the variations of the original set of data
Factor-
◦ A factor is a linear combination of variables.
◦ It is not directly observable but needs to be inferred from the input variables
◦ The factors are statistically independent
◦
Factors Underlying Selected Psychographics
and Lifestyles
Factor 2

Football Baseball

Evening at home
Factor 1
Go to a party
Home is best place

Plays
Movies
Conditions for factor analysis
Metric data- The data should be either interval or ratio scale in nature.

The responses need to be standardized, especially when the scales are

different
The standardization helps in comparison of different responses from such scales

Sample size – Should be at least 4-5 times of the number of

variables/statements/items
 Factor analysis model
If the variables are standardized, the factor analysis model may be represented as:

Xi = Ai 1F1 + Ai 2F2 + Ai 3F3 + . . . + AimFm + ViUi

where

Xi = i th standardized variable
Aij = standardized multiple regression coefficient of variable i on common
factor j
F = common factor
Vi = standardized regression coefficient of variable i on unique factor i
Ui = the unique factor for variable i
m = number of common factors
 Conditions for factor analysis
Correlation between variables -
◦ Basic principle behind the application of factor analysis -initial set of variables should be
highly correlated
◦ Factor analysis may not be appropriate technique if this correlation is small
◦ Measured by - Bartlett’s test and KMO statistics

(A) Bartlett’s test of sphericity- The hypothesis

H0 : Correlation matrix is insignificant i.e. correlation matrix is an identity matrix (diagonal value = 1,
rest= 0)

H1 : Correlation matrix is significant

The test converts it into a chi-square statistics with df equal to [(k(k-1))/2], where k is the
number of variables on which factor analysis is applied. The significance (if sig. < 0.05) of the
correlation matrix ensures that a factor analysis exercise could be carried
Conditions for factor analysis
(B) The KMO statistics
◦ Compares the magnitude of observed correlation coefficients with the
magnitudes of partial correlation coefficients
◦ It takes value between 0 and 1
◦ Should be greater than 0.5 for the application of factor analysis
Steps in factor analysis exercise
(a) Extraction of factors
◦ Decision on how many factors to be extracted needs to be made - Principal
components analysis (PCA) is widely and most commonly used
◦ Input is total variance of the standardized variables
◦ Many other methods available –Common factor analysis, unweighted least
squares, generalized least squares, maximum likelihood, principal image
factoring etc.

◦ PCA- Since factors are linear combinations of the variables which are supposed to
be highly correlated. Therefore the mathematical form could be
Steps in factor analysis exercise

The principal component methodology involves searching for those values

of Wi so that the first factor explains the largest portion of total variance.
This is called the first principal factor.
This explained variance is then subtracted from the original input matrix so
as to yield a residual matrix
A second principal factor is extracted from the residual matrix in a way such
that the second factor takes care of most of the residual variance.
Imp- The second principal factor has to be statistically independent of the
first principal factor. The same principle is then repeated until there is little
variance to be explained.
Steps in factor analysis exercise
Decision of number of factors-
◦ Scree plot- the point where it becomes flat
◦ Eigen value >1 (Eigen value Ratio of between group variance to within group variance)
◦ Based on % of variance- accepted is atleast more than 60%. This should be supported by
theory
◦ Based on split half reliability- Based on significance of the eigen value
Steps in factor analysis
(b) Rotation of factors
◦ The initial factors are difficult to interpret, hence initial solution is rotated
◦ Varimax (orthogonal basis) rotation is used
◦ Non orthogonal rotation is called oblique rotation

◦ Varimax rotation-
◦ Maximizes the variance of the loadings within each factor.
◦ The variance of the factor is largest when its smallest loading tends towards zero and its largest
loading tends towards unity.
◦ Therefore, the rotation is carried out in such way so that the factor loadings as in the first step
are close to unity or zero

Note: The total variance explained by Principal component method and Varimax rotation is same.
However, the variance explained by each factor could be different.
The communalities of each variable remains unchanged by both the methods
Rotation of the factors
Factor 2

Football Baseball

Evening at home
Factor 1
Go to a party
Home is best place

Plays
Movies
Steps in factor analysis
(c) Interpretation - To interpret the results, a cut-off point on the factor loading is
selected.

No hard and fast rule to decide on the cut-off point. However, generally it is taken to be
greater than 0.5.

All variables attached to a factor are used for naming the factors. This is a subjective
procedure and different researchers may name same factors differently.

A variable should have a high loading only on one factor and a low loading on other
factors.

.
Steps in factor analysis
(c) Validation – Residual from initial vs. reproduced correlation matrix,

◦ Thumb rule- “Non-redundant residuals with absolute values greater than 0.05” should not
be more than 50%
Conducting a factor analysis
Establishing the strength of factor analysis solution (KMO and Bartlett’s test)
Initial factor interpretation
Rotation of factors - Establishing statistical independence of extracted factors
Factor score coefficient matrix
Factor loadings and eigenvalues- total variance accounted by the extracted factor
Communalities
Labelling or naming the factors

Usage of selected factors in other techniques

 Take Home Exercise
1. Students were asked about their attitude towards the statistics, and data was collected on a
5 point scale, 5 being strongly disagree. The collected data is provided in Statistics Attitude
Questionnaire file. Conduct a factor analysis and interpret the results. Based on your
findings, give the appropriate recommendations.

2. Reasons for car usage were asked on a 7 point scale, 7 being the highest score. The
collected data is provided in ‘Car Usage data_Factor Analysis’ file. Conduct a factor analysis
and interpret the results. Based on your findings, give the appropriate
recommendations
Thank You
Key terms in factor analysis
Factor Matrix (Component Matrix) – It contains the factor loadings of all the variables on
all the extracted factors

Eigenvalue – The percentage of variance explained by each factor is computed by

eigenvalue
◦ The eigenvalue of any factor is obtained by taking the sum of squares of the factor loadings of
each component.

Communality - It indicates how much of each variable is accounted for by the underlying
factors taken together
◦ In other words, it is a measure of the percentage of variable’s variation that is explained by the
factors.

Factor Loading – Correlation coefficient between the factor score and the variables of the
study

Factor Scores – Composite scores estimated for each respondent on the extracted factors
Uses of Factor Analysis
Scale construction: Especially in multi-item scale
Establish antecedents: This method reduces multiple input variables into
grouped factors. Thus, the independent variables can be grouped into
broad factors.
Psychographic profiling: Different independent variables are grouped to
measure independent factors. These are then used for identifying
personality types e.g. MBTI
Segmentation analysis: E.g. there could be different sets of
two-wheelers-customers because of different importance they give to
factors like prestige, economy etc.
Marketing studies: E.g. identify attributes that influence brand choice,
insight into media habits, identify characteristic of price sensitive
customers
Applications of factor analysis in
other multivariate techniques
Multiple regression – Factor scores can be used in place of independent variables in
a multiple regression estimation. This way we can overcome the problem of
multicollinearity.
Simplifying the discrimination solution – A number of independent variables in a
discriminant model can be replaced by a set of manageable factors before estimation.
Simplifying the cluster analysis solution - To make the data manageable, the variables
selected for clustering can be reduced to a more manageable number using a factor
analysis
Perceptual mapping in multidimensional scaling - Factor analysis that results in
factors can be used as dimensions with the factor scores as the coordinates to
develop attribute-based perceptual maps where one is able to comprehend the
placement of brands or products according to the identified factors under study.