It is important to realize that the linear factor model is not identifiable. This means that there are two or more parametrizations which are observationally equivalent, or in other words, there exist an infinite number of different matrices \(\mathbf\Lambda\) which may generate the same \(\mathbf x\) values.
That is why factor analysis usually proceeds in two stages. In the first, one set of loadings, \(\mathbf\Lambda\), is calculated which yields theoretical variances and covariances that fit the observed ones as closely as possible. These loadings, however, may not provide a reasonable interpretation. Thus, in the second stage, the loadings, \(\mathbf\Lambda\), are transformed in an effort to arrive at another set that fit equally well the observed variances and covariances, but are more easily interpreted.
The process of transforming a factor pattern is generally referred to as rotation. There are two basic types of transformations: orthogonal (uncorrelated factors) and oblique (correlated factors). Using orthogonal rotation, we preserve the independence of the factors. Oblique rotation factors are allowed to correlate. Two popular methods are the varimax method for orthogonal factor transformation, and the promax method for oblique factors rotation. Both methods are implemented in R.
The varimax rotation method maximizes the variance of the squared loadings for each factor, thus making some of these loadings as large as possible, and the rest as small as possible in absolute value. Consequently, the variables become divisible into groups such that the loadings within each group are high on a single factor, moderate to low on a few factors and negligible on the remaining factors. The varimax method encourages the detection of factors each of which is related to few variables. It discourages the detection of factors influencing all variables (Tryfos 1997).
Citation
The E-Learning project SOGA-R was developed at the Department of Earth Sciences by Kai Hartmann, Joachim Krois and Annette Rudolph. You can reach us via mail by soga[at]zedat.fu-berlin.de.
You may use this project freely under the Creative Commons Attribution-ShareAlike 4.0 International License.
Please cite as follow: Hartmann, K., Krois, J., Rudolph, A. (2023): Statistics and Geodata Analysis using R (SOGA-R). Department of Earth Sciences, Freie Universitaet Berlin.