It is important to realize that the linear factor model is not identifiable. This means that there are two or more parametrizations which are observationally equivalent, or in other words, there exist an infinite number of different matrices \(\Lambda\) which may generate the same \(\mathbf x\) values.

That is why factor analysis usually proceeds in two stages. In the first, one set of loadings, \(\Lambda\), is calculated which yields theoretical variances and covariances that fit the observed ones as closely as possible. These loadings, however, may not provide a reasonable interpretation. Thus, in the second stage, the loadings, \(\Lambda\), are transformed in an effort to arrive at another set that fit equally well the observed variances and covariances, but are easier to interpret.

The process of transforming a factor pattern is generally referred to as rotation. There are two basic types of transformations: orthogonal (uncorrelated factors) and oblique (correlated factors). Using orthogonal rotation, we preserve the independence of the factors. With oblique rotation factors are allowed to correlate. Two poplar methods are the varimax method for orthogonal factor transformation, and the promax method for oblique factors rotation. Both methods are implemented in R.

The varimax method maximizes the variance of the squared loadings for each factor, thus making some of these loadings as large as possible, and the rest as small as possible in absolute value. Consequently, the variables become divisible into groups such that the loadings within each group are high on a single factor, moderate to low on a few factors and negligible on the remaining factors. The varimax method encourages the detection of factors each of which is related to few variables. It discourages the detection of factors influencing all variables (Tryfos 1997).