Having specified the factor model, we want to know how much of the variability in \(\mathbf X\), given by the covariance matrix \(\mathbf \Sigma\), where \(\mathbf \Sigma=\text{Cov}(\mathbf X) = (\mathbf X- \mathbf M)(\mathbf X- \mathbf M)^T\), is explained by the factor model.

Suppose there are \(p\) original variables, to be explained by \(m\) factors (\(m<p\)). Factor analysis decomposes the \(p \times p\) variance-covariance matrix \(\mathbf \Sigma\) of the original variables \(\mathbf X\) into a \(p \times m\) loadings matrix \(\mathbf \Lambda\), where \(\mathbf \Lambda= \text{Cov}(XF)\), and a \(p \times p\) diagonal matrix of unexplained variances of original variables, \(\mathbf \Psi\), where \(\mathbf \Psi = \text{Cov}(\mathbf E)\), such that

\[\mathbf \Sigma = \mathbf \Lambda \mathbf \Lambda^T+ \mathbf \Psi\]

This equation indicates that we know the variability in \(\mathbf X\), given by \(\mathbf \Sigma\), if we know the loadings matrix \(\mathbf \Lambda\) and the diagonal matrix of unexplained variances \(\mathbf \Psi\). Thus, more conceptual, we explain the \(\mathbf \Sigma\) by two terms. The first term, the loadings matrix \(\mathbf \Lambda\) gives the coefficients \((\lambda_{jm})\) that relate the factors \((F_{jm})\) to each particular observations \((x_j)\). These coefficients, as we will discuss in the subsequent sections, may be estimated given the observational data. Consequently, the term \(\mathbf\Lambda \mathbf\Lambda^T\) corresponds to the variability, which may be explained by the factors. This proportion of the overall variability, explained be a linear combination of factors, is denoted as communality. In contrast, the proportion of variability, which can not be explained by a linear combination of the factors, given by the term \(\mathbf \Psi\) is denoted as uniqueness.

\[\mathbf \Sigma = \underbrace{\mathbf\Lambda\mathbf\Lambda^T}_{\text{communality}} + \underbrace{\mathbf\Psi}_{\text{uniqueness}}\]

---

Citation

The E-Learning project SOGA-R was developed at the Department of Earth Sciences by Kai Hartmann, Joachim Krois and Annette Rudolph. You can reach us via mail by soga[at]zedat.fu-berlin.de.

You may use this project freely under the Creative Commons Attribution-ShareAlike 4.0 International License.

Please cite as follow: Hartmann, K., Krois, J., Rudolph, A. (2023): Statistics and Geodata Analysis using R (SOGA-R). Department of Earth Sciences, Freie Universitaet Berlin.