Having specified the factor model, we want to know how much of the variability in $$\mathbf X$$, given by the covariance matrix $$\mathbf \Sigma$$, where $$\mathbf \Sigma=\text{Cov}(\mathbf X) = (\mathbf X- \mathbf M)(\mathbf X- \mathbf M)^T$$, is explained by the factor model.

Suppose there are $$p$$ original variables, to be explained by $$m$$ factors ($$m<p$$). Factor analysis decomposes the $$p \times p$$ variance-covariance matrix $$\mathbf \Sigma$$ of the original variables $$\mathbf X$$ into a $$p \times m$$ loadings matrix $$\mathbf \Lambda$$, where $$\mathbf \Lambda= \text{Cov}(XF)$$, and a $$p \times p$$ diagonal matrix of unexplained variances of original variables, $$\mathbf \Psi$$, where $$\mathbf \Psi = \text{Cov}(\mathbf E)$$, such that

$\mathbf \Sigma = \mathbf \Lambda \mathbf \Lambda^T+ \mathbf \Psi$

This equation indicates that we know the variability in $$\mathbf X$$, given by $$\mathbf \Sigma$$, if we know the loadings matrix $$\mathbf \Lambda$$ and the diagonal matrix of unexplained variances $$\mathbf \Psi$$. Thus, more conceptual, we explain the $$\mathbf \Sigma$$ by two terms. The first term, the loadings matrix $$\mathbf \Lambda$$ gives the coefficients $$(\lambda_{jm})$$ that relate the factors $$(F_{jm})$$ to each particular observations $$(x_j)$$. These coefficients, as we will discuss in the subsequent sections, may be estimated given the observational data. Consequently, the term $$\mathbf\Lambda \mathbf\Lambda^T$$ corresponds to the variability, which may be explained by the factors. This proportion of the overall variability, explained be a linear combination of factors, is denoted as communality. In contrast, the proportion of variability, which can not be explained by a linear combination of the factors, given by the term $$\mathbf \Psi$$ is denoted as uniqueness.

$\mathbf \Sigma = \underbrace{\mathbf\Lambda\mathbf\Lambda^T}_{\text{communality}} + \underbrace{\mathbf\Psi}_{\text{uniqueness}}$

Citation

The E-Learning project SOGA-R was developed at the Department of Earth Sciences by Kai Hartmann, Joachim Krois and Annette Rudolph. You can reach us via mail by soga[at]zedat.fu-berlin.de. You may use this project freely under the Creative Commons Attribution-ShareAlike 4.0 International License.

Please cite as follow: Hartmann, K., Krois, J., Rudolph, A. (2023): Statistics and Geodata Analysis using R (SOGA-R). Department of Earth Sciences, Freie Universitaet Berlin.