Consider \(p\) manifest variables, denoted by \(x_1, x_2, ..., x_p\), and the variables mean, denoted by \(\mu_1, \mu_2, ..., \mu_p\), and the covariance matrix of \(\mathbf X\), denoted by \(\Sigma\).

\[ \begin{align} \mathbf X_{p \times 1}= \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_p \\ \end{bmatrix}\text{,}\qquad \mathbf \mu_{p \times 1}= \begin{bmatrix} \mu_1 \\ \mu_1 \\ \vdots \\ \mu_p \\ \end{bmatrix}\text{,}\qquad \mathbf \Sigma_{p \times p}= \begin{bmatrix} \sigma_{11} & \sigma_{12} & \dots & \sigma_{1p} \\ \sigma_{12} & \sigma_{22} & \dots & \sigma_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ \sigma_{1p} & \sigma_{2p} & \dots & \sigma_{pp} \\ \end{bmatrix}\text{,}\qquad \end{align} \]

Further consider \(m\) factors, denoted by \(F_1, F_2,...,F_m\)

\[ \begin{align} \mathbf F_{m \times 1}= \begin{bmatrix} F_1 \\ F_2 \\ \vdots \\ F_m \\ \end{bmatrix} \end{align} \]

The basic idea behind factor analysis is that of regression, or conditional expectation, which means that we express each of the manifest (observed) variables as a linear combinations of latent variables (or factors). Thus, if we have \(p\) manifest variables and \(m\) factors we max write

\[x_j = \mu_j + \lambda_{j1}F_{j1}+ \lambda_{j2}F_{j2}+...+ \lambda_{jm}F_{jm}+ e_j\qquad j = 1,2,...,p\text{,}\]

where the factors, \(F\), are assumed to have zero means and unit standard deviations.The error term \(e_j\) is also assumed to have zero mean and standard deviation, \(\sigma_j\).

Expressed in matrix form this equation becomes

\[ \begin{align} \begin{bmatrix} x_1 \\ x_2 \\ \vdots \\ x_p \\ \end{bmatrix}= \begin{bmatrix} \mu_1 \\ \mu_2 \\ \vdots \\ \mu_p \\ \end{bmatrix}+ \begin{bmatrix} \lambda_{11} & \lambda_{12} & \dots & \lambda_{1m} \\ \lambda_{21} & \lambda_{22} & \dots & \lambda_{2m} \\ \vdots & \vdots & \ddots & \vdots \\ \lambda_{p1} & \lambda_{p2} & \dots & \lambda_{pp} \\ \end{bmatrix} \begin{bmatrix} F_1 \\ F_2 \\ \vdots \\ F_p \\ \end{bmatrix}+ \begin{bmatrix} e_1 \\ e_2 \\ \vdots \\ e_p \\ \end{bmatrix}\text{,} \end{align} \] which can be expressed more compact in matrix notation.

\[\mathbf X_{p\times 1} = \mathbf{\mu}_{p\times 1}+\Lambda_{p\times m} F_{m\times 1}+\mathbf e_{p\times 1}\] Rewriting this equation gives us the exploratory factor model:

\[\mathbf X-\mu = \Lambda F + \mathbf e\]

---