The logit function maps probabilities to values in the real space. Thus, the probability of an event/outcome/success being true \((y=1)\) given the set of predictors \(x_i\), which is our data, is written as
\[logit(P(y=1|x_i))= \beta_0+ \beta_1x_1+ \beta_2x_2+ ... +\beta_kx_k\text{.}\]
For simplification we express the inverse of the function above as
\[\phi(\eta) = \frac{1}{1+e^{-\eta}}\text{,}\]
where \(\eta\) is the linear combination of coefficients \((\beta_i)\) and predictor variables \((x_i)\), calculated as \(\eta = \beta_0+ \beta_1x_1+ \beta_2x_2+ ... +\beta_kx_k\).
The parameters \((\beta_i)\) of the logit model are estimated by the method of maximum likelihood. However, there is no closed-form solution, so the maximum likelihood estimates are obtained by using iterative algorithms such as Newton-Raphson, iteratively re-weighted least squares (IRLS) or gradient descent, among others.
The output of the sigmoid function is interpreted as the probability of a particular observation belonging to class 1. It is written as \(\phi(\eta)=P(y=1|x_i,\beta_i)\), the probability of success \((y=1)\) given the predictor variables \(x_i\) parametrized by the coefficients \(\beta_i\). For example, if we compute \(\phi(\eta)=0.65\) for a particular observation, this means that the chance of this observation belonging to class 1 is 65 %. Similarly, the probability of this observation belonging to class 2 is calculated as \(\phi(\eta)=P(y=0|x_i,\beta_i)= 1 - P(y=1|x_i,\beta_i)=1-0.65=0.35\) or 35 %. For class assignment the predicted probability is then converted into a binary outcome via a unit step function:
\[ \hat y = \begin{cases} 1, & \text{if $\phi(\eta) \ge$ 0.5} \\ 0, & \text{otherwise.} \end{cases} \]
Citation
The E-Learning project SOGA-R was developed at the Department of Earth Sciences by Kai Hartmann, Joachim Krois and Annette Rudolph. You can reach us via mail by soga[at]zedat.fu-berlin.de.
You may use this project freely under the Creative Commons Attribution-ShareAlike 4.0 International License.
Please cite as follow: Hartmann, K., Krois, J., Rudolph, A. (2023): Statistics and Geodata Analysis using R (SOGA-R). Department of Earth Sciences, Freie Universitaet Berlin.