The logit function maps probabilities to values in the real space. Thus, the probability of an event/outcome/success being true $$(y=1)$$ given the set of predictors $$x_i$$, which is our data, is written as

$logit(P(y=1|x_i))= \beta_0+ \beta_1x_1+ \beta_2x_2+ ... +\beta_kx_k\text{.}$

For simplification we express the inverse of the function above as

$\phi(\eta) = \frac{1}{1+e^{-\eta}}\text{,}$

where $$\eta$$ is the linear combination of coefficients $$(\beta_i)$$ and predictor variables $$(x_i)$$, calculated as $$\eta = \beta_0+ \beta_1x_1+ \beta_2x_2+ ... +\beta_kx_k$$.

The parameters $$(\beta_i)$$ of the logit model are estimated by the method of maximum likelihood. However, there is no closed-form solution, so the maximum likelihood estimates are obtained by using iterative algorithms such as Newton-Raphson, iteratively re-weighted least squares (IRLS) or gradient descent, among others.

The output of the sigmoid function is interpreted as the probability of a particular observation belonging to class 1. It is written as $$\phi(\eta)=P(y=1|x_i,\beta_i)$$, the probability of success $$(y=1)$$ given the predictor variables $$x_i$$ parametrized by the coefficients $$\beta_i$$. For example, if we compute $$\phi(\eta)=0.65$$ for a particular observation, this means that the chance of this observation belonging to class 1 is 65 %. Similarly, the probability of this observation belonging to class 2 is calculated as $$\phi(\eta)=P(y=0|x_i,\beta_i)= 1 - P(y=1|x_i,\beta_i)=1-0.65=0.35$$ or   35 %. For class assignment the predicted probability is then converted into a binary outcome via a unit step function:

$\hat y = \begin{cases} 1, & \text{if \phi(\eta) \ge 0.5} \\ 0, & \text{otherwise.} \end{cases}$

Citation

The E-Learning project SOGA-R was developed at the Department of Earth Sciences by Kai Hartmann, Joachim Krois and Annette Rudolph. You can reach us via mail by soga[at]zedat.fu-berlin.de.

You may use this project freely under the Creative Commons Attribution-ShareAlike 4.0 International License.

Please cite as follow: Hartmann, K., Krois, J., Rudolph, A. (2023): Statistics and Geodata Analysis using R (SOGA-R). Department of Earth Sciences, Freie Universitaet Berlin.