Any decision made based on a hypothesis test may be incorrect. In the framework of hypothesis tests there are two types of errors: Type I error and type II error. A type I error occurs when a true null hypothesis is rejected (a “false positive”), while a type II error occurs when a false null hypothesis is not rejected (a “false negative”). In other words, a type I error is detecting an effect that is not present, while a type II error is failing to detect an effect that is present.

\[ \begin{array}{l|ll} & H_0 \text{ is true} & H_0 \text{ is false} \\ \hline \text{Do not reject } H_0 & \text{Correct decision} & \text{Type II error} \\ \text{Reject } H_0 & \text{Type I error} & \text{Correct decision} \\ \hline \end{array} \]

If you are confused about type I and type II errors, you may find this illustration helpful (here).

---

Conducting a hypothesis test always implies, that there is a chance of making an incorrect decision. The probability of the type I error (a true null hypothesis is rejected) is commonly called the significance level of the hypothesis test and is denoted by \(\alpha\). The probability of a type II error (a false null hypothesis is not rejected) is denoted by \(\beta\). Keep in mind, that for a fixed sample size the smaller we specify the significance level, \(\alpha\), the larger will be the probability, \(\beta\), of not rejecting a false null hypothesis.

The outcome of a hypothesis test is a statement in favor of the null hypothesis or in favor of the alternative hypothesis. If the null hypothesis is rejected, the data does provide sufficient evidence to support the alternative hypothesis. If the null hypothesis is not rejected, the data does not provide sufficient evidence to support the alternative hypothesis. If the hypothesis test is performed at the significance level \(\alpha\) and the null hypothesis is rejected, one may state that the test results are statistically significant at the \(\alpha\) level. If the null hypothesis is not rejected at the significance level \(\alpha\), one may state that the test results are not statistically significant at the \(\alpha\) level.

---

Citation

The E-Learning project SOGA-R was developed at the Department of Earth Sciences by Kai Hartmann, Joachim Krois and Annette Rudolph. You can reach us via mail by soga[at]zedat.fu-berlin.de.

You may use this project freely under the Creative Commons Attribution-ShareAlike 4.0 International License.

Please cite as follow: Hartmann, K., Krois, J., Rudolph, A. (2023): Statistics and Geodata Analysis using R (SOGA-R). Department of Earth Sciences, Freie Universitaet Berlin.