Any decision made based on a hypothesis test may be incorrect. In the framework of hypothesis tests, there are two types of errors: Type I error and type II error. A type I error occurs when a true null hypothesis is rejected (a “false positive”), while a type II error occurs when a false null hypothesis is not rejected (a “false negative”). In other words, a type I error detect an effect that is not present, while a type II error fails to detect an effect that is present.
$H_{0}$ is true | $H_{0}$ is false | |
---|---|---|
Do not reject $H_{0}$ | Correct decision | Type II error |
Reject $H_{0}$ | Type II error | Correct decision |
If you are confused about type I and type II errors, you may find this illustration helpful (here).
Conducting a hypothesis test always implies that there is a chance of making an incorrect decision. The probability of the type I error (a true null hypothesis is rejected) is commonly called the significance level of the hypothesis test and is denoted by $\alpha$. The probability of a type II error (a false null hypothesis is not rejected) is denoted by $\beta$. Remember that for a fixed sample size, the smaller we specify the significance level, $\alpha$, the larger will be the probability, $\beta$, of not rejecting a false null hypothesis.
The outcome of a hypothesis test is a statement in favour of the null hypothesis or favour of the alternative hypothesis. If the null hypothesis is rejected, the data does provide sufficient evidence to support the alternative hypothesis. If the null hypothesis is not rejected, the data does not provide sufficient evidence to support the alternative hypothesis. If the hypothesis test is performed at the significance level $\alpha$ and the null hypothesis is rejected, one may state that the test results are statistically significant at the $\alpha$ level. If the null hypothesis is not rejected at the significance level $\alpha$, one may state that the test results are not statistically significant at the $\alpha$ level.
Citation
The E-Learning project SOGA-Py was developed at the Department of Earth Sciences by Annette Rudolph, Joachim Krois and Kai Hartmann. You can reach us via mail by soga[at]zedat.fu-berlin.de.
Please cite as follow: Rudolph, A., Krois, J., Hartmann, K. (2023): Statistics and Geodata Analysis using Python (SOGA-Py). Department of Earth Sciences, Freie Universitaet Berlin.