The \(\chi^2\) goodness-of-fit test is applied to perform hypothesis tests on the distribution of a qualitative (categorical) variable or a discrete quantitative variable, that has only finitely many possible values.
The basic logic of the \(\chi^2\) goodness-of-fit test is to compare the frequencies of two variables. We compare the observed frequencies of a sample with the expected frequencies.
Consider a simple example:
On September 22, 2013 the German Federal Election 2013 was held. More than 44 million people turned up to vote. 41.5 % of German voters decided to vote for the Christian Democratic Union (CDU) and 25.7 % for the Social Democratic Party (SPD). For the sake of simplicity we subsume the remaining percentage of votes (32.8 %) as Others.
Based on that data, we may build a frequency table:
\[ \begin{array}{|l|c|} \hline \ \text{Party} & \text{Percentage} & \text{Relative frequency}\\ \hline \ \text{CDU} & 41.5 & 0.415 \\ \ \text{SPD} & 25.7 & 0.257 \\ \ \text{Others} & 32.8 & 0.328 \\ \hline \ & 100 & 1 \\ \hline \end{array} \]
The third column of the table above corresponds to the relative frequencies of the German population/voters. For this exercise we take a random sample. We ask 123 students of FU Berlin about their party affiliation and record the following answers:
## [1] "CDU" "SPD" "Others" "SPD" "SPD" "CDU" "Others" "SPD"
## [9] "Others" "Others" "SPD" "Others" "Others" "Others" "CDU" "SPD"
## [17] "CDU" "CDU" "CDU" "SPD" "SPD" "Others" "Others" "SPD"
## [25] "Others" "SPD" "Others" "Others" "CDU" "CDU" "SPD" "SPD"
## [33] "Others" "SPD" "CDU" "Others" "SPD" "CDU" "CDU" "CDU"
## [41] "CDU" "Others" "Others" "CDU" "CDU" "CDU" "CDU" "Others"
## [49] "CDU" "SPD" "CDU" "Others" "SPD" "CDU" "Others" "CDU"
## [57] "CDU" "SPD" "SPD" "Others" "Others" "CDU" "Others" "CDU"
## [65] "SPD" "Others" "SPD" "SPD" "SPD" "Others" "SPD" "Others"
## [73] "SPD" "CDU" "Others" "CDU" "Others" "Others" "CDU" "CDU"
## [81] "CDU" "Others" "Others" "SPD" "CDU" "Others" "SPD" "SPD"
## [89] "SPD" "CDU" "CDU" "Others" "CDU" "Others" "CDU" "CDU"
## [97] "SPD" "CDU" "Others" "Others" "Others" "CDU" "Others" "SPD"
## [105] "Others" "SPD" "SPD" "Others" "Others" "CDU" "SPD" "CDU"
## [113] "CDU" "SPD" "SPD" "CDU" "Others" "SPD" "Others" "Others"
## [121] "Others" "CDU" "CDU"In the next step we count the occurrence of each category (party) in our sample. These quantities are the observed frequencies:
## sample_FUB
## CDU Others SPD
## 43 44 36In the next step we compute the expected frequency, denoted \(E\), for each category:
\[E = n \times p\text{,}\]
where \(n\) is the sample size and \(p\) is the corresponding relative frequency taken from the table above.
\[E_{CDU} = n\times p = 123 \times 0.415 = 51.045\]
\[E_{SPD} = n\times p = 123 \times 0.257 = 31.611\]
\[E_{Others} = n\times p = 123 \times 0.328 = 40.344\]
Note: Although we deal with individual counts, represented by integer values, the expected frequency, \(E\), is a floating point number. That is fine.
Now, we put the observed frequencies and the expected frequencies together into one table:
\[ \begin{array}{|l|c|} \hline \ \text{Party} & \text{Observed frequency} & \text{Expected frequency}\\ \hline \ \text{CDU} & 43 & 51.045 \\ \ \text{SPD} & 36 & 31.611 \\ \ \text{Others} & 44 & 40.344 \\ \hline \ & 123 & 123 \\ \hline \end{array} \]
Great! Once we have the expected frequencies, we have to check for two assumptions. First, we have to make sure that all expected frequencies are 1 or greater, and second, at most 20 % of the expected frequencies are less than 5. By looking at the table we may confirm that both assumptions are fulfilled.
Now, we have all ingredients we need, except the test statistic, to perform a \(\chi^2\) goodness-of-fit test.
The \(\chi^2\) test statistic for a goodness-of-fit is given by
\[\chi^2=\sum \frac{(O-E)^2}{E}\text{,}\] where \(O\) corresponds to the observed frequencies and \(E\) to the expected frequencies. The test statistic \(\chi^2\) approximates a chi-square distribution, if the null hypothesis is true. The number of degrees of freedom is 1 less than the number of possible values (categories) for the variable under consideration:
\[df = c-1\]
Based on the observed and expected frequencies given in the table above it is fairly straightforward to calculate the \(\chi^2\)-value. However, to make the calculation procedure easier to follow, we put together all the necessary computational steps into one table:
\[ \begin{array}{|l|c|} \hline \ \text{Religion} & \text{Observed} & \text{Expected} & \text{Difference} & \text{Square of difference} & \chi^2\text{ subtotal}\\ \ & \text{frequency} & \text{frequency} & O-E & (O-E)^2 & (O-E)^2/E\\ \hline \ \text{CDU} & 43 & 51.045 & -8.045 & 64.722025 & 1.2679405\\ \ \text{SPD} & 36 & 31.611 & 4.389 & 19.263321 & 0.6093866\\ \ \text{Others} & 36 & 40.344 & 3.656 & 13.366336 & 0.3313091\\ \hline \ & 123 & 123 & 3.5527137\times 10^{-15} & & 2.2086363\\ \hline \end{array} \]
In our example the \(\chi^2\) test statistic for a goodness-of-fit evaluates to
\[\chi^2=\sum \frac{(O-E)^2}{E} \approx2.209\]
If the null hypothesis is true, the observed and expected frequencies are roughly equal. This results in a small value of the \(\chi^2\) test statistic, thus, supporting \(H_0\). If, however, the value of the \(\chi^2\) test statistic is large, the data provides evidence against \(H_0\).
In our case, we may compare the empirical \(\chi^2\) test statistic with the corresponding critical \(\chi^2\) value for a significance level of 95 % with a degree of freedom of 3 categories minus 1:
qchisq(0.95, df = 3 - 1)## [1] 5.991465Since our empirical \(\chi^2\) value is smaller than the critical \(\chi^2\) value, we cannot reject the null hypotheses!
\(\chi^2\) goodness-of-fit test: An Example
In order to get some hands-on experience we apply the \(\chi^2\) goodness-of-fit test in an exercise. For this we load the students data set, which you may also download here.
students <- read.csv("https://userpage.fu-berlin.de/soga/data/raw-data/students.csv")The students data set consists of 8239 rows, each of them representing a particular student, and 16 columns, each of them corresponding to a variable/feature related to that particular student. These self-explaining variables are: stud.id, name, gender, age, height, weight, religion, nc.score, semester, major, minor, score1, score2, online.tutorial, graduated, salary.
Recall, \(\chi^2\) goodness-of-fit
tests are applied for qualitative (categorical) variables or discrete
quantitative variables. There are several categorical variables in the
students data set, such as gender,
religion, major, minor and
graduated, among others.
In order to showcase the \(\chi^2\) goodness-of-fit test we examine, if religions are equally distributed among students compared to the distribution of religions among the population of the European Union. The data on continental scale is provided in the report “Discrimination in the EU in 2012” (European Union: European Commission, Special Eurobarometer, 393, p. 233). The report provides data for 8 categories: 48 % of the people are ascribed as Catholic, 16 % as Non believer/Agnostic, 12 % as Protestant, 8% as Orthodox, 7% as Atheist, 4 % as Other Christian, 3 % as Other religion/None stated and 2 % as Muslim. We plot the data in form of a pie chart for a better understanding:
data <- c(48, 16, 12, 8, 7, 4, 3, 2)
data_labels <- c(
"Catholic", "Non believer/\nAgnostic", "Protestant",
"Orthodox", "Atheist", "Other Christian",
"Other religion/None stated", "Muslim"
)
par(mar = c(3, 2, 3, 2))
library(RColorBrewer)
cols <- brewer.pal(length(data), "Set3")
pie(
x = data,
labels = data_labels,
col = cols,
radius = 1
)
Data preparation
We start with data exploration and data preparation.
First, we want to know which categories are available in the
students data set. Therefore, we apply the
unique() function, which provides access to the levels
(categories) of a variable.
unique(students$religion)## [1] "Muslim" "Other" "Protestant" "Catholic" "Orthodox"Obviously, in the students data set there are 0 different categories, compared to 8 categories provided by the report of the EU. Thus, in order to make comparisons, we summarize the categories of EU report to 5 categories: “Catholic”, “Muslim”, “Orthodox”, “Protestant” and “Other”. Be careful not to mix-up categories during that step!
# set category names
data_labels <- c("Catholic", "Muslim", "Orthodox", "Other", "Protestant")
# recode European data according to category names
data_raw <- c(48, 2, 8, sum(16, 7, 4, 3), 12)
# generate a data.frame object
data <- data.frame(data_raw / 100)
row.names(data) <- data_labels
colnames(data) <- "relative_frequency"
data## relative_frequency
## Catholic 0.48
## Muslim 0.02
## Orthodox 0.08
## Other 0.30
## Protestant 0.12Now, we take a random sample. We randomly pick 256 students and count
the number of students in each particular category of the
religion using the table() function. Recall,
that this quantity corresponds to the observed
frequencies.
n <- 256
students_sample <- sample(students$religion, n)
O_frequencies <- table(students_sample)
O_frequencies## students_sample
## Catholic Muslim Orthodox Other Protestant
## 83 15 16 82 60With one line of code we insert the observed frequencies into
data, the data.frame we constructed above.
data$observed_frequencies <- O_frequencies
data## relative_frequency observed_frequencies
## Catholic 0.48 83
## Muslim 0.02 15
## Orthodox 0.08 16
## Other 0.30 82
## Protestant 0.12 60In the next step we calculate the expected frequencies. Recall the equation:
\[E = n \times p\]
We insert the expected frequencies as a new column in
data.
n <- 256
data$expected_frequencies <- n * data$relative_frequency
data## relative_frequency observed_frequencies expected_frequencies
## Catholic 0.48 83 122.88
## Muslim 0.02 15 5.12
## Orthodox 0.08 16 20.48
## Other 0.30 82 76.80
## Protestant 0.12 60 30.72Once we know the expected frequencies, we have to check for two assumptions. First, we have to make sure that all expected frequencies are 1 or greater. Second, at most 20 % of the expected frequencies should be less than 5. By looking at the table we may confirm that both assumptions are fulfilled.
Perfect, now we are done with the preparation! The data set is ready to be analyzed with the \(\chi^2\) goodness-of-fit test. Recall the question we are interested in: Is the religion equally distributed among students compared to the distribution of the religion among the population of the European Union?
Hypothesis Testing
In order to conduct the \(\chi^2\) goodness-of-fit test we follow the step-wise implementation procedure for hypothesis testing. The \(\chi^2\) goodness-of-fit test follows the same step-wise procedure as hypothesis tests for the population mean:
\[ \begin{array}{l} \hline \ \text{Step 1} & \text{State the null hypothesis } H_0 \text{ and alternative hypothesis } H_A \text{.}\\ \ \text{Step 2} & \text{Decide on the significance level, } \alpha\text{.} \\ \ \text{Step 3} & \text{Compute the value of the test statistic.} \\ \ \text{Step 4} &\text{Determine the p-value.} \\ \ \text{Step 5} & \text{If } p \le \alpha \text{, reject }H_0 \text{; otherwise, do not reject } H_0 \text{.} \\ \ \text{Step 6} &\text{Interpret the result of the hypothesis test.} \\ \hline \end{array} \]
Step 1: State the null hypothesis \(H_0\) and alternative hypothesis \(H_A\)
The null hypothesis states that the religion is equally distributed among students compared to the distribution of the religion among the population of the European Union:
\[H_0: \quad \text{The variable has the specified distribution}\]
Alternative hypothesis:
\[H_A: \quad \text{The variable does not have the specified distribution} \]
Step 2: Decide on the significance level, \(\alpha\)
\[\alpha = 0.01\]
alpha <- 0.01Step 3 and 4: Compute the value of the test statistic and the p-value
For illustration purposes we manually compute the test statistic in R. Recall the equation for the test statistic from above:
\[\chi^2=\sum \frac{(O-E)^2}{E}\]
# compute the value of the test statistic
x2 <- sum((data$observed_frequencies - data$expected_frequencies)^2 / data$expected_frequencies)
x2## [1] 61.24772The numerical value of the test statistic is \(\approx 61.25\).
In order to calculate the p-value we apply the
pchisq() function. Recall, how to calculate the degrees of
freedom:
\[df = (c - 1)\]
# compute df
df <- nrow(data) - 1
# compute the p-value
p <- pchisq(q = x2, df = df, lower.tail = FALSE)
p## [1] 1.585774e-12\(p = 1.5857736\times 10^{-12}\).
Step 5: If \(p \le \alpha\), reject \(H_0\); otherwise, do not reject \(H_0\)
p <= alpha## [1] TRUEThe p-value is smaller than the specified significance level of 0.01; we reject \(H_0\). The test results are statistically significant at the 1 % level and provide very strong evidence against the null hypothesis.
Step 6: Interpret the result of the hypothesis test
At the 1 % significance level the data provides very strong evidence to conclude, that the religion distribution among students differs from the religion distribution of the population of the European Union.
Hypothesis Testing in R
We just manually completed a \(\chi^2\) goodness-of-fit test in R. Very cool, but now we redo that example and make use of the R machinery to obtain the same result as above in just one line of code!
In order to conduct a \(\chi^2\)
goodness-of-fit test in R we apply the chisq.test()
function. We provide two vectors as input data:
data$observed_frequencies and
data$expected_frequencies.
chisq.test(data$observed_frequencies, p = data$relative_frequency)##
## Chi-squared test for given probabilities
##
## data: data$observed_frequencies
## X-squared = 61.248, df = 4, p-value = 1.586e-12Worked out fine! Compare the output of the chisq.test()
function with our result above. Again, we may conclude that at the 1 %
significance level the data provides very strong evidence to conclude,
that the religion distribution among students differs from the religion
distribution of the population of the European Union.
Exercise: With his famous pea plant experiments Augustinian monk Gregor Mendel discovered the inheritance law of recessive and dominant traits in genes. His results show a 1:3 ratio of green to yellow peas from cross-bred seeds. Assume we repeated his experiment and got 123 green and 355 yellow pea plants. Does our observation confirm Mendel’s inheritance law? Perform a test with 95 % significance level!
### your code hereShow code
chisq.test(c(123, 355), p = c(0.25, 0.75))##
## Chi-squared test for given probabilities
##
## data: c(123, 355)
## X-squared = 0.13668, df = 1, p-value = 0.7116
Citation
The E-Learning project SOGA-R was developed at the Department of Earth Sciences by Kai Hartmann, Joachim Krois and Annette Rudolph. You can reach us via mail by soga[at]zedat.fu-berlin.de.
You may use this project freely under the Creative Commons Attribution-ShareAlike 4.0 International License.
Please cite as follow: Hartmann, K., Krois, J., Rudolph, A. (2023): Statistics and Geodata Analysis using R (SOGA-R). Department of Earth Sciences, Freie Universitaet Berlin.