Statistical Significance and Hypothesis Testing

Inferential Statistics

Inferential statistics allows one to draw conclusions about the population based on sample data. Unlike descriptive statistics, which simply describes the collected data, inferential statistics enables one to generalize results.

Hypotheses

Null Hypothesis (H₀)

States that there is no significant difference, relationship, or effect. This is the "status quo" hypothesis that we attempt to disprove.

Example: H₀: There is no statistically significant difference in employee satisfaction between office and remote workers.

Alternative Hypothesis (H₁ or Hₐ)

States that a difference, relationship, or effect exists. This is the hypothesis we aim to confirm.

Example: H₁: There is a statistically significant difference in employee satisfaction between office and remote workers.

Statistical Significance

p-value — this is the probability of obtaining the observed result (or a more extreme one), if the null hypothesis is true.

Interpretation of p-value:

p < 0.05 — the result is considered statistically significant (standard threshold). The null hypothesis is rejected.
p < 0.01 — the result is highly significant
p < 0.001 — the result is very highly significant
p > 0.05 — the result is not significant. There is no reason to reject the null hypothesis.

Important: Statistical significance ≠ practical significance. A very small difference may be statistically significant with a large sample size, but have no practical importance.

Errors in Hypothesis Testing

Type I Error — rejection of a true null hypothesis (false positive result). Probability = α (usually 0.05). Example: Concluding that the training increased productivity when in fact it had no effect.

Type II Error — failure to reject a false null hypothesis (false negative result). Probability = β. Example: Concluding that the training had no effect when in reality it increased productivity.

Confidence Intervals

Confidence interval — this is a range of values which, with a certain probability, contains the true value of the population parameter.

95% confidence interval means: if the study is repeated 100 times, the true value will fall within the calculated interval in about 95 cases.

Example: Mean satisfaction = 3.8 (95% CI: 3.5 – 4.1). We are 95% confident that the true mean satisfaction in the population is between 3.5 and 4.1.

Main Statistical Tests

t-test

Compares the means of two groups.

Independent samples t-test — comparison of means of two independent groups. Example: Comparing the average satisfaction of men and women.

Paired samples t-test — comparison of means for one group under two conditions. Example: Comparing employee satisfaction before and after training.

ANOVA (Analysis of Variance)

Compares the means of three or more groups. Example: Comparing satisfaction between three departments (marketing, finance, IT).

If ANOVA shows a significant difference (p < 0.05), it means that at least one pair of groups differs significantly
Post-hoc tests (Tukey, Bonferroni) are used to determine which groups specifically differ

Correlation

Measures the strength and direction of the linear relationship between two variables.

Pearson's correlation coefficient (r):

r = +1: perfect positive correlation
r = 0: no linear relationship
r = −1: perfect negative correlation

Interpretation of correlation strength (according to Cohen):

|r| < 0.3 — weak
0.3 ≤ |r| < 0.5 — moderate
|r| ≥ 0.5 — strong

Important: Correlation does not imply causation!

Regression Analysis

Models the relationship between a dependent variable and one or several independent variables.

Simple linear regression: Y = a + bX

Y — dependent variable
X — independent variable
a — intercept
b — regression coefficient (slope)

Multiple regression: Y = a + b₁X₁ + b₂X₂ + ... + bₙXₙ

R² (coefficient of determination) — proportion of the variance in the dependent variable explained by the model. R² = 0.45 means the model explains 45% of the variation.

Practical Assignments

Assignment 1

Question: The researcher compared job satisfaction in two groups: office workers (n=80, M=3.9, SD=0.8) and remote workers (n=70, M=4.3, SD=0.7). The independent samples t-test showed: t(148) = -3.24, p = 0.001. Interpret the results.

Solution:

p = 0.001 < 0.05 — the result is statistically significant. We reject the null hypothesis.
There is a statistically significant difference in satisfaction between office and remote workers
Remote workers (M=4.3) are significantly more satisfied than office workers (M=3.9)
The difference in means = 0.4 points on a 5-point scale
t = -3.24 (negative because the first group has a lower mean)
df = 148 (degrees of freedom ≈ n₁ + n₂ − 2)
Limitation: A correlational design does not allow a claim that remote work causes higher satisfaction (self-selection is possible)

Assignment 2

Question: The correlation coefficient between salary and job satisfaction was r = 0.35, p = 0.002. Can it be asserted that higher salary causes greater satisfaction?

Solution: No, causality cannot be asserted:

r = 0.35 indicates a moderate positive correlation (according to Cohen's classification)
p = 0.002 < 0.05 — the relationship is statistically significant
However, correlation ≠ causation. Possible explanations:
- Higher salary → higher satisfaction (direct causal link)
- Higher satisfaction → better work → salary increase (reverse causality)
- A third variable (for example, education level) affects both variables (confounding variable)
To establish a causal relationship, experimental design with variable control is necessary