Inferential Statistics • Lesson 19
Type I & Type II Errors + Statistical Power
Learn what can go wrong when making statistical decisions, how Type I and Type II errors differ, and how statistical power helps us detect real effects.
Lesson Progress
1/8
Concept 1
Type I vs Type II Error
Two Ways a Decision Can Be Wrong
Hypothesis testing uses sample data to make a decision about a population. Because samples contain randomness, the decision can sometimes be wrong. There are two fundamental error types.
Visual Explanation
Two possible statistical errors
The key is to compare the decision made by the test with what is actually true in the population.
Your Turn
A test rejects H₀ even though H₀ is actually true. What type of error is this?
Hint: Look at the combination: the null hypothesis is actually true, but the test rejected it.
Analyst Workflow
Think About Statistical Decisions This Way
State H₀
Define the baseline condition.
Collect Data
Use sample evidence.
Make Decision
Reject or fail to reject H₀.
Consider Errors
Ask what could be missed or falsely detected.
Quick Reference
Type I, Type II and Power
| Concept | Meaning | Symbol |
|---|---|---|
| Type I Error | Reject a true H₀ | α |
| Type II Error | Fail to reject a false H₀ | β |
| Statistical Power | Correctly reject H₀ when a specified alternative is true | 1 − β |
Real-World Analytics Example
A/B Testing
An e-commerce company tests whether a redesigned checkout changes the population conversion rate.
Type I Error
Concluding that the checkout design changes conversion when it actually does not.
Type II Error
Failing to detect a real change in the population conversion rate.
Power
The probability of detecting the specified real conversion-rate effect under the assumed conditions.
Factors That Affect Power
Sample Size
Larger samples generally provide more information and can increase power.
Effect Size
Larger true effects are generally easier to detect than smaller effects.
Variability
Lower variability generally makes a given effect easier to detect.
Significance Level
Increasing α generally increases power, but also increases the Type I error rate.
Important:
Increasing sample size does not automatically guarantee high power. Power also depends on effect size, variability, α, test design, and other assumptions.
Business Decision Thinking
Why Do These Errors Matter?
Statistical study design involves balancing the consequences of different errors. The appropriate balance depends on the real-world cost of false positives and false negatives.
False Positive Cost
What happens if we act as though an effect exists when it does not?
False Negative Cost
What happens if we miss a real effect?