Hypothesis Testing
Learn how statisticians use sample data to evaluate claims about a population and make evidence-based statistical decisions.
What Is Hypothesis Testing?
Hypothesis testing is a statistical framework for evaluating a claim about a population using sample data.
We begin with a statement about the population, collect sample data, measure how compatible the data are with that statement, and then make a statistical decision according to a predefined rule.
Claim → Sample Data → Test Statistic → Evidence → Decision
What You Will Learn
What hypothesis testing means
Why a null hypothesis is needed
The role of the alternative hypothesis
How sample evidence is measured
What a test statistic represents
How statistical decisions are made
Reject vs fail to reject the null hypothesis
Real-world analytics applications
Start With a Population Claim
Imagine an e-commerce company claims that its average delivery time is 30 minutes.
Company claim
μ = 30 minutes
We collect a sample of delivery times and ask whether the observed sample provides enough evidence to challenge the claim.
The Basic Hypothesis Testing Process
State
Set up the hypotheses
Collect
Gather sample data
Calculate
Compute a test statistic
Evaluate
Measure the evidence
Decide
Make a statistical decision
The Null Hypothesis
The null hypothesis, written as H₀, represents the baseline claim that the statistical test evaluates.
Example
H₀: μ = 30
The population mean delivery time is 30 minutes.
Important:
The null hypothesis is the reference point against which the sample evidence is evaluated.
The Alternative Hypothesis
The alternative hypothesis, written as H₁ or Hₐ, represents the competing claim that the data may provide evidence for.
Two-sided
Hₐ: μ ≠ 30
Tests whether the mean is different from 30.
Greater
Hₐ: μ > 30
Tests whether the mean is greater than 30.
Less
Hₐ: μ < 30
Tests whether the mean is less than 30.
Sample Data Provide the Evidence
Suppose the company claims that average delivery time is 30 minutes.
Claimed mean
30 min
Sample size
100
Sample mean
32 min
Sample SD
8 min
The sample mean is 2 minutes above the claimed value. The important question is whether that difference is large relative to the sampling variability we would expect if the null hypothesis were true.
Test Statistic
A test statistic measures how far the observed sample result is from the value specified by the null hypothesis, relative to its standard error.
Test Statistic =(Observed − Null Value)/Standard Error
Example using the sample above
SE = 8 / √100 = 0.8
t = (32 − 30) / 0.8 = 2.5
The observed sample mean is 2.5 standard errors above the null value.
Statistical Decision
Hypothesis testing does not prove that a hypothesis is absolutely true or false. Instead, we use a predefined statistical rule to decide whether the sample provides sufficient evidence against the null hypothesis.
Reject H₀
The sample provides sufficient evidence against the null hypothesis according to the chosen significance level and test.
Fail to Reject H₀
The sample does not provide sufficient evidence against the null hypothesis according to the chosen statistical rule.
Important distinction:
“Fail to reject H₀” does not mean that H₀ has been proven true. It means the available sample evidence was not sufficient to reject it under the chosen procedure.
A Complete Hypothesis Testing Workflow
1. Define the population question.
2. State H₀ and Hₐ before examining the result.
3. Choose an appropriate statistical test.
4. Set the significance level α.
5. Collect and analyze the sample data.
6. Calculate the test statistic.
7. Evaluate the evidence using a p-value or critical-value rule.
8. Reject or fail to reject H₀.
9. State the conclusion in the context of the original question.
Website Performance Example
A company claims that its average website response time is 500 milliseconds.
Null value
500 ms
Sample size
100
Sample mean
520 ms
Sample SD
80 ms
H₀: μ = 500
Hₐ: μ ≠ 500
SE = 80 / √100 = 8 ms
t = (520 − 500) / 8 = 2.5
The test statistic measures the observed difference relative to the sampling variability expected under the null model.
Test Your Understanding
CHECK YOUR UNDERSTANDING
What is the main purpose of hypothesis testing?
CHECK YOUR UNDERSTANDING
What does H₀ represent?
CHECK YOUR UNDERSTANDING
Which statement represents a two-sided alternative hypothesis for a population mean of 30?
CHECK YOUR UNDERSTANDING
What does a test statistic measure?
CHECK YOUR UNDERSTANDING
What does 'fail to reject H₀' mean?
CHECK YOUR UNDERSTANDING
If the observed result is far from the null value relative to its standard error, what generally happens to the strength of evidence against H₀?
Fill in the Blanks
The symbol H₀ represents the ______ hypothesis.
The competing hypothesis is called the ______ hypothesis.
A test statistic compares the observed result with the null value relative to its standard ______.
If the evidence is insufficient against H₀, we ______ to reject H₀.
Delivery Time Claim
A company claims that average delivery time is 30 minutes. A random sample of 100 deliveries has a mean of 32 minutes and a standard deviation of 8 minutes.
Set up the hypotheses for testing whether the mean is different from 30 minutes.
H₀: μ = 30
Hₐ: μ ≠ 30
Calculate the test statistic.
SE = 8 / √100 = 0.8
t = (32 − 30) / 0.8 = 2.5
The final statistical decision requires a specified significance level and the appropriate reference distribution.
Final Challenge
A website claims that its average response time is 500 ms. An analyst obtains a sample mean of 520 ms with a standard error of 8 ms.
Test Statistic = (520 − 500) / 8
= 2.5
The result is 2.5 standard errors above the null value. Whether this is sufficient evidence to reject H₀ depends on the chosen significance level and the appropriate statistical test.
Lesson Summary
Hypothesis testing evaluates a population claim using sample evidence.
H₀ is the null hypothesis and provides the baseline claim.
Hₐ or H₁ is the alternative hypothesis.
The test statistic measures the observed difference from the null value relative to sampling variability.
A statistical decision is made using a predefined rule.
Rejecting H₀ means the evidence is sufficient against H₀ under the chosen procedure.
Failing to reject H₀ does not prove that H₀ is true.
The conclusion should always be stated in the context of the original analytics question.