Confidence Intervals

How certain are we? Quantifying uncertainty

← Module 3: Sampling & CLT Module 4 of 8 Module 5: Hypothesis Testing →

⏳ Loading R... (first load takes ~15 seconds)

📌 Before You Start

What you need: Module 3 (CLT and standard error) completed. Understanding of the normal distribution.

What you’ll learn: What a confidence interval actually means. How to calculate one manually and with t.test(). How confidence level affects interval width. The correct — and incorrect — ways to interpret a CI.

📖 The Concept: Confidence Intervals

A confidence interval (CI) gives a range of plausible values for a population parameter (like μ). It pairs our best estimate with a measure of uncertainty.

A 95% confidence interval means: if we repeated this study many times and built a CI each time, about 95% of those intervals would contain the true population parameter.

Wider CI = less precision, more uncertainty (small n, high variability, or high confidence level)
Narrower CI = more precision (larger n, lower variability)
The t-distribution is used instead of the normal when we don’t know the population SD (which is almost always in practice)
The t* critical value depends on the confidence level and degrees of freedom (df = n − 1)

🔢 The Formula

CI = x̄ ± t* × (s / √n)

x̄ = sample mean | t* = critical value | s = sample SD | n = sample size

t* = qt(1 - α/2, df = n - 1)

For 95% CI: α = 0.05, so qt(0.975, df=n-1)

💻 In R — Worked Example (read-only)

Two ways to get a confidence interval — manual calculation and the quick way with t.test(). Both give the same result.

# Confidence intervals in R set.seed(42) sample_data <- rnorm(25, mean=70, sd=12) # Method 1: Manual calculation n <- length(sample_data) x_bar <- mean(sample_data) s <- sd(sample_data) se <- s / sqrt(n) t_star <- qt(0.975, df=n-1) # 95% CI, two-tailed lower <- x_bar - t_star * se upper <- x_bar + t_star * se cat(sprintf("95%% CI (manual): (%.2f, %.2f)\n", lower, upper)) # Method 2: Using t.test (easier!) result <- t.test(sample_data) cat("95% CI (t.test): (", round(result$conf.int[1],2), ",", round(result$conf.int[2],2), ")\n") cat("Same result both ways!\n")

⚠️ Common Misconception

A 95% CI does NOT mean "there is a 95% probability the true mean is in this interval." The true mean is either in the interval or it isn’t — probability doesn’t apply to a fixed (unknown) constant. The correct interpretation: the method produces intervals that contain the true mean 95% of the time across many repetitions.

🖐️ Your Turn

Exercise 1 — Three Confidence Levels

A sample of 20 students has mean = 78 and SD = 9. Calculate 90%, 95%, and 99% confidence intervals. Notice how the interval widens as you demand more confidence.

n <- 20
x_bar <- 78
s <- 9
se <- s / sqrt(n)

cat("Sample: n =", n, "| mean =", x_bar, "| SD =", s, "\n")
cat("Standard Error:", round(se, 3), "\n\n")

# Calculate CIs for three confidence levels
for (conf_level in c(0.90, 0.95, 0.99)) {
  alpha <- 1 - conf_level
  t_star <- qt(1 - alpha/2, df = n - 1)
  lower <- x_bar - t_star * se
  upper <- x_bar + t_star * se
  width <- upper - lower
  cat(sprintf("%d%% CI: (%.2f, %.2f)  width = %.2f  t* = %.3f\n",
              round(conf_level * 100), lower, upper, width, t_star))
}

cat("\nKey observation: Higher confidence = wider interval.\n")
cat("More certainty always comes at the cost of precision.\n")

Output will appear here...

💡 What to notice: The 99% CI is wider than the 95% CI, which is wider than the 90% CI. You pay for higher confidence with a less precise interval.

Exercise 2 — Simulate 50 Confidence Intervals

Generate 50 random samples of n = 30 from a Normal(70, 10) population. Build a 95% CI for each. Count how many intervals contain the true mean of 70. It should be close to 47–48 (95% of 50).

set.seed(2024)
true_mean <- 70
true_sd   <- 10
n         <- 30
n_sims    <- 50

# Storage
lowers <- numeric(n_sims)
uppers <- numeric(n_sims)

for (i in 1:n_sims) {
  samp <- rnorm(n, mean = true_mean, sd = true_sd)
  result <- t.test(samp, conf.level = 0.95)
  lowers[i] <- result$conf.int[1]
  uppers[i] <- result$conf.int[2]
}

# Count how many intervals contain the true mean
contains_true <- (lowers <= true_mean) & (uppers >= true_mean)
n_contain <- sum(contains_true)
cat("Number of CIs that contain true mean (70):", n_contain, "out of", n_sims, "\n")
cat("That's", round(n_contain/n_sims * 100, 1), "% — should be close to 95%\n\n")

# Visualize: plot all 50 CIs
plot(NULL, xlim = c(min(lowers)-1, max(uppers)+1), ylim = c(0, n_sims+1),
     xlab = "Value", ylab = "Simulation #",
     main = "50 Confidence Intervals (red = misses true mean)")
abline(v = true_mean, col = "#004D40", lwd = 2, lty = 2)

for (i in 1:n_sims) {
  col <- ifelse(contains_true[i], "#80CBC4", "#C62828")
  segments(lowers[i], i, uppers[i], i, col = col, lwd = 1.5)
  points(c(lowers[i], uppers[i]), c(i, i), col = col, pch = 19, cex = 0.4)
}
legend("topright", legend = c("Contains true mean", "Misses true mean"),
       col = c("#80CBC4", "#C62828"), lwd = 2)

Output will appear here...

💡 This is the real meaning of "95% CI." The red intervals in the plot are the ones that missed. With 50 simulations at 95% confidence, you expect about 2–3 misses.

Exercise 3 — Interpret Three Real CIs

Plain-language interpretation practice. Three studies report confidence intervals. For each, interpret what the CI tells us — and what it does NOT tell us.

cat("=== Interpreting Confidence Intervals ===\n\n")

cat("Study 1: Average daily sleep\n")
cat("95% CI: (6.8, 7.4) hours\n")
cat("Interpretation: We are 95% confident that the true population\n")
cat("mean sleep time is between 6.8 and 7.4 hours per day.\n")
cat("The interval does NOT contain 0, so this is a meaningful estimate.\n\n")

cat("Study 2: Effect of a new treatment on blood pressure\n")
cat("95% CI: (-2.3, 8.7) mmHg reduction\n")
cat("Interpretation: The interval CONTAINS 0. This means the treatment\n")
cat("could have no effect, a small negative effect, or a positive effect.\n")
cat("With this CI, we cannot conclude the treatment is effective.\n\n")

cat("Study 3: Proportion voting 'Yes' on a ballot measure\n")
cat("95% CI: (0.51, 0.58)\n")
cat("Interpretation: We are 95% confident the true proportion is between\n")
cat("51% and 58% — entirely above 0.50, suggesting majority support.\n")
cat("The entire interval is above 0.5, so we have evidence of majority support.\n\n")

cat("--- General rule ---\n")
cat("If CI contains 0: cannot claim effect is non-zero.\n")
cat("If CI is entirely positive or negative: evidence of a real effect.")

Output will appear here...

🧠 Brain Break

A confidence interval is not about the probability that the parameter is inside — it’s about how often the method works.

Quick check: If you want a narrower CI without changing the confidence level, what would you do? (Collect a larger sample! SE = s/√n decreases as n increases.)

✅ Key Takeaway

A 95% CI means the method captures the true parameter 95% of the time — not that there’s a 95% chance the parameter is in this specific interval. Use t.test() in R for confidence intervals. Wider CI = lower precision; narrower CI = higher precision.

🏆 Module 4 Complete!

You can now build and correctly interpret confidence intervals — one of the most commonly misunderstood concepts in all of statistics. Next up: the closely related idea of hypothesis testing.

Continue to Module 5: Hypothesis Testing →

← Module 3: Sampling & CLT Module 4 of 8 Module 5: Hypothesis Testing →