Learn Without Walls
← Previous LessonLesson 3 of 4Next Lesson →

Lesson 3: When Samples Go Wrong

About 30 minutes — Discussion-based lesson

What You Will Learn

This lesson covers:

Selection bias: your sample systematically excludes certain people

This section covers the key ideas about selection bias: your sample systematically excludes certain people. Discuss with your group or family and explore the concepts together.

Self-selection bias: only people with strong opinions respond

This section covers the key ideas about self-selection bias: only people with strong opinions respond. Discuss with your group or family and explore the concepts together.

Survivorship bias: you only see the successes, not the failures

This section covers the key ideas about survivorship bias: you only see the successes, not the failures. Discuss with your group or family and explore the concepts together.

Famous sampling failures: the 1936 Literary Digest poll

This section covers the key ideas about famous sampling failures: the 1936 literary digest poll. Discuss with your group or family and explore the concepts together.

Check Your Understanding

1. What is selection bias?

Answer: When your sampling method systematically excludes certain groups. Surveying only during business hours misses people who work during the day.

2. What is self-selection bias?

Answer: When only people who feel strongly about an issue choose to respond. Online reviews skew extreme because satisfied people rarely bother to write reviews.

3. What is survivorship bias?

Answer: When you only see the survivors/successes and not the failures. Successful companies get studied, but the many failed companies that tried the same strategies are invisible.

4. What happened with the 1936 Literary Digest poll?

Answer: They surveyed 2.4 million people but mostly by phone and car registration — which were luxuries during the Depression. Their sample was wealthy and unrepresentative, and they predicted the wrong winner.

Key Takeaways

Ready for More?

Next Lesson

Continue to Lesson 4: Evaluating Real Surveys

Start Lesson 4

Session Home

Review all lessons and materials.