Skip to main content
Resource

Six Rules of Thumb for Determining Sample Size and Statistical Power

About

Presents guidelines and useful diagrams to explain the relationship between sample size and statistical power to better support policymakers and researchers interested in conducting random assignment studies. “The ability of an evaluation to detect a meaningful impact of a program is determined by the evaluation’s sample size and statistical power. This is a tool for policymakers and practitioners that describes some of the factors that affect statistical power and sample size.”

The statistical power, or power, of an evaluation reflects the likelihood of detecting any meaningful changes in an outcome of interest brought about by a successful program. In the process of designing a randomized evaluation, researchers conduct power analyses to inform decisions such as:

  • Whether to conduct the evaluation
  • At which unit to randomize (e.g., individual, household, or group)
  • How many units to randomize
  • How many units or individuals to survey
  • How many times to survey each unit or individual over the course of the evaluation
  • How many different program alternatives to test
  • How much baseline information to collect
  • Which outcomes to measure
  • How to measure the outcomes of interest

It is important to understand how the factors above are interrelated and affect the overall power and sample size needed for a randomized evaluation. The rules of thumb outline the key relationships between the determinants of statistical power and sample size and demonstrate how to design a high-powered randomized evaluation” (p.2).

MAJOR FINDINGS & RECOMMENDATIONS

The tool outlines the following six rules of thumb:

  • “Rule of thumb #1: a larger sample increases the statistical power of the evaluation… Larger samples are more likely to be representative of the original population…and are more likely to capture impacts that would occur in the population.” (p.4).
  • “Rule of thumb #2: if the effect size of a program is small, the evaluation needs a larger sample to achieve a given level of power. The effect size of an intervention is the magnitude of the impact of the intervention on a particular outcome of interest.” (p.4)

Materials

Comments

Content Details

Topics:

Target Populations:

Programs:

Geographic Locations:

Industry Sectors:

  • Last Updated:
  • Created:
  • Resource Publication Date: 2022
  • Posted by: Justin Venneri
  • Posted in: Evaluation and Research Hub

Error

An error has occurred. The details of this error have been forwarded to our technical team. If you continue to experience issues, please contact our support team at support@workforcegps.org.

Event has ended