What Sample Size for a Pilot Study?

CLAHRCs are all about developing and testing health service innovations so the CLAHRC WM Director was interested in a recent article on sample sizes for pilot studies.[1] Many of these end-points of a pilot study go under the heading ‘detecting unforeseen problems’. The authors suggest setting the minimum probability of the problem that should be detected and the alpha value depending on the seriousness of the problem for the future trial. To be honest, the CLAHRC WM Director was not highly impressed by the approach. “If the problem occurs more frequently than X, then I want to know a 95% probability of observing at least one instance”, does not make much sense to the context of study design. Why not just calculate the 95% (or 80%) confidence intervals (CIs) of probabilities at given levels. Investigators know that problems can occur, so detailing an instance of the problem is not interesting. It is not like detailing a case of election fraud in a sample of ballot stations. In such a context, the authors’ method, which is nearly equivalent to ‘the rule of three’, would be more useful. The rule of three, to remind you, calculates the 95% CI on zero instances of an event, given n observations by the equation 3/n. So if there were no cases of fraud in 60 polling stations the 95% CI is 3/60 = 0.05. Do others share the CLAHRC WM Director’s concern regarding the underlying logic behind this method in a pilot study?

— Richard Lilford, CLAHRC WM Director


  1. Veichtbauer W, Smits L, Kotz D, et al. A simple formula for the calculation of sample size in pilot studies. J Clin Epidemiol. 2015; 68: 1375-9.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s