Why do you think your paper is highly cited?

It is unsettling when a seemingly logical, widely touted method is suddenly declared inappropriate. Using power analysis to interpret statistical test results was advocated in more than 20 applied science journals and in some statistics texts but we showed this is inappropriate.

  Does it describe a new discovery or new methodology that’s useful to others?

We did not propose new methodology. Instead we showed how statistical practice in applied journals had gone astray and pointed to appropriate methodology. The paper is of interest to researchers because nobody likes to be caught using inappropriate methods.

  What were some of the circumstances that led you to do this research?

The paper came about because some journal editors insisted that inappropriate power analyses be done. This was an abuse of power we decided to fight.

  Could you summarize the significance of your paper in layman’s terms?

There is confusion among scientists and policy makers about how burden of proof relates to statistical tests. Consider the discharge of an industrial effluent into a river. The industrialist would say he or she should be allowed to discharge the effluent unless it can be shown to be harmful. He or she would adopt the null hypothesis that the effluent is safe and would stop the discharge only if the null hypothesis were rejected according to a statistical test. In this procedure, the chances of rejecting the null hypothesis when in fact the null hypothesis is true can be fixed at any desired level, say 5%. This error rate is called the Type 1 error.

A conservationist is likely to feel the burden of proof should be on the industrialist to prove the discharge is safe. The conservationist is concerned with the chances of failing to reject the industrialist's null hypothesis when the null hypothesis is in fact false (the effluent is in fact harmful). The chances of this kind of error, known as a Type 2 error, are unknown. Many people thought the chances of a type 2 error could be determined or estimated from the data using power analysis but this is not valid. Others thought, mistakenly, that power analysis would shed light on whether the null hypothesis was credible even if it couldn’t quantify the chances of an error.

So what should the conservationist do if power analysis is not appropriate for decision making? The conservationist can reverse the burden of proof by assuming the effluent is harmful unless it can be proven safe. The null hypothesis then becomes that the effluent is harmful. The chances of rejecting the null hypothesis (concluding the effluent is safe) when it is in fact true (the effluent is dangerous) can then be fixed at any desired level (such as 5%).