An apparently commonly cited axiom in evidence-based medical science and statistics is “Cochrane’s Aphorism”, named for Archie Cochrane.
Cochrane’s Aphorism states:
“Before ordering a test, decide what you will do if it is (1) positive or (2) negative. If both answers are the same, don’t take the
test.”–Archie Cochrane
This quote is probably best known among doctors and nurses as an aid in making decisions about ordering laboratory tests for additional evidence, which published studies have shown is prone to overuse, redundancy, and sometimes also egregious errors. However, people also tend to use this aphorism when speaking about experimental design and analysis in the sciences. In particular, the Aphorism is popular when scientists urge parsimonious use of statistical tests, to avoid errors with multiple hypotheses testing, and when differentiating between the value and usefulness of an analysis. For example, I have heard the author of Snow’s penultimate test for normal distribution of a variable refer to this one routinely.
In scientific research and writing, I feel it is always good to stick to basic principles of good study design and writing organization
and style, and to be skeptical about adding more methods to an analysis for a manuscript. Yet, when doing friendly reads or reviewing manuscripts for journals in my fields of evolutionary biology and molecular ecology, I find that I sometimes encounter unnecessary statistical tests or analyses that don’t add much value to a manuscript. Of course, I’ll also admit that earlier in my career I too learned this the hard way myself, during reviews of my papers. So, I think we molecular ecologists would do well to remind ourselves of Cochrane’s Aphorism every time we 1) conduct a study or 2) write one up. Specifically, we can re-phrase Cochrane’s Aphorism for Molecular Ecologists to state,
“Before conducting a new analysis or statistical test, decide what you will do if it (1) rejects the hypothesis or (2) fails to reject the hypothesis. If both answers are the same, don’t add the analysis.”
Put another way with a focus on the actual writing of a paper, Cochrane’s Aphorism for Molecular Ecology Papers might be as follows:
1. “Before adding an analysis to your paper, decide whether it will help you (1) test your assumptions or hypotheses (2) or not. If the answer to is “not”, then don’t add the analysis.” OR: 2. “Before adding an analysis to your paper, decide what you will do if it is (1) changes your story or (2) doesn’t change your story. If both answers are the same, don’t add the analysis.”
I think that some, or perhaps many, people already think this way, and their papers are likely the better off for it. And some have already blogged on this topic, but not made it specific to any field I work in. For example, on applying Cochrane’s Aphorism to science in general, CheckmateScientist quipped, ”Much time, money, and distress would be saved in any field of research if only every PhD was taught this during their first month of work.” And I would have to say that I agree wholeheartedly.
~J