"How often? Nieuwenhuis looked at 513 papers published in five
prestigious neuroscience journals over two years. In half the 157
studies where this error could have been made, it was. They broadened
their search to 120 cellular and molecular articles in Nature
Neuroscience, during 2009 and 2010: they found 25 studies committing
this fallacy, and not one single paper analysed differences in effect
sizes correctly."
I wonder in how many other fields of research this has happened?
This is extraordinary. I wonder if the simple solution is good software.
Aftter all, they're all just using numbers to work this sort of stuff
out. If the software they used ensured it outputted the right figures
and got the science spot on then perhaps this sort of error wouldn't happen.
Of course if the software writers got it wrong or the current
statistical science was incorrect then the error in research would be
far greater. an opensource-model project on the statistical science
required to say something works and something doesn't - with
contributions from inside and outside academia - might be the way to get
the science right and build software which outputs stuff which makes
sense to anyone as well as gets the science spot on.
Clearly the editors at these peer reviewed journals didn't spot the
errors. Even worse - neither did the readers. This is the whole point of
peer review. When something goes wrong the people who read the journal
are meant to spot it and correct it.
This paper and article is a bad one for the robustness of the peer
review process. This is a big science balls up and it seems no one
spotted it through the conventional peer review process. As I've already
asked, how many other peer reviewed journals are letting these same
errors slsip by?
No comments:
Post a Comment