Saturday, June 03, 2006

Errors in publication...

There's an interesting selection-effect in scientific publishing, in that it is much more likely that positive results are being published than negative ones. This seems to be enforced both from the research-end (are you going to put a lot of effort into publishing that you were looking for something and found nothing?) and the publishing side (peer-review is taken in certain disciplines to the point where there's a requirement for something to be there to be published for your paper to be accepted), and the fact that "nothing's there" is a valuable result is often overlooked.

You set out to study a certain effect. You spend some time and effort, but you simply don't see anything. What do you do? You can't necessarily rule out that there's something there, but your research certainly isn't sensitive to it.

This gets even more problematic, when the research in question isn't particularly high-profile, or maybe even questionable. "Should we really study how exactly drug-users kill themselves?" - Now if you had some breakthrough result, you could probably publish even if its on a somewhat unfashionable topic, but if your result is of the usual Hum-Hah quality (as is most cutting-edge research -- that's what makes it cutting-edge) then there's not much of a chance of publication.

There's a wealth of studies on these magnetic water-treatment thingies, for example, that you can buy and plug onto your water-pipe and they're supposed to keep your heater crud-free. Every other engineer that is worth his money has been intrigued by the glaringly obvious (and simple) experiment of putting one on their water-pipe but not the neighbor's (or something similarly straightforward). The circumstances are generally not particularly well-controlled (the neighbor has a slightly different model water-heater for example), since there's no research-money in it and so you do this kind of thing on the side. Then, after a couple years, you look into the heaters and usually there's some difference. Not less crud, but the crud is in a different place or has slightly different color. -probably- a result of the fact that you have copper-pipes and your neighbor has plastic. Then again: who knows. There's essentially a null-result, with the possibility of -maybe- something being there (then again, maybe not) and wwhatever result may be there is certainly deep in the noise.

And hence there's nothing published. End of story.

Even worse for obvious but unfruitful approaches. Lets say you want to study something and there's a way of looking at it that nobody seems to have done before and you set out to do it and you mess around for some weeks, moths, maybe even years and - you discover that there's a deep, subtle thinking error here, that prevents you from examining the thing in question in this particular way.

And you wonder: how many people have gone this wrong way to the dead end? In general, mistakes aren't being published - in general, someone will not write a paper detailing how they spent lots of time (and possibly money) going some route rust to discover that its an invalid approach.

And hence a year later, somewhere on the world, someone will make the same mistake again. People go into the dead end, but it is not considered worthwhile to put up a sign that says "dead end" to keep others from following you. "Dead end" is not publishable.



Post a Comment

<< Home