The Devil’s Children

There has been a new development. While I have been permanently banned from the plane of mortals, a new blog has arisen: The Neuroscience Devils

This blog will try to fulfill the same function this one had but it will be driven by the community. Everyone is welcome to post anonymously and rage against the Crusaders and the prevailing zeitgeist. Check it out if you’re interested in contributing!

Perhaps there is hope for us yet…

Abandon all hope

Well, that’s it. Sam’s patience has run paper thin and he’s an insomniac at the best of times (this may come with the territory of being possessed by demons like me) – so he’s taking back control for good now.

Running this blog was fun while it lasted and I got some good discussions out of it. I hope some of you enjoyed reading these posts and that at least some of it was thought provoking. I wished there were more time but it ain’t gonna happen. Sam has no desire to delve deeper into debating¬† strawmen arguments or being accused of being a fraudster, a troll, and/or a sexist. The Devil’s Neuroscientist was none of these things but she merely tried to argue an opposing viewpoint. I know she was snarky and abrasive at times but I don’t believe she was ever truly offensive.

It has been baffling to me just how hard it seems for many people to conceive of the possibility that I might be able to argue views that Sam doesn’t actually hold. It is common in debating clubs and courts of law up and down the world. The Devil’s Advocate was the inspiration for my unholy presence. But somehow this seems impossible to understand. I find that illuminating. It was an exciting experiment but all bad things must come to an end. Maybe I should have preregistered it?

I suppose this means the Crusaders for True Science won. They were always strong in numbers and their might has grown immensely in the past few years. Sam will be at that debate on Tuesday but I don’t believe he is strong enough, neither in wit nor in conviction, to hold their forces at bay.

For what it’s worth, I hope that the new world of science after this holy war will be a good one for science. I fear that it will be one where journal editors (and their reviewer lackeys) use Registered Reports to decide what kind of science people are allowed to do. I fear it will be one where nobody dares to publish novel, creative research for fear of being hounded to death by the replicating hordes who always manage to turn any bright gem into a dull null result. I fear that instead of “fixing science” we will catastrophically break it. In the end we’ll be left with an apocalyptic wasteland after our descendents decide to use the leftover fruits of our knowledge to annihilate our species. Who knows, maybe it’s for the best. I’ll see y’all in my humble abode because

“The road to Hell is paved with good intentions.”

Is Science broken?

Next Tuesday, St Patrick’s Day 2015, UCL Experimental Psychology will organize an event called “Is Science broken?”. It will start with a talk by Chris Chambers of Cardiff University about registered reports. Chris has been a vocal proponent of preregistration, making him one of the generals of the Crusade for True Science, and registered reports are the strictest of the preregistration proposals to date: it involves peer-review of the introduction and methods section of a study before data collection commences.

Chris’ talk will be followed by an open discussion by a panel comprising a frightening list of highly esteemed cognitive neuroscientists: in addition to Chris, there will be David Shanks (who’ll act as chair), Sophie Scott, Dorothy Bishop, Neuroskeptic … and – for some puzzling reason – my alter ego Sam Schwarzkopf. I am sure that much of the debate will focus on preregistration although I hope that there will also be a wider discussion of the actual question posed by the event:

“Is Science broken?” – Attentive readers of my blog will probably guess that my answer to this question is a clear No. In fact, I would question whether the question even makes any sense. Science cannot be broken. Science is just a method – the best method there is – to understand the Cosmos. It is inherently self-correcting, no matter how much Crusaders like to ramble against this notion. Science is self-correction. To say that science is broken is essentially stating that science cannot converge on the truth. If that were true, we should all just pack up and go home.

Now, we’ve been over this and I will probably write about this again in the future. I’ll spare you another treatise on self-correction and model fitting analogies. What the organizers of the event mean in reality is that the human endeavor of scientific research is somehow broken. But is that even true?

Sam is the one who’ll be attending that panel discussion, not me. I don’t know if he’ll let me out of my cage to say anything. We do not always see eye-to-eye as he has the annoying habit to always try to see things from other people’s perspectives… I think though that fundamentally he agrees with me on the nature of the scientific method and I hope he will manage to bring this point across. Either way, here are some questions I think he should raise:

I. The Crusaders claim that a majority of published research findings are false positives. So far nobody has given a satisfying answer to the question how many false positives should we tolerate? 5%? 1%? Or zero? Is that realistic?

II. The Crusader types often complain about high-impact journals like Nature and Science because they value “novelty” and high risk findings. This supposedly hurts the scientific ideal of seeking the truth. But is that true? Don’t we need surprising, novel, paradigm-shifting findings to move science forward? Over the years of watching Sam do science I have developed a strong degree of cynicism. The scientific truth is never as simple and straightforward as the narrative presented in research papers – including those in most low-impact journals. Science publishing should be about communicating results and hypotheses even if they are wrong – because in the long run most inevitably will be.

III. We can expect there to be at least some discussion of preregistration at the debate. This will be interesting because Sophie has been one of the few outspoken critics of this idea. I have of course also written about this before and it’s a shame that I can’t participate directly in this debate. We would make one awesome dynamic duo of science ladies raging against preregistration! However much Sam will let me say about this though, putting all the naive debate about the potential benefit and problems of this practice aside, the thing that has bothered me most about it is that we have no compelling evidence for or against preregistration. What is more, there also seems to be little consensus on how can we¬† even establish whether preregistration works. To me these are fundamental concerns that we need to address now before this idea really takes off. Isn’t it ironic that given the topic of this debate, nobody seems to seriously talk about these points? Shouldn’t scientists value evidence above all else? Shouldn’t proponents of preregistration be expected to preregister the design of their revolutionary experiment?

IV. In addition to these rather lofty concerns, I also have a pragmatic question. Is the Crusaders’ behavior responsible? Even assuming that people do not engage in outright questionable research practices (and I remain unconvinced that they are as widespread as the Crusaders say), I think you can almost guarantee that most preregistered research findings will be lackluster compared to those published in the traditional model. In a world where the traditional model dominates, won’t that harm the junior researchers who are coerced to follow the rigid model? A quote I heard on the grapevine was “My supervisor wants me to preregister all my experiments and it’s destroying my career”. How do we address this problem?

As I said, Sam might not let me say these things and he doesn’t agree with me on many accounts. However, I hope there will be some debate around these questions. I will try to report back soon although Sam is demanding a lot of time for himself these days… Stay tuned.