Making Neuroscience More Credible at BNA 2019

During our attendance at the recent BNA Festival of Neuroscience 2019 in Dublin, we noticed a booth quite different from all the others. A sign marked “Credibility Zone” hung above an open plan area, which had a discussion table placed in the center. Leaflets and guides on improving research quality were dotted around, and over the 3 days of the conference exhibition, the topic of research credibility was introduced and discussed by the BNA’s Credibility Advisory Board in casual chats around the table. Young researchers were able to come and learn what credibility in neuroscience was, and how it could improve their work.

However, not all young researchers were able to attend BNA 2019, so we pulled aside one of the credibility board – Dr Verena Heise, a postdoc in the Nuffield Department of Population Health in Oxford – to interrogate her about credibility in neuroscience, how she became involved in promoting it, and how other players in research can help advance the credibility cause. 

Ruairi Mackenzie (RM): What exactly is credibility in science?

Verena Heise (VH): Sure, so credibility is really all about producing the most reliable research evidence that you can possibly produce. I think the main issue that we’re facing at the moment is that we incentivize researchers to create lots of papers – we are really going for high quantity of papers and we’re incentivized to produce what’s called a high impact factor paper. But we’re not really incentivized to get the science right that we produce, and credibility is all about getting the science right from the start.  From how you develop your own research question, how you design your project, collect and analyze the data and then through to how you publish the project and research output such as data and materials.  So, partly it’s related to open science as well; open science practices are super important as part of that journey, but really it’s also back to basics in terms of good research practice and study design and just really focusing on the things around designing research studies that we’re not getting taught as well as we should be.

RM: How did you first become interested in credibility?

VH: I really got involved in reproducibility about two years ago, when I did a workshop on improving my own research practices. I then got together with a group of other interested people in Oxford who were really keen on developing that work and teaching other people how to improve their practices, but also I think we need a much bigger movement to change research culture and that’s what’s really driving me now, partly in my research but mostly in my lobbying interests.

RM: The BNA have been strongly promoting credibility at this year’s Festival of Neuroscience – could you tell us a bit about what’s been happening?

VH: I think the BNA has been really amazing in terms of a society that really helps with credibility, they’ve been really influential in the sense that they decided that this was something that they wanted to focus on and they now have a credibility advisory board. We get together every couple of months to talk about what BNA could potentially do and here at the festival we’ve had one seminar series where we had several speakers talk about different aspects of credibility; we’ve just had a great keynote lecture from Uta Frith on what she calls, the new three R’s, so it’s Replicability, Reproducibility and Reliability of research. So, on the one hand the BNA is putting on all these events but I think in the future we will also think about how the BNA can help with driving culture change and research as well.

RM: What other ways can journals get involved in promoting credibility?

VH: Yeah, that’s a very good question and I think the full answer would probably take an hour to go over. I think one of the things that really frustrates me about really high impact factor journals for example, is that they put the methods at the end or they put the methods online somewhere after the supplementary material where you really can’t find it.  So, I think one of the first things that really high impact factor journals can do is to put the methods section where it belongs. Because I think nowadays, we focus so much on the results; I think that’s not necessarily helpful. What we really need to focus on in our research is how we produce those results and how robust the results are, much more than simply putting a result out there that’s surprising, sexy and newsworthy but ultimately not robust.

Furthermore, just because we get a null result, doesn’t mean it’s useless.  So, I think in terms of what can journals do, publish null results, and think about registered reports. I think registered reports is a really cool journal format. It’s a manuscript format which basically means that you send in your paper and it’s going to be evaluated based on the scientific question and whether or not your study design is appropriate to answer it before you even start data collection.  I think that’s a great initiative because basically, your study gets peer reviewed at a stage where that review can still make a difference. You get really good feedback on your method section and you can change your project before you’ve actually started it and I think that’s a really great and valuable thing.

RM: For other researchers who are only just discovering these initiatives, what advice cans you give them into getting involved with credibility initiatives? 

VH: Absolutely! So my sort of pet project is to get early career researchers involved in reproducibility and I think that one of the easiest things to do is check out the UK Reproducibility Network, they have a website and soon they will have a list of their local network leads on there, too. In Oxford, for example, we have a local UK reproducibility network node and we’re there to help – so the idea is that anyone should just get in touch if they want to know more about what we’re doing and then look out for all the training events that are out there. I think if you don’t have an event or local group yet at your own University, the best thing that you can do is to get together with people who are also interested in the topic, get together with friends in your department, and maybe think about setting up a journal club and just read some papers.

Start very low key and then develop from there on – think about running your own training events! There are lots of training events that are put out there at the moment and we and others are thinking about putting our training events online, so that in the end, people can profit from this stuff, even if they’re not in Oxford or even if they’re not based in the UK.

If you are a young researcher, and would like to talk to Verena about credibility in research, drop her an email at: verena.heise@ndph.ox.ac.uk