Promoting Credibility in Neuroscience Research: The 3Rs at the BNA Festival of Neuroscience
Complete the form below to unlock access to ALL audio articles.
Every two years, a community of more than 1000 neuroscientists, with research interests ranging from the study of ion channels to human behavior, join in a cross-disciplinary neuroscience event hosted by the British Neuroscience Association (BNA). This 4-day event aims to disseminate information about key research going on in neuroscience via talks from more than 200 speakers, to engage the wider public in neuroscience via public events, and to encourage discussion and advancement of the quality of research methods used in neuroscience via workshops and plenaries.
This last aim was a focus of this year’s ‘Festival of Neuroscience’ in Dublin, where promoting credibility in neuroscience research was a key theme. While the term ‘credibility’ can mean slightly different things in different contexts, here it describes the extent to which research findings can be considered trustworthy or believable. Therefore, the credibility of research depends on the extent to which the data obtained can be relied upon to really represent what it is supposed to represent, and to support the conclusions drawn from the research. This focus on research credibility at the festival comes as part of the BNA’s new strategic plan to enhance openness and reproducibility in neuroscience research. The main take-away message from this ‘campaign for credibility’ at the festival was that we must place the ‘new 3Rs’ firmly at the foundation of all scientific inquiry.
The new 3Rs: reproducibility, replicability & reliability of research
Both as individual researchers, and as a wider community of neuroscientists, we must show more consideration of the ‘new 3Rs’ if we wish to restore faith in our research findings. This handy mnemonic is a play on the original 3Rs framework that was developed for doing more humane animal research relying on Replacement, Reduction and Refinement of animal models to do better, more conscientious science. The new 3Rs similarly describe 3 overlapping concepts which promote higher quality science, but this time, with a focus on ensuring that scientific findings are trustworthy. They are:
• Reproducibility – the ability of an entire experiment to be duplicated by an independent researcher. This relies on effective sharing of study data and detailed information about methods and statistical approaches to allow experiments or analysis to be exactly replicated.
• Replicability – the ability to obtain the same (or consistent) results when a study is repeated in a different context by a second researcher.
• Reliability – the degree to which our experiments produce stable and consistent results, which relies primarily on the trustworthiness of our experimental tools.
Uta Frith calls for ‘slow science’
Uta Frith, a renowned neurodevelopmental researcher and member of the BNA’s new ‘Credibility Advisory Board’, was the keynote speaker tasked with promoting the new 3Rs at the festival. She highlighted several specific danger points which undermine the role of the 3Rs in current neuroscience research, such as the practice of HARKing (Hypothesizing After the Results are Known), where researchers formulate their hypotheses after they have seen the results of the study, which increases the chance of false-positive results and inflated effect sizes. Such practices undermine the reliability of our research findings and may result in wasted time and resources carrying out further studies that seek to replicate these false-positive findings. Other threats to the 3Rs that were highlighted include the use of unreliable statistical methods and mistaking correlation for causation when interpreting study results.
While specific solutions have been suggested to deal with each of these danger points, Uta’s key proposition was that a major change to today’s prevailing highly competitive ‘publish or perish’ research culture is needed. The current research climate, where career progression and funding rely largely on ‘high-impact’ publications, encourages researchers to ‘cut corners’ in their effort to meet performance targets. Frith’s call was for a ‘Slow Science’, where quality and rigor of research is valued above traditional publication metrics. She insisted that we must slow down the scientific process so that we have time to discuss, think and fail, if we are to reach reliable scientific conclusions in our research. Notably, she emphasized the importance of collaboration to enable constant monitoring and discussion of ideas and research findings. This, Frith said, will allow us to reach reliable conclusions about the bigger picture, rather than individually persisting down ‘blind alleys’ of inquiry. This slow, steady, methodical science is more likely to yield applicable and trustworthy findings, but it is only possible in an environment where researchers can take their foot off the accelerator and take the time required to do quality science.
Credibility elsewhere at the Festival
Although Uta’s keynote speech carried the core message about credibility at the festival, the BNA made sure that ideas about research credibility were evident in many other places throughout the event. The ‘credibility zone’ was a permanent fixture in the main exhibition hall and was dedicated as an area for discussion about common practices that can affect the credibility of research, with the aim of making researchers more mindful of their own implicit biases and how these may influence different aspects of the design, analysis and interpretation of their studies.
There was also a special workshop on safeguarding credibility and reproducibility in neuroscience, raising issues like selective reporting of outcomes, how to improve preclinical practice to promote translation from bench to bedside, and how pre-registration can be used as a tool to minimize publication bias by ensuring that the studies which are published are those which seek to rigorously investigate questions of scientific importance, rather than those with ‘novel’ or ‘exciting’ findings. In line with this message about the benefits of pre-registration, the BNA introduced ‘pre-registration posters’ for the first time at this year’s festival, allowing individuals to present ideas about future and ongoing studies and receive feedback on their methods and hypotheses. Together, these different opportunities for discussion and consideration of research credibility represent the initial steps taken by the BNA in their new drive to secure credibility in neuroscience.
The message for the neuroscience community…
The idea of a ‘replication crisis’ extends beyond neuroscience into many different areas of social and biological science, but neuroscience is one field that seems to have come out particularly badly in the replication crisis, possibly, as Frith argues, because it is a relatively young science which lacks the secure framework of robust overarching theories to guide scientific inquiry. Uta points out that scientific progress is not possible without trust in scientific findings, which is why change in research practices is so desperately needed for the advancement of neuroscience. Research culture must change from being highly competitive, valuing quantity of novel or positive publications above all else, to being collaborative, with an emphasis on taking one’s time to do the best, most rigorous science possible.
Currently, the pressure to publish makes it tough for researchers, and in particular early-career researchers, who are seeking to establish themselves in the field, to engage in ‘slow science’. It is therefore extremely encouraging that a large organisation like the BNA, which has the power to affect change widely across many fields in neuroscience, is prepared to lead the effort for a change in research practices and wider research culture. However, if we are truly to transform our research culture to one that prioritizes rigorous science, it will be the responsibility of all individual neuroscience researchers to push for this change. Only a concerted effort will allow us to secure society’s trust in neuroscientific findings.
