IRMR (International Register of Meta-Research) is an open-access database of prospectively registered meta-research studies in the setting of healthcare.
Meta-research has been defined as ‘the study of research itself: its methods, reporting, reproducibility, evaluation and incentives.’ (1) As research on research, meta-research uses scientific methods to explore how research is done and interpreted, with the goal of verifying the validity of research techniques and improving the reliability of conclusions.
The key goals of IRMR are:
- To provide free access to submit records to a database of prospective meta-research protocols in order to reduce bias in meta-research.
- To provide free access to a database of protocols and subsequent findings to aid accessibility and communication of meta-research.
- To encourage transparency, consistency and collaboration when conducting and reporting meta-research.
The Meta-Research Innovation Center at Stanford (METRICS) has divided meta-research in to five focal areas (2), defining them as:
Methods – The phase of designing and conducting research. Rigorous research methods are at the heart of good science. These methods include everything from question formation to study design, conduct, statistical analysis and interpretation. Meta-researchers use both theoretical and empirical approaches to identify flaws and biases in the research enterprise, then develop and test methods to minimize them and define best practices.
Reporting – The phase of communicating researchFor research to be useful it needs to be communicated properly to scientists and others who use the results. This reporting is done through research papers, written by scientists, or by various forms of media that convey scientific results to broader audiences. Meta-researchers are developing standards for reporting results and experimenting with new means to improve communication to scientists, policy makers, patients, and the general public.
Evaluation – The phase of evaluating researchScientific quality is constantly evaluated in journal publications, funding decisions, academic promotions, industry investment, medical practice guidelines, standard setting, and policy decisions. Peer review has long served as the cornerstone of such evaluations, but it is widely acknowledged to be imperfect and inefficient. Meta-researchers across multiple disciplines are assessing its effectiveness and experimenting with new approaches to improve the assessment of scientific quality.
Reproducibility – The phase of verifying researchReplicating research results is central to scientific credibility. Meta-research has repeatedly shown that accurate reporting and sound peer-review do not by themselves guarantee the reproducibility of a scientific claim and that active replications of previous research are too rarely done. Meta-researchers are developing new ways to assess the reproducibility of findings, make experimental data available, and promote policies that can make verifying research more routine and more effective.
Incentives – The phase of rewarding researchResearch is a rewarding activity in itself, but funding, recognition, and career advancement are critical to a successful scientific career. If the scientific reward system is not aligned with the adoption of best scientific practices, such as study replication and publishing negative results, such practices will happen rarely. Changing the criteria and processes by which rewards are distributed in science can help ensure that the most reliable research is encouraged.
(1) Ioannidis JPA. Meta-research: Why research on research matters. PLoS Biol.
2018 Mar 13;16(3):e2005468. doi: 10.1371/journal.pbio.2005468
Georgina Imberger (Project Manager)
Martin Nguyen (IT Manager)
Georgina is a specialist Anaesthetist at Western Health in Melbourne, Australia, and a Senior Fellow at the Centre for Integrated Critical Care (CICC), University of Melbourne.
She has an interest in meta-research and completed her PhD with the Faculty of Health and Medical Sciences at the University of Copenhagen, at the Copenhagen Trial Unit, with the thesis: “Multiplicity and sparse data in systematic reviews of anaesthesiological interventions cause increased risk of random errors and lack of reliability of conclusions.”
Martin is a specialist Anaesthetist at Western Health in Melbourne, Australia, and a Senior Lecturer at the Centre of Integrated Critical Care (CICC), University of Melbourne. Martin has a passion for using technology to solve complex problems in healthcare. He is a co-founder of Corsonis, WeRoster and the Medical Pantry.
- Centre for Integrated Critical Care (CICC), Department of Medicine and Radiology, Faculty of Medicine, Dentistry & Health Sciences, University of Melbourne, Victoria, Australia
- Department of Anaesthesia and Pain Medicine, Western Health, Victoria, Australia.