What is meta-research?

Meta-research has been defined as ‘the study of research itself: its methods, reporting, reproducibility, evaluation and incentives.’ (1) As research on research, meta-research uses scientific methods to explore how research is done and interpreted, with the goal of verifying the validity of research techniques and improving the reliability of conclusions.

 (1) Ioannidis JPA. Meta-research: Why research on research matters. PLoS Biol. 
2018 Mar 13;16(3):e2005468. doi: 10.1371/journal.pbio.2005468

How do I submit a protocol?

To register a protocol, you need to first sign up as a user of IRMR. Your login details will allow you to update your record if and when required. You then need to submit some basic information and upload the protocol itself. Once submitted, we will screen your submission to check that it fulfils our definition of a protocol for meta-research. When we have confirmed eligibility, the record will be permanently entered on to the register.

Please note that the register is currently not uploading attachments correctly. We hope to have this functionality repaired shortly. In the meantime, please feel free to submit the details of your protocol via the website and email us the full protocol. We will keep your protocol in a secure file and upload it to your record as soon as the functionality is resolved.

What are the focal areas of meta-research?

The Meta-Research Innovation Center at Stanford (METRICS) has divided meta-research in to five focal areas. (1)

When you register your protocol, you will be asked to choose the focal area that best describes your project.

The five focal areas are:

Methods – The phase of designing and conducting research

Rigorous research methods are at the heart of good science. These methods include everything from question formation to study design, conduct, statistical analysis and interpretation. Meta-researchers use both theoretical and empirical approaches to identify flaws and biases in the research enterprise, then develop and test methods to minimize them and define best practices.

Reporting – The phase of communicating research

For research to be useful it needs to be communicated properly to scientists and others who use the results. This reporting is done through research papers, written by scientists, or by various forms of media that convey scientific results to broader audiences. Meta-researchers are developing standards for reporting results and experimenting with new means to improve communication to scientists, policy makers, patients, and the general public.

Evaluation – The phase of evaluating research

Scientific quality is constantly evaluated in journal publications, funding decisions, academic promotions, industry investment, medical practice guidelines, standard setting, and policy decisions. Peer review has long served as the cornerstone of such evaluations, but it is widely acknowledged to be imperfect and inefficient. Meta-researchers across multiple disciplines are assessing its effectiveness and experimenting with new approaches to improve the assessment of scientific quality.

Reproducibility – The phase of verifying research

Replicating research results is central to scientific credibility. Meta-research has repeatedly shown that accurate reporting and sound peer-review do not by themselves guarantee the reproducibility of a scientific claim and that active replications of previous research are too rarely done. Meta-researchers are developing new ways to assess the reproducibility of findings, make experimental data available, and promote policies that can make verifying research more routine and more effective.

Incentives – The phase of rewarding research

Research is a rewarding activity in itself, but funding, recognition, and career advancement are critical to a successful scientific career. If the scientific reward system is not aligned with the adoption of best scientific practices, such as study replication and publishing negative results, such practices will happen rarely. Changing the criteria and processes by which rewards are distributed in science can help ensure that the most reliable research is encouraged.

(1) https://metrics.stanford.edu

Will the record be screened?

We will screen the record to confirm that it fulfils our definition of a protocol for meta-research. We will not review the protocol with regard to its scientific quality.

How to make changes to a record

By using your login details, you can make changes to a record at any time. Changes will be logged as amendments and the most recent version will be available for immediate viewing on the register. Previous iterations will also remain present and visible within the record, allowing for full transparency.

How to cite an IRMR registration

We recommend that you cite your IRMR registration in the following way:

Author(s). Title. IRMR: International Register of Meta-Research. Year of registration. Unique Identification number (IRMR_xxxxxx). Available from: hyperlink for the record.

For example: 

Chong S, The Relationship of Study Findings to Publication Outcome in Anaesthesia Research Post Implementation of Mandatory Trial Registration, IRMR: International Register of Meta-Research. 2019. IRMR_00003. https://www.irmr.org/records/IRMR_000003

We also suggest that you include the prospective registration of your protocol in your final report.

For example:

A protocol for this research was prospectively registered on IRMR (International Register of Meta-Research). This registration, including a full copy of the protocol, can be accessed at https://www.irmr.org/records/IRMR_000003.