Questionable Research Practices
Answer
What are Questionable Research Practices?
Questionable Research Practices (QRPs) are a range of activities that intentionally or unintentionally distort data in favour of a researcher’s own hypotheses (Fortt 2021). They are one factor which can lead to low degrees of reproducibility in scientific research. Caution should be taken when evaluating QRPs, which range in seriousness, and can be highly context-specific.
Often QRPs aren’t inherently problematic, but are made so either by insufficient reporting, or by exploiting flexibility available in the scientific research process to achieve desired results. For example, selective inclusion of data or hypothesising after the results are known (referred to as ‘HARKing’). However, practices such as p-hacking or data fabrication have more serious implications for the validity of research outputs.
Examples of QRPs
-
P-hacking: Performing multiple analyses, or selective reporting of analyses in search of a statistically significant result.
-
Salami slicing: Manipulating data from a single study to generate multiple publications.
-
HARKing: ‘Hypothesising after the results are known’ is the presentation of exploratory analyses as confirmatory analyses.
-
Data Manipulation: A wide range of practices abusing the degrees of freedom available to researchers. Examples include the inappropriate use of outliers, variance manipulation by restricting range between groups, or only reporting on desirable subgroups.
Avoiding QRPs
QRPs can be avoided through increased research transparency. For example:
-
Preregistration clearly shows readers when an analysis has been performed post-hoc.
-
Open data reveals where data has been manipulated to produce a particular result.
-
Techniques such as multiverse analyses aim to address issues around analytic flexibility.
-
Reforming scholarly publishing with interventions such as registered reports are designed to disincentivise the use of QRPs, removing pressure for researchers to produce positive results, often seen as a requirement to publish in prestigious journals.
More information
Guidance from UKRIO on Questionable Research Practices