Background:
Inappropriate analysis and reporting of biomedical research remain a problem despite advances in statistical methods and efforts to educate researchers.
Objective:
To determine the frequency and severity of requests biostatisticians receive from researchers for inappropriate analysis and reporting of data during statistical consultations.
Design:
Online survey.
Setting:
United States.
Participants:
A randomly drawn sample of 522 American Statistical Association members self-identifying as consulting biostatisticians.
Measurements:
The Bioethical Issues in Biostatistical Consulting Questionnaire soliciting reports about the frequency and perceived severity of specific requests for inappropriate analysis and reporting.
Results:
Of 522 consulting biostatisticians contacted, 390 provided sufficient responses: a completion rate of 74.7%. The 4 most frequently reported inappropriate requests rated as “most severe” by at least 20% of the respondents were, in order of frequency, removing or altering some data records to better support the research hypothesis; interpreting the statistical findings on the basis of expectation, not actual results; not reporting the presence of key missing data that might bias the results; and ignoring violations of assumptions that would change results from positive to negative. These requests were reported most often by younger biostatisticians.
Limitations:
The survey provides information on the reported frequency of inappropriate requests but not on how such requests were handled or whether the requests reflected researchers' maleficence or inadequate knowledge about statistical and research methods. In addition, other inappropriate requests may have been made that were not prespecified in the survey.
Conclusion:
This survey suggests that researchers frequently make inappropriate requests of their biostatistical consultants regarding the analysis and reporting of their data. Understanding the reasons for these requests and how they are handled requires further study.
Primary Funding Source:
U.S. Department of Health and Human Services.
References
- 1.
Swazey JP ,Anderson MS ,Louis KS . Ethical problems in academic research. Am Sci. 1993;81:542-53. Google Scholar - 2.
Greenberg M ,Goldberg L . Ethical challenges to risk scientists: an exploratory analysis of survey data. Sci Technol Human Values. 1994;19:223-41. [PMID:11652278 ] CrossrefMedlineGoogle Scholar - 3.
Marco CA ,Larkin GL . Research ethics: ethical issues of data reporting and the quest for authenticity. Acad Emerg Med. 2000;7:691-4. [PMID:10905651 ] CrossrefMedlineGoogle Scholar - 4.
National Academy of Sciences (US); National Academy of Engineering (US); Institute of Medicine (US) Panel on Scientific Responsibility and the Conduct of Research . Responsible Science: Ensuring the Integrity of the Research Process. Vol. 1. Washington, DC: National Academies Pr; 1992. Google Scholar - 5.
Eastwood S ,Derish P ,Leash E ,Ordway S . Ethical issues in biomedical research: perceptions and practices of postdoctoral research fellows responding to a survey. Sci Eng Ethics. 1996;2:89-114. [PMID:11657788 ] CrossrefMedlineGoogle Scholar - 6.
Bebeau MJ ,Davis EL . Survey of ethical issues in dental research. J Dent Res. 1996;75:845-55. [PMID:8655786 ] CrossrefMedlineGoogle Scholar - 7.
Frankel MS . Ethics in research: current issues for dental researchers and their professional society. J Dent Res. 1994;73:1759-65. [PMID:7983263 ] CrossrefMedlineGoogle Scholar - 8.
Ellenberg SS . Fraud is bad, studying fraud is hard. Control Clin Trials. 2000;21:498-500. [PMID:11018566 ] CrossrefMedlineGoogle Scholar - 9.
Broome ME ,Pryor E ,Habermann B ,Pulley L ,Kincaid H . The Scientific Misconduct Questionnaire–Revised (SMQ-r): validation and psychometric testing. Account Res. 2005;12:263-80. [PMID:16578917 ] CrossrefMedlineGoogle Scholar - 10.
Pryor ER ,Habermann B ,Broome ME . Scientific misconduct from the perspective of research coordinators: a national survey. J Med Ethics. 2007;33:365-9. [PMID:17526690 ] CrossrefMedlineGoogle Scholar - 11.
Habermann B ,Broome M ,Pryor ER ,Ziner KW . Research coordinators' experiences with scientific misconduct and research integrity. Nurs Res. 2010;59:51-7. [PMID:20010045 ] doi:10.1097/NNR.0b013e3181c3b9f2 CrossrefMedlineGoogle Scholar - 12.
Marusic A ,Wager E ,Utrobicic A ,Rothstein HR ,Sambunjak D . Interventions to prevent misconduct and promote integrity in research and publication. Cochrane Database Syst Rev. 2016;4:MR000038. [PMID:27040721 ] doi:10.1002/14651858.MR000038.pub2 CrossrefMedlineGoogle Scholar - 13.
Li G ,Kamel M ,Jin Y ,Xu MK ,Mbuagbaw L ,Samaan Z ,et al . Exploring the characteristics, global distribution and reasons for retraction of published articles involving human research participants: a literature survey. J Multidiscip Healthc. 2018;11:39-47. [PMID:29403283 ] doi:10.2147/JMDH.S151745 CrossrefMedlineGoogle Scholar - 14.
Nogueira TE ,Gonçalves AS ,Leles CR ,Batista AC ,Costa LR . A survey of retracted articles in dentistry. BMC Res Notes. 2017;10:253. [PMID:28683764 ] doi:10.1186/s13104-017-2576-y CrossrefMedlineGoogle Scholar - 15.
Lei L ,Zhang Y . Lack of improvement in scientific integrity: an analysis of WoS retractions by chinese researchers (1997-2016). Sci Eng Ethics. 2017. [PMID:28889329 ] doi:10.1007/s11948-017-9962-7 CrossrefMedlineGoogle Scholar - 16.
Bozzo A ,Bali K ,Evaniew N ,Ghert M . Retractions in cancer research: a systematic survey. Res Integr Peer Rev. 2017;2:5. [PMID:29451549 ] doi:10.1186/s41073-017-0031-1 CrossrefMedlineGoogle Scholar - 17.
Moylan EC ,Kowalczuk MK . Why articles are retracted: a retrospective cross-sectional study of retraction notices at BioMed Central. BMJ Open. 2016;6:e012047. [PMID:27881524 ] doi:10.1136/bmjopen-2016-012047 CrossrefMedlineGoogle Scholar - 18.
Boutron I ,Ravaud P . Misrepresentation and distortion of research in biomedical literature. Proc Natl Acad Sci U S A. 2018;115:2613-9. [PMID:29531025 ] doi:10.1073/pnas.1710755115 CrossrefMedlineGoogle Scholar - 19.
Kingori P ,Gerrets R . Morals, morale and motivations in data fabrication: medical research fieldworkers views and practices in two sub-Saharan African contexts. Soc Sci Med. 2016;166:150-9. [PMID:27566044 ] doi:10.1016/j.socscimed.2016.08.019 CrossrefMedlineGoogle Scholar - 20.
Johnson DR ,Ecklund EH . Ethical ambiguity in science. Sci Eng Ethics. 2016;22:989-1005. [PMID:26169696 ] doi:10.1007/s11948-015-9682-9 CrossrefMedlineGoogle Scholar - 21.
Greeff M ,Rennie S . Phronesis: beyond the research ethics committee—a crucial decision-making skill for health researchers during community research. J Empir Res Hum Res Ethics. 2016;11:170-9. [PMID:27230235 ] doi:10.1177/1556264616650070 CrossrefMedlineGoogle Scholar - 22.
Thiese MS ,Walker S ,Lindsey J . Truths, lies, and statistics [Editorial]. J Thorac Dis. 2017;9:4117-24. [PMID:29268423 ] doi:10.21037/jtd.2017.09.24 CrossrefMedlineGoogle Scholar - 23.
Mahmud S ,Bretag T . Integrity in postgraduate research: the student voice. Sci Eng Ethics. 2015;21:1657-72. [PMID:25488333 ] doi:10.1007/s11948-014-9616-y CrossrefMedlineGoogle Scholar - 24.
Hofmann B ,Helgesson G ,Juth N ,Holm S . Scientific dishonesty: a survey of doctoral students at the major medical faculties in Sweden and Norway. J Empir Res Hum Res Ethics. 2015;10:380-8. [PMID:26333685 ] doi:10.1177/1556264615599686 CrossrefMedlineGoogle Scholar - 25.
Wang MQ ,Katz RV ,Howard D ,Harris BM ,Yan F . Bioethical issues in biostatistical consulting: development of a survey. Psychol Rep. 2007;100:191-4. [PMID:17451024 ] CrossrefMedlineGoogle Scholar - 26. American Association for Public Opinion Research. Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys. Oakbrook Terrace, IL: American Association for Public Opinion Research; 2016. Accessed at www.aapor.org/AAPOR_Main/media/publications/Standard-Definitions20169thEditionfinal.pdf on 28 August 2018. Google Scholar
- 27. The Carnegie Classification of Institutions of Higher Learning. 2017. Accessed at www.carnegieclassifications.iu.edu on 28 August 2018. Google Scholar
- 28.
Wang MQ ,Yan AF ,Katz RV . Identifying bioethical issues in biostatistical consulting: findings from a US national pilot survey of biostatisticians. BMJ Open. 2017;7:e018491. [PMID:29146653 ] doi:10.1136/bmjopen-2017-018491 CrossrefMedlineGoogle Scholar - 29.
Ranstam J ,Buyse M ,George SL ,Evans S ,Geller NL ,Scherrer B ,et al . Fraud in medical research: an international survey of biostatisticians. ISCB Subcommittee on Fraud. Control Clin Trials. 2000;21:415-27. [PMID:11018560 ] CrossrefMedlineGoogle Scholar - 30.
Liao QJ ,Zhang YY ,Fan YC ,Zheng MH ,Bai Y ,Eslick GD ,et al . Perceptions of Chinese biomedical researchers towards academic misconduct: a comparison between 2015 and 2010. Sci Eng Ethics. 2018;24:629-45. [PMID:28397174 ] doi:10.1007/s11948-017-9913-3 CrossrefMedlineGoogle Scholar - 31.
Olabarrieta-Landa L ,Romero AC ,Panyavin I ,Arango-Lasprilla JC . Perception of ethical misconduct by neuropsychology professionals in Spain. NeuroRehabilitation. 2017;41:527-38. [PMID:28946581 ] doi:10.3233/NRE-162144 CrossrefMedlineGoogle Scholar - 32.
Godecharle S ,Fieuws S ,Nemery B ,Dierickx K . Scientists still behaving badly? A survey within industry and universities. Sci Eng Ethics. 2017. [PMID:28971354 ] doi:10.1007/s11948-017-9957-4 CrossrefMedlineGoogle Scholar - 33.
Bouter LM ,Tijdink J ,Axelsen N ,Martinson BC ,Ter Riet G . Ranking major and minor research misbehaviors: results from a survey among participants of four World Conferences on Research Integrity. Res Integr Peer Rev. 2016;1:17. [PMID:29451551 ] doi:10.1186/s41073-016-0024-5 CrossrefMedlineGoogle Scholar - 34.
Pickett JT ,Roche SP . Questionable, objectionable or criminal? Public opinion on data fraud and selective reporting in science. Sci Eng Ethics. 2018;24:151-71. [PMID:28281156 ] doi:10.1007/s11948-017-9886-2 CrossrefMedlineGoogle Scholar - 35.
George SL . Research misconduct and data fraud in clinical trials: prevalence and causal factors. Int J Clin Oncol. 2016;21:15-21. [PMID:26289019 ] doi:10.1007/s10147-015-0887-3 CrossrefMedlineGoogle Scholar - 36.
Bouter LM . Commentary: perverse incentives or rotten apples? Account Res. 2015;22:148-61. [PMID:25635847 ] doi:10.1080/08989621.2014.950253 CrossrefMedlineGoogle Scholar - 37.
Holm S ,Hofmann B . Investigating the reliability and factor structure of Kalichman's “Survey 2: Research Misconduct” questionnaire: a post hoc analysis among biomedical doctoral students in Scandinavia. J Empir Res Hum Res Ethics. 2017;12:199-205. [PMID:28707501 ] doi:10.1177/1556264617714658 CrossrefMedlineGoogle Scholar
Author, Article and Disclosure Information
University of Maryland School of Public Health, College Park, Maryland (M.Q.W.)
University of Wisconsin–Milwaukee Joseph J. Zilber School of Public Health, Milwaukee, Wisconsin (A.F.Y.)
New York University College of Dentistry, New York, New York (R.V.K.)
Grant Support: By grant 1 ORIIR160027-01-00 from the Office of Research Integrity of the U.S. Department of Health and Human Services.
Disclosures: Disclosures can be viewed at www.acponline.org/authors/icmje/ConflictOfInterestForms.do?msNum=M18-1230.
Corresponding Author: Ralph V. Katz, DMD, MPH, PhD, Department of Epidemiology and Health Promotion, New York University College of Dentistry, 433 1st Avenue, Room 706, New York, NY 10010; e-mail, ralph.
Current Author Addresses: Dr. Wang: University of Maryland School of Public Health, 2387 School of Public Health Building, College Park, MD 20742.
Dr. Yan: 1660 North Prospect Avenue, Unit 505, Milwaukee, WI 53202.
Dr. Katz: Department of Epidemiology and Health Promotion, New York University College of Dentistry, 433 1st Avenue, Room 706, New York, NY 10010.
Author Contributions: Conception and design: M.Q. Wang, A.F. Yan, R.V. Katz.
Analysis and interpretation of the data: M.Q. Wang, A.F. Yan, R.V. Katz.
Drafting of the article: M.Q. Wang, R.V. Katz.
Critical revision for important intellectual content: M.Q. Wang, A.F. Yan, R.V. Katz.
Final approval of the article: M.Q. Wang, A.F. Yan, R.V. Katz.
Provision of study materials or patients: M.Q. Wang.
Statistical expertise: M.Q. Wang, A.F. Yan.
Obtaining of funding: M.Q. Wang.
Administrative, technical, or logistic support: M.Q. Wang, A.F. Yan.
Collection and assembly of data: M.Q. Wang, A.F. Yan.
This article was published at Annals.org on 9 October 2018.
Submit a Comment
Contributors must reveal any conflict of interest. Comments are moderated. Please see our information for authorsregarding comments on an Annals publication.
*All comments submitted after October 1, 2021 and selected for publication will be published online only.