A surprising number of researchers in the field of management are framing their results to make their work appear more significant, say a team of business professors who reported the findings of a survey of their peers in an article published in the January 2016 Journal of Management.
"Questionable research practices don’t necessarily translate into academic misconduct," said Dr. Charles White, associate professor of management and assistant dean of the College of Business and Economics at Longwood University, who co-authored the article with colleague Dr. Cheryl Adkins and researchers from other institutions. "Some practices, like falsifying data, clearly constitute an ethical breach. Other practices may be acceptable or not, depending on the circumstances. The bottom line is that all of these practices should be disclosed so that peer reviewers and readers are as informed as possible about the procedures that were followed."
In a wide-ranging survey of management researchers, White and his colleagues found that, while nearly no one reported falsifying data, more than one-third of those surveyed said they engaged in post-hoc exclusion of data for the express purpose of supporting hypotheses with statistical significance. Half said they selectively reported hypotheses on the basis of statistical significance or presented a revised hypothesis as the original.
"We see these practices crop up time and time again to the point of pervading the entire field," said White. "While they don’t constitute outright lying, they cloak the research so we are not getting the full picture. Because of that, we may be left with studies that are not replicable or reliable."
White and his colleagues found evidence of several forms of questionable research methods, including:
Selectively reporting hypotheses- Excluding data that does not support the hypothesis
- Hypothesizing after results are known
- Selectively including control variables
- Rounding off probability values
The research draws from a survey of 749 active management researchers combined with analysis of dozens of published studies. Analysis not only revealed the prevalence of the use of questionable research methods, but the origins of those methods and ultimately ways to combat their continued use.
The root causes of the use of questionable research methods in management journal articles are as varied as the methods themselves. To White, however, one stands head and shoulders above the rest: the overwhelming pressure to publish at all costs.
"Universities have increasingly high standards, especially for those on the job market and nontenured faculty, on quantity and quality of publications," said White. "When you combine this intense pressure to publish high-quality academic work with the fact that the best journals only want to publish large-scale, ground-breaking studies, there is little surprise that some researchers report employing questionable research practices in order to fulfill that publishing requirement."
The study also suggests that these methods may be learned in graduate school.
What to do to combat the use of these methods? White and his colleagues say the more open the scientific process, the more the results will stand up to rigorous challenges. Steps researchers should take, they say, include pre-registering hypotheses, research questions, study design and actual surveys, as well as sharing data. This would not preclude deviation from the original course of inquiry, but it would document changes so results are verifiable.
Leave a Comment