The Proliferation of Garbage Academic Work in “Peer-Reviewed” Journals
[guest post by JVW]
The Chronicle of Higher Education carried a remarkable account (note: article behind a paywall but you can sign up for an account and get one freebie) by Tom Bartlett about how a journal published by the venerable academic publisher Springer found itself recently appending “editorial expressions of concern” to hundreds of “research” articles they have recently published which can almost certainly be categorized as, to use a fancy publishing industry term, utter bullshit:
A peer-reviewed journal recently published a mind-bending paper. It begins with a highly technical section about groundwater seepage before delving into a lively discussion of dance training. The paper shifts back and forth between the two topics, informing the reader about rare-earth elements before urging dancers to “tighten buttocks” during warm-ups. There are tables and graphs, citations and hyperlinks. It’s all very sober and scientific-seeming and yet, at the same time, completely bonkers.
The paper appeared last month in the Arabian Journal of Geosciences, which is one of several thousand journals put out by the publishing giant Springer Nature. If this was just one weird paper in an obscure journal, it probably wouldn’t be noteworthy. But hundreds — 412, to be exact — of equally bizarre or suspicious papers have popped up in the same journal in recent months. One examines college sports-injury insurance along with rainfall on the Loess Plateau, in China. Another deals with sea-level height and aerobics teaching. In what purports to be a legitimate geosciences journal there are at least five papers on swimming and seven on basketball.
Is this a prank, meant to highlight the ubiquity of mindless postmodernism in today’s academy like the Sokol Hoax of a quarter-century ago and the more recent Sokol Squared controversy at Portland State University? It doesn’t seem so. For one thing, the groundwater/athletics papers are a bit too arch and dull to actually be funny in a parodic way, jumbled and incoherent though they may be, and they don’t particularly deliver a lot of the Foucaultian and Derridan nonsense upon which the Sokol hoaxes were built. Also, nobody as of this writing has stepped up to take credit for the massive spamming of this journal. What other clues might we have?
[. . . ] Some of the papers, though not all, were published as part of a special issue of the journal edited by Sheldon Williamson, a professor of electrical, computer, and software engineering at the University of Ontario Institute of Technology. Williamson told Retraction Watch that his email account had been hacked.
That part is just delicious isn’t it? The computer science professor whose email is hacked. Or was it?
When I spoke with Williamson, he said he didn’t know for sure that his email had been hacked, but he assumed it had been. He said he was just as perplexed as everyone else about how so many ridiculous papers, with his name listed as the responsible editor, had made it into the journal. “I don’t know which ones are legitimate and which ones are not,” he said. As for what happened here, he said: “I believe people are desperate to publish. I don’t know. It could be anything.”
It helps if you attempt to view Professor Sheldon Williamson of the University of Ontario Institute of Technology as played by the late great Peter Sellers. But that’s neither here nor there. Back to the narrative:
[. . .] I also spoke to Abdullah Al-Amri, the founder and editor in chief of the journal, and a professor of geophysics at King Saud University, in Saudi Arabia. He assured me that he reads every paper that appears in the journal, which is remarkable considering that it publishes two issues each month. In September alone, the journal published 276 papers. At that clip, Al-Amri would be reading roughly 10 papers a day, every day, including weekends.
“You have to believe there are some people hacking the journal,” he told me. “I know which papers I have to approve, and which papers I have to disapprove. I know my job for the last 30 years. But if some people sign into the Springer system on my behalf, I don’t know how this happens.”
And that’s a pretty good insight into the editorial processes of modern academic journals. Is it any wonder that so much of their output later turns out to be either intellectual piffle or outright fraudulent? Here’s some more bad news for Springer:
What happened with the geosciences journal is extreme, but it isn’t unique. Springer also recently found that another of its journals, Personal and Ubiquitous Computing, had published 24 papers that appear to be nonsense. In May, yet another Springer journal, Current Psychology, retracted more than a dozen articles that had been part of a special issue due to “problems with editorial handling and peer review.” Again, some of the articles seemed to have little to do with the journal’s topic and, as detailed in notes on each article, were riddled with errors and methodological problems.
You will be elated (or perhaps exasperated) to learn that Springer is not alone in suffering the humiliation of publishing journals which carry debased and decrepit work:
And it’s not just Springer. Elsevier, another journal-publishing giant, recently issued editorial expressions of concern about some 400 articles that had fallen “beneath the high standards” for one of its journals. Meanwhile, Taylor & Francis retracted a special issue because the guest editor had been “impersonated by a fraudulent entity.” One lesson here seems to be that handing the keys of a journal over to an unpaid guest editor might be a bad idea.
Speculation is that much of this bogus output comes from China, where PhD students are required to publish in a journal before they are awarded their doctoral degree, and current PhDs are paid bonuses by the state for being published in [ahem, ahem] prestigious academic journals. China has managed to infiltrate and subvert a great deal of English-language academia, so at some point the world’s leading universities — especially those here in the United States — are going to have to ask themselves the difficult question of whether all of that Chinese tuition and research money is worth debasing your academic standards.
Mr. Bartlett clearly gets this, and he is brave enough to call out the unsustainable and unnecessary (those are my words, not his) growth in graduate studies in marginal topics that has taken place over the last few decades, leading to esoteric research into subjects that virtually no one cares all that much about. Though this phenomenon is most prevalent in the social sciences, where buttressing the dominant ideological narrative and trendy woke posturing are far more important than using the discipline to solve long-standing problems, this whole scandal with the Arabian Journal of Geosciences shows that the sciences can also produce its share of meaningless dreck.
[. . .] [T]his strange episode brings to mind broader questions about academic publishing, including whether way too much subpar research is being pumped out each year and whether peer review is all it’s cracked up to be. I spoke to one of the PubPeer commenters, Nicholas Wise, who helped uncover the nonsense papers in the geosciences journal. He asked a question that’s worth contemplating: “Is anyone actually reading this journal?”
As long as we continue to expand graduate programs to the extent where second-rate minds are in faculty positions mentoring third-rate students who are doing research in minutiae simply because it’s a field as of yet unexplored, expect this kind of flim-flam to continue. Dialing back the issuance of questionable advance degrees which provide a financial windfall to the university community without really benefiting society as a whole is an educational reform that is long overdue.
– JVW
China is just turning up as a bad actor in virtually every field in which it participates, isn’t it? I’ve turned into a pretty staunch China hawk over the past few years, and I have recently become a huge fan of reading The Spectator whose coverage of China is routinely excellent.
JVW (ee64e4) — 9/30/2021 @ 8:09 amEveryone reads the abstracts. Maybe the first few paragraphs and the conclusions. The stuff in between? Not so much.
Kevin M (ab1c11) — 9/30/2021 @ 8:18 amJVW, you need to read The Structure of Scientific Revolutions by Thomas Kuhn. It’s one of the most profound books written at the end of the 20th century.
Gawain's Ghost (c6fd3b) — 9/30/2021 @ 8:40 amThe only explanation for this that I can think of is that this paper is checked by a computer software program. )ptherwise it would look more plausible)
It must check for plagiarism, at least against papers in the same field, and for words used in that field (so some of it must be on topic) and for quantity of material and maybe for containing genuine citations, working hyperlinks, tables and graphs.
The whole thing must be based on no human being actually caring what is written there. And the last thing they want is for anybody to cite it, except for other papers created with the same strategy.
BTW, don’t see where it says any of them were peer-reviewed. That is a different issue. These weren’t peer reviewed unless somebody just took the money. But it doesn’t say anyone besides the editor reviewed those papers.
Sammy Finkelman (51cd0c) — 9/30/2021 @ 8:50 amChina is just turning up as a bad actor in virtually every field in which it participates, isn’t it?
And we’re paying them to do it.
Reaganomics!
nk (1d9030) — 9/30/2021 @ 8:50 am3. Gawain’s Ghost (c6fd3b) — 9/30/2021 @ 8:40 am
1962 was not near the end of the 20th century.
Sammy Finkelman (51cd0c) — 9/30/2021 @ 8:53 amKevin M @2.
But why throw in junk? Just publish tose few paragraphs.
The answer must be to bulk it up, without triggering a plagiarism detector.
Sammy Finkelman (51cd0c) — 9/30/2021 @ 8:55 am1962 was at the beginning of the end
steveg (e81d76) — 9/30/2021 @ 8:56 amI was about to publish a statement saying those articles shouldn’t stand a chinamans chance of being published, but after peer review we decided to amend it to “a Uighur’s chance” in the interests of scrupulous academic rigor. In the article, I cover the mating habits of newts, toe injuries in ballerinas, and conclude with how those two things enable hypersonic weaponry.
steveg (e81d76) — 9/30/2021 @ 9:13 amI am seeking a position at Bumfork CC which as Joe Biden astutely notes, “this outfit over there past the pudding cups should be free of charge”
8 1962 is only 5/8 of the way through.
Sammy Finkelman (51cd0c) — 9/30/2021 @ 9:28 amThis and the replication crisis in modern science should give anyone pause when listening to some who says they “follow science.”
When science has a couple of big problems like this, “following science” is not that easy. Or as closed-and-shut as they would like.
Hoi Polloi (ade50d) — 9/30/2021 @ 10:22 amThis and the replication crisis in modern science should give anyone pause when listening to some who says they “follow science.”
Precisely. As so many of us have been pointing out, science is a process and not an end result. And when your “science” is receiving exactly zero peer review and consists instead of “trust us: we have credentials,” it’s a very shallow process indeed.
JVW (ee64e4) — 9/30/2021 @ 10:27 amWell, and we call so many things “science” these days. There’s more science to fly-fishing that dome of these fields. Even back in prehistoric times when I was in college, sociology was trying to make itself a science by crunching numbers to prove that their garbage-out was heavily influenced by their garbage-in.
Kevin M (ab1c11) — 9/30/2021 @ 10:29 amThe Scientific Method (possibly the single best invention of mankind):
1. Observe a natural phenomenon and define a question about it
2. Make a hypothesis, or potential solution to the question
3. Test the hypothesis
4. If the hypothesis is true, find more evidence or find counter-evidence
5. If the hypothesis is false, create a new hypothesis or try again
6. Draw conclusions and report same.
It was, however, perverted from the start as the first “scientific journals” started as part of the Newton-Leibniz war of the late 1600s. Sides were picked and the journals were invested in them.
Government funding of research, when Government holds a preferred outcome, is the modern perversion.
Kevin M (ab1c11) — 9/30/2021 @ 10:35 amTrue.
I want science to explain to me why I used to be able to tie my own lures and now everything is do darn small that I can’t see it.
Hoi Polloi (ade50d) — 9/30/2021 @ 10:42 am14. Kevin M (ab1c11) — 9/30/2021 @ 10:35 am
7. Everybody is supposed to agree. So anybody who wants to see he is indisputably right, calls it science.
Karl Marx called what he wrote “scientific socialism”
8. It takes time ti alter conventional wisdom, so “science” is always behind the times. Especially when nobody is supposed to jump ahead, as in medicine.
Its totally ridiculous, and the fact that people and doctors can be incredibly stupid doesn’t make it un-ridiculous.
Sammy Finkelman (51cd0c) — 9/30/2021 @ 11:21 amI’ve always said that 90% of what is in scholarly publications is of limited value…work that’s not particularly original, work that is so down in the weeds or inexplicably complex that it is of extremely limited practical use, work that is sloppy or wrong, and work that misses enough detail that it can’t be independently verified. Of the remaining 10%, maybe 2% is truly significant. Why? The philosophy of publish-or-perish compels submitting everything…which results in a lot of chaff.
I don’t think that it discredits scientific research or science itself but it shows that the pressure to publish creates perverse incentives and sometimes sloppy peer review. Now what is illustrated here in this post falls into a different category of satire…and absurdity…and hopefully provides a lesson on due diligence. But I would hate for it to be used as another strike against expertise. THERE ARE experts out there that produce great work…and there is value in other people checking and reproducing it. It doesn’t mean the anonymous crank in his Idaho fallout shelter should (necessarily) be considered a reliable source because he one time stayed at a Holiday Inn Express. There are great climatologists, virologists, and lawyers out there…that make valuable contributions. We shouldn’t minimize that….
AJ_Liberty (ec7f74) — 9/30/2021 @ 1:22 pmI want science to explain to me why I used to be able to tie my own lures and now everything is do darn small that I can’t see it.
Which science? Gerontology or optometry? There are actually ways to make those things bigger again.
Kevin M (ab1c11) — 9/30/2021 @ 2:20 pm“And we’re paying them to do it.
Reaganomics!”
nk the liberal skillfully eliding any analysis of the 90s, aka The Decade When The Big China Selloff Actually Happened. Yah Lin “Charlie” Trie? John Huang? Johnny Chung? Lost in time, like tears in the rain, dismissed as conspiracy back then, and thus allowed to permeate both major parties and Chambers of Commerce.
If America had just been more racist and nativist, imagine all the terrible, expensive disasters we could have avoided! 9/11, the subprime loan scandal, the industrial selloff, the C-Virus! Sadly, it was not, and we did not. Perot wept.
H. Ross Perot (978ee8) — 9/30/2021 @ 2:46 pm16. SF: Karl Marx called what he wrote “scientific socialism”
It was actually Engels who attempted to use that word to declare Karl Marx infallible.
https://en.wikipedia.org/wiki/Scientific_socialism
Sammy Finkelman (02a146) — 9/30/2021 @ 3:03 pm15. Hoi Polloi (ade50d) — 9/30/2021 @ 10:42 am
At precisely the age of 43, human eyes lose their ability to change focal length and become more like fixed focus, which is usually longer distance.
That’s why even the best baseball hitters have to retire at precisely the age of 42 or 43.
https://www.aoa.org/healthy-eyes/eye-health-for-life/adult-vision-41-to-60-years-of-age
https://www.aarp.org/health/conditions-treatments/info-2019/eye-changes-with-age.html
https://www.sciencedirect.com/topics/medicine-and-dentistry/presbyopia
Somewhere I read the age of 43, but I can’t find a reference.
Sammy Finkelman (02a146) — 9/30/2021 @ 3:15 pm17. AJ_Liberty (ec7f74) — 9/30/2021 @ 1:22 pm
It tends to discredit the institution of science.
I don’t think it’s satire. It’s gamming the system. It must be that some of the checking is automated.
Sammy Finkelman (02a146) — 9/30/2021 @ 3:18 pm22″ rainbow on a size 20 hook with 6x, I could never see the hole in the hook at my age today without magnification.
mg (8cbc69) — 9/30/2021 @ 4:40 pmThe 20-20 club.
But I would hate for it to be used as another strike against expertise. THERE ARE experts out there that produce great work…and there is value in other people checking and reproducing it.
I would agree entirely with that. But my point in this post is that we should not be content to say to ourselves, “Well, this person published a paper in some peer-reviewed journal so whatever his conclusions are, they must be correct.” We have ample evidence not only that a great deal of academic research is heavily politicized, but that the peer review process is often garbage. So I think it’s entirely reasonable anytime an “expert” tells us that he has published research findings, we should respond by asking which exact journal and how strict was the review process. The days in which we should take these findings at face value — especially where there is a contentious social policy controversy at stake — are long gone.
JVW (ee64e4) — 9/30/2021 @ 5:41 pm#14, not exactly. “Test the hypothesis” should be “attempt to falsify the hypothesis.” The hypothesis can be falsified but never true-ified.
GKHoffman (2a3a16) — 9/30/2021 @ 6:00 pmRegarding garbage academic work:
National School Boards Association asks Biden admin to look into angry parents as possible ‘domestic terrorism and hate crime’ threats
https://thepostmillennial.com/national-school-boards-association-asks-biden
I wonder what they thought of last summer’s rioters.
Obudman (18ecb6) — 9/30/2021 @ 6:35 pm24.
Peer review doesn’t check for fraud, or statistical evidence of made up numbers, which is the only thing worth checking for.
You can also check for premises and logic, but anyone can judge for themselves.
Sammy Finkelman (51cd0c) — 9/30/2021 @ 7:55 pm“In a single day, the Biden administration said parents shouldn’t be the “primary stakeholder” in their children’s education and the National School Board Association asked the feds to crack down on anti-critical race theory protests as “domestic terrorism.””
There are conservatives who voted for this.
Obudman (18ecb6) — 9/30/2021 @ 9:16 pm“Regarding garbage academic work:”
I died of irony poisoning seeing this post followed by a link to the Post Millennial.
Here’s the actual letter, which for some reason is not linked in the article:
https://nsba.org/-/media/NSBA/File/nsba-letter-to-president-biden-concerning-threats-to-public-schools-and-school-board-members-92921.pdf
Davethulhu (017f04) — 9/30/2021 @ 9:39 pmSpeaking of garbage academic work…
Paul Montagu (5de684) — 9/30/2021 @ 10:43 pmThat letter by the NSBA is so amazingly overwrought that it is almost laughable. You would think that school board members around the country are being beaten, horsewhipped, tarred & feathered, and ridden around town on a rail. Yet when you do a rudimentary search on “school board member assaulted” what you get are results for school board members being “harassed” — with harassed usually meaning “were yelled at by meeting attendees” — or “intimidated” or some such interaction which falls well short of being physically assaulted. Yeah, there’s a lot of obnoxious behavior being documented, but why do school board members think they would be treated any differently than city council members, state legislators, Congress members, etc., all of whom have to deal with angry activists of all stripes screaming in their faces about this and that hobbyhorse?
The most laughable article from my search is the one from U.S. News & World Report. Some guy who says he’s been on the school board for 21 years is complaining about how unruly the community has become at meetings. First of all, dude, nobody should be serving for two decades on a school board; it is quite literally the sort of volunteer job that you do for four, maybe eight, years and then let someone else have their turn. He ought to be turned out of office just for holding on to the seat for so long. The board member and the reporter also act as if it is a major source of alarm that conservative political groups are now taking an interest in who serves on the school board in their communities. As if the teachers’ unions, allied fully with Democrats and left-wing interest groups, haven’t been using their political organization and forced dues to dominate those boards for decades. But it’s so typical of the left to suddenly be outraged when the other side adopts the tactics that have worked so well for their own side for all these years.
Thanks for linking that letter, Davethulhu. It’s the sort of progressive bed-wetting that actually makes me think that the protesters just might be doing something right, even if their tactics often leave a lot to be desired.
JVW (ee64e4) — 9/30/2021 @ 10:55 pmhttps://www.aier.org/article/how-activist-academia-destroyed-scholarly-peer-review/
Then you arise at the other problem in garbage academic work, when the authors deliberately misstate previous works to support their position and use false information to promote their works.
NJRob (eb56c3) — 10/1/2021 @ 7:42 am