Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain
the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in
Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles
and JavaScript.
Researchers have identified three warning signs that suggest journals might be engaging in suspicious publishing activities. Using these, they have found dozens of titles indexed in the leading scholarly database Scopus that could be problematic.
The three red flags are, for any given journal in a single year: a doubling of publication output; having more than 70% of published articles originating from a single country; and having at least 15% of articles flagged by the Problematic Paper Screener, a free online tool.
Using titles extracted from the freely available database OpenAlex, the researchers identified 3,753journals that met at least one of these criteria. They cross-referenced these with journals indexed in Scopus and found that, of those, 5% had been delisted. The findings were published in a preprint1 on 20 October.
The researchers, and research-integrity specialists, say that the signals help to identify journals that merit further investigation — but do not prove that titles are problematic. They serve as “rudimentary flags”, says Achal Agrawal, a co-author of the preprint and freelance data scientist in Raipur, India. And they indicate the problems that databases face as poor publishing practices proliferate, he adds.
Rankings basis
Bibliometric databases such as Scopus are important in the global research ecosystem because they inform many rankings used to evaluate individual researchers and institutions. Inclusion in Scopus, run by the Dutch publisher Elsevier, typically indicates that a journal is reputable.
Scopus regularly reviews the journals it lists, using a tool to spot signs of suspicious activities that suggest titles are churning out low-quality research that doesn’t adhere to accepted scholarly standards. These include “changes in publication volume and journal scope, unusual patterns in authorship and author collaboration, anomalous citation behaviour, and indications that a journal has become vulnerable to questionable content”, a spokesperson for Elsevier told Nature.
For flagged journals, Scopus then conducts “a careful, individual assessment of all available information, including consideration of the journal’s own response”, the spokesperson added. It removes journals that do not meet certain metrics thresholds. Scopus delisted 156 journals last year, and 100 so far this year.
“No single signal, and no fixed threshold, determines the outcome,” Elsevier’s spokesperson said. “A signal is a prompt to investigate, and where needed the journal is put forward for re-evaluation” by Scopus’s content-selection board.
The study reported that the roughly 3,500 journals that have not been delisted together contributed more than half a million papers in the years that they were flagged by at least one signal. Nearly 300 journals were flagged by two signals and 28 journals by all three.
Each signal warrants a further look into the journal, but on its own is not “a definite decision to delist”, says Anna Abalkina, a research-integrity sleuth and social scientist at the Free University of Berlin. “There could be a lot of reasons why a journal duplicates the volume of publication.” For example, that often happens for a new journal, or as part of a publisher’s business plan, she says. But Abalkina thinks that a journal with all three red flags suggests suspicious activity that is difficult to find legitimate justification for.
Álvaro Cabezas Clavijo, who studies research evaluation at the International University of La Rioja in Logroño, Spain, agrees that the three signs should be interpreted with care. For instance, for small journals, having a large proportion of papers from one country might not be “a mark of questionable practices”.
Enjoying our latest content?
Log in or create an account to continue
Access the most recent journalism from Nature's award-winning team
Explore the latest features & opinion covering groundbreaking research