Regional profile of questionable publishing

In this study, we compare the affiliation information between authors and journals. We use the country information from the affiliations with each country’s gross domestic product (GDP) by analyzing the scholarly publication data from SCOPUS. We first extracted 20,864,247 papers, 6,971,029 authors, and 27,938 journals from the January 2019 dump of Scopus CUSTOM XML DATA. We then restricted data dating from 2010 to 2018 and having affiliation information, which the author’s data contains 906,610 affiliations with 177 countries. To reduce the potential statistical error, i.e., fluctuation from the low number of publications, we excluded journals that published less than 30 publications during the timeframe.

To identify the questionable and unquestioned journals, we employ the pre-complied list from our own previous study based on Beall’s list (You et~al., 2022). We cross-checked their ISSN and name between the SCOPUS dataset and the list, and excluded journals that publish less than 30 articles per year. In total, we found 766 questionable journals along with their 1,293 unquestioned counterparts. One questionable journal can be matched with multiple unquestioned journals because we assigned an unquestioned journal for each ASJC category of a target questionable journal.

We defined comparative unquestioned journals, matched with questionable journals, using three criteria from our previous study (You et~al., 2022): (1) similar journal impact, (2) same subject area classification (ASJC) in Scopus, and (3) a similar amount of annual publications. First, we limited the candidate journals to those belonging to the same subject area (ASJC). Second, we divided journals into three groups according to their annual publication volume and limited the candidate journals to those within the same group. Finally, we selected unquestioned journals with a similar journal impact. Note that we computed the journal impact using the Journal Citation Reports’ Impact Factor methodology, but calculated it with the SCOPUS dataset. As a result, we yield 1,293 unquestioned journals since one questionable journal belongs to multiple subject areas. We should note that the purpose of the comparison focuses on figuring out the overall patterns of questionable journals and publishers rather than revealing differences between two individual journals. To compare the tendencies of general journals without considering journal impact, we used another set called other journals, which consisted of all 25,879 Scopus journals not included in the two above sets.

When deriving country profiles for both journals and papers, it is essential to assign the correct country to each. We assume that the first author plays the most significant role in the publication; therefore, we consider the paper to belong to the first author’s country. Since a single author may have multiple affiliations, we set the sum of the author’s countries as given equal weight as one: as an illustrative example, $2/3$ of country A and $1/3$ of country B are assigned to the paper if the first author has two affiliations with country A and one affiliation with country B. Some papers have equally contributing first authors, yet we consider a single author who appears first in the author list to be the first author regardless of their contribution markups. Additionally, the corresponding author may play a central role in the publication process. We found that $91.3\%$ of papers list the same country for both the first and corresponding authors; thus, considering the corresponding author as the first author should not significantly change the results. In the meantime, we employ the country of the journals belonging to as listed in the Scimago Journal & Country Ranking (SCImago, 2017); we hereafter call it as “publishing country”.

To define neighboring countries for a given country, we construct two types of networks: i) by the geographical distance and ii) by academic distance as the inverse number of co-authored papers between countries. The geographical distance between two countries is measured by the haversine distance between their respective centroids (Google Developers, 2022). We also count the number of co-authored papers as the number of papers whose affiliations include both countries. Here, we consider papers with two or more authors solely to exclude the case that a single author has multiple affiliations with different countries. We then select five topologically, geographically, and academically nearest countries as the neighboring countries regarding a given publishing country.

According to former studies (Macháček and Srholec, 2017), the incidence of questionable publications is higher in less developed countries than in developed countries (Figure 1A). We discovered that 122 countries out of 177 publish less than 5% of their publications in questionable journals. Meanwhile, low-GDP countries publish more than 10% of their publication in questionable journals. These countries are mainly located in Southeast and Middle Asia, and their GDPs are less than 5% of that of the country with the highest GDP. The result is consistent with previous studies indicating that less developed countries are producing an increasing number of questionable publications.

We discovered, however, that the contribution of less developed nations to questionable publishing seems to be overestimated. The Pearson correlation between the number of publications in questionable journals and the GDP is $0.76$ (Figure 1B), while the correlation between the rate of questionable publications and the GDP is $-0.07$ (Figure 1A). Although the proportion is low in comparison to the total number of published papers, the high-GDP countries produce a larger number of questionable publications because they produce more publications in total (Figure S1). Countries publishing more than 10% of their publications in questionable journals occupy 9% of entire questionable publications but only 2% of all academic publications during the same period. In contrast, the United States, the highest-GDP country, is responsible for 20% of all scholarly publications and 9% of all questionable publications. This finding suggests that low-GDP countries have a limited impact on the growth of questionable publications. Meanwhile, despite a small publication rate, countries with a high GDP produce more questionable publications. According to previous studies (Beall, 2012; Memon, 2019), most questionable journals employ the open-access model, which indicates that high-GDP countries contribute more to growing questionable journals in terms of both publication volume and profit. In conclusion, the disparity in the denominator, i.e., the total number of publications in the country, considerably contributes to the overestimation of questionable publishing in countries with a low GDP.

Note that the majority of countries are distributed along the diagonal line $y\sim x$ (Figure 1B). Even though we do not consider the country in the filtering process for unquestioned journals, we discovered that the country produces a similar number of questionable and unquestioned publications. The result merely indicates that there is no disparity in the selection of journals at the country level. Therefore, although authors from less developed countries have a greater likelihood of publishing in questionable journals, their contribution to such publications may not be significant, regarding total impact.

According to the aforementioned observations, the impact of countries with a high GDP is frequently overlooked in questionable publications regarding their significant contributions. Nonetheless, another noteworthy aspect is the location where journals are published (Figure S2). There is a strong correlation between non-questionable journals and the GDP of the country of publishing ($0.72$ for unquestioned journals and $0.75$ for other Scopus journals). Thus, the greater a country’s economic strength, the more scientific journals it publishes. However, the questionable journals have a weaker Pearson correlation with the country’s GDP of only $0.30$.

Authors select journals based on their empirical and local information, including academic reputation, quantitative metrics, editor, and personal preference (Frank, 1994). Authors tend to perceive that international readership has been influenced by the journal’s title (Jamali, 2024). Quantitative indicators are likely to be low for these regional journals because their readership is less extensive than that of international journals, which have potential readers from all over the globe (Abramo et al., 2016). Nonetheless, because these regional journals mainly tackle issues of regional significance, they play a crucial role in the academic ecosystem of the region, which is not reflected in a quantitative indicator.

From this perspective, the concept of regionality may provide additional insight into the current trend of questionable publishing. If questionable journals address significant issues in a certain region, these journals will play an important role as new alternatives within the regional academic community. This possibility can be inferred from the fact that questionable journals have a larger market share in developing nations. To test this hypothesis, we first calculate the proportion of the most-published countries in a given journal. The simple calculation demonstrates that the majority of questionable journals are not occupied by a single country (Figure 2A). Here, we define the primary country as the author’s country that has the largest share of a given journal. Questionable journals have a lower share of the primary country ($0.38$) than unquestionable ($0.47$) journals and other Scopus journals ($0.51$) on average. To say the dominant country is the country that publishes more than $60\%$ of papers in a single journal, i.e., more than a half of publications are occupied by a single country, only $15\%$ of questionable journals have dominant country while $34\%$ of other types of journals have one. This behavior can be understood by another regionality reflected in the written language of papers from the primary countries. We discovered that 148 unquestionable journals accept articles in non-English, whereas only seven questionable journals do so. In other words, while a significant proportion of unquestioned journals accept non-English articles for publication in their respective local academia, questionable journals with comparable quantitative metrics did not.

The difference between the primary and publishing countries is a possible explanation for why questionable journals are less dominated by the primary country. We analyze the primary country’s publication rate with its relationship to the publishing country of the journal (Figure S3). 53% of all journals display the same country of publication and primary country, whereas 47% do not. For journals with identical primary and publishing countries, the publication share of primary countries is distributed nearly uniformly, with a peak near 1. There are 30.7% of these journals where the primary country generates more than 80% of all publications. In contrast, if the two countries are dissimilar, the publication proportion of primary countries in a journal is reduced. We discovered that only 22.3% of questionable journals are published in their home country, suggesting that the primary country has a lower publication rate in the remaining 77.7% of questionable journals. In short, if the primary country is identical to the publishing country, the primary country may dominate more. Since the primary country is different from the publishing country of many questionable journals, the primary country’s publication rate is low.

However, the relationship between the primary and publishing country cannot completely explain regionality. For instance, a substantial number of journals have high publication rates in primary countries that are not the same as the publishing country. The rapid expansion of mega-publishers, to which an increasing number of journals now belong, is one possible explanation for this circumstance. Citation indexes commonly attribute the location of publishers as the country of publication, although they are representative of a particular regional academic community. As an illustration, the “Korean Journal of Chemical Engineering” is the official publication of the “Korean Institute of Chemical Engineers”; however, SJR lists its publishing country as the United States because it is published by Springer New York. In addition, the local academic community may encompass neighboring nations that share their human resources and historical contexts. Therefore, we introduce a new analysis using the neighboring nations. We consider two distinct definitions of neighboring countries to gain a comprehensive understanding of regionality: i) geographical neighbors and ii) academic neighbors (for details, see Section 2). Here, the five closest countries are selected as the neighboring countries for a given country.

We first compare the share of the primary country and two types of neighboring groups (Figure S4). Comparing them reveals that the total proportion of neighboring countries (including the publishing country itself) increases when the primary country is included in the neighboring countries, regardless of whether it is the publishing country (Figure S4 A–B and D–E). For journals with identical primary and publishing countries, 66.7% (76%) of all journals are dominated by the publishing country and its geographical (academic) neighbors (i.e., have a publication share greater than 60%). When the primary country is not the publishing country, only 2.7% of journals have a primary country among their geographical neighbors, while 70.8% of journals have a primary country among their academic neighbors. According to these statistics, geographical distance plays no significant role in publishing on a regional scale. However, the relationship between the primary country and neighboring countries changed the publication shares. For journals that neighboring countries include the primary country, a publication share increases by 15.8% (19.3%) on average. In contrast, the publication share of the neighboring countries is lower by 24.3% (14.2%) than the primary country’s publication rate when the primary country is not included in the neighboring countries (Figures S4 C and F). In short, the profile of neighboring countries may explain the extended regionality of journals, thereby can support the validity of the analysis.

We then apply the concept of neighbors to illustrate the regional characteristics of both questionable and unquestionable journals. Comparing the publication share of geographical neighbors (Figure 2A and B), unquestioned journals and other journals show similar distributions, which displays a two-peaked distribution with the most frequent or rare publishing; thus, in part, there are regional journals. In contrast, questionable journals lack geographical characteristics. The majority of geographical counterparts of questionable journals rarely publish in the journal. In other words, various countries, not just the geographical neighbors, publish their papers in questionable journals showing their international characteristics. This trend also holds for the academic neighbors (Figure 2C) with a two-peaked distribution at the maximum and moderate publication share for the unquestionable journals, whereas the distribution for the questionable journals is only single-peaked at the intermediate publication share. Our observation of the primary country’s publication share suggests that there are approximately two types of journals: regional journals with a large contribution from the publishing and neighboring countries, and international journals with a negligible contribution from these countries.

A natural step forward is in-depth exploring the share of countries in a given journal. We examined the journal’s regional dominance by examining the share of publications by country’s rank in a given journal (Figure 3A). The decline of share by rank is slight for journals in that the primary country’s publication rate is low. For journals that have the primary country’s publication rate of $20-30\%$, the top three countries publish more than $10\%$, and even the twentieth-largest country can produce more than $1\%$ of the journal’s total publications. When the primary country’s publication rate increases, the publication share is concentrated on the highly ranked countries. For journals that the primary country’s publication rate is $80-90\%$, only the primary country produces more than $10\%$ and only the five countries can produce more than $1\%$ of the journal’s total publications. Therefore, the publication rate of the primary country can be employed as a proxy for regionality.

To confirm if the proxy for regionality supports the hypothesis of regional academia, we examine regional dominance, which is a comparative analysis of the selected portion of journals by primary country. Specifically, we calculated the number of journals from each country that is the primary country of the journal (Figure  S5 for the journal distribution). Using Ward’s method (Ward~Jr, 1963), we identified three country groups based on their selection of journals, where 21, 31, and 12 countries consist of each group, respectively. Group 1, comprising countries that publish predominantly in international journals with a low proportion of primary country, was followed by Group 2, comprising balanced publishing countries that publish in journals with a low to high proportion of primary country, and Group 3, comprising countries that publish predominantly in regional journals with a high proportion of primary country (Figure 3B). In contrast to the classification of many Asian countries as Group 3 (Figure 3C), European countries are predominantly categorized into Group 1 and Group 2. Group 3 comprises countries with the lower GDP, whereas Group 1 or Group 2 comprises countries with the higher GDP. Group 1 and Group 2 countries have respective mean GDPs $\simeq\$1.6\times 10^{12}$ and $\simeq\$1.1\times 10^{12}$, whereas Group 3 countries have an average GDP $\simeq\$5\times 10^{11}$. The fact that many high-GDP nations are classified as Group 1 or Group 2 suggests that these countries have a greater propensity to publish in international journals. In the academia of these countries, regional journals might not hold significant prominence for authors. In contrast, low-GDP nations depend on regional journals, which suggests the endorsement, support, and even indispensability of regional journals from the regional academia of such countries.

A common notion of international journals is their higher impact than regional journals. As a logical step, we measured the share of the primary and neighboring countries to its impact factor by the types of journals (Figure 4). We observed clear differences between questionable journals and other types of journals. For the unquestioned and other journals, those with a low share of primary and neighboring countries have a higher impact factor than those with a high share, aligning with the common notion. We observe a tail with a gradual decrease in journal impact by the share of the primary and neighboring countries for the unquestioned journals and other Scopus journals, however, questionable journals have a tiny number of journals in those tails. The questionable journals exhibit a low share of the primary and neighboring countries regardless of their journal impact. This observation is valid regardless of the definition of neighbors, which differs questionable from other types of journals. In closing, it is difficult to concur with the hypothesis of questionable journals contributing to regional academia, given the rarity of their function as regional journals.

The list of questionable journals, i.e., Beall’s list, have biased criteria for selecting questionable publishing; thus, it has inherent limitations for defining criteria. For example, Beall’s list is criticized for its negative outlook on open access (Krawczyk and Kulczycki, 2021). In addition, there are potentially questionable but non-listed questionable journals, yet we neglected them in the analysis. The publishing country of some questionable journals is purposefully misplaced for various reasons (Demir, 2018), but the main result, that is the low percentage of the primary country’s publishing, is independent of its declared publication location. Since this study focused on the massive behavior of questionable publishing, individual behavior, e.g., government policy for each country or greedy authors which use salami slicing or citation cartel (Kojaku et al., 2021), remains as further study.

We utilized a controlled experiment to examine questionable publishing of unquestioned journals at a comparable position in academia. One clear advantage of a controlled experiment is that one can compensate for the bias from the population. However, one should note that the Scopus dataset cannot cover all scientific publishing; therefore, the comparative analysis, with all Scopus journals, even can have potential bias (Mongeon and Paul-Hus, 2016). To surmount such a bias, we also presented the result of all other Scopus journals regardless of their academic positions and found that it shows similar collective behaviors with the unquestioned journals. In addition, to reduce selection bias, we restrict the interpretation to collective behavior rather than individual differences between journals. For instance, when comparing the publication rate of a questionable journal to that of an unquestioned journal, the result may vary depending on the degree of overall regionality of such journals. Consequently, the comparison can distinguish between questionable and unquestionable journals, at least in a collective manner.

The data of the list of questionable publishers and journals is public from beallslist.net. The Socpus data can be requested via Elsevier.

The authors declare no competing interests.

This research was supported by the MSIT (Ministry of Science and ICT), Republic of Korea, under the Innovative Human Resource Development for Local Intellectualization support program (IITP-2022-RS-2022-00156360) supervised by the IITP (Institute for Information & Communications Technology Planning & Evaluation). This work was also supported by the National Research Foundation of Korea (NRF) funded by the Korean government (grant No. NRF-2022R1C1C2004277 (T.Y.) and 2022R1A2C1091324 (J.Y.)). The Korea Institute of Science and Technology Information (KISTI) also supported this research with grant No. K-23-L03-C01 (J.P.) and by providing KREONET, a high-speed Internet connection.

All authors designed the experiment, collected the data, and wrote the manuscript. T.Y. analyzed the results. All authors reviewed, edited, and approved the manuscript.

A draft version of this paper was posted on arXiv: https://arxiv.org/abs/2312.07844