The production of academic journal depends on the voluntary contributions of reviewers [4]. Given the status of Journal of Korean Neurosurgical Society (JKNS), constructive reviews of high-quality are of utmost importance [5]. Unlike top-ranked journals that select the best from well-prepared manuscripts, most submissions to JKNS require significant improvements, akin to polishing a rough stone into a jewel. This is why JKNS features photographs of the best reviewers on the cover of the first issue every 2 years.

This article provides a more detailed assessment of review statistics compared to previous editorials, including the results of reviewer evaluation system initiated in 2023 and the impact of the reviewer merit system introduced in 2024. The aim is to gain insights on how to improve the review process and enhance the progress of JKNS.

Review data was analyzed using on-line submission data for the last 2 years, from January 2023 to December 2024. The number of reviewers enlisted in the system, the number of reviewers who received review requests, the number of completed reviews, and data on review period were collected.

The data on review quality scores, judged by editors, were also collected for each reviewer. The editor assessed the quality of reviews as very good (score 3), good (score 2), and fair (score 1) for each reviewer’s comments. This system was launched in 2023 and became mandatory for editors in 2024.

Data on reviewer merit system, launched in January 2024, were also collected. Reviewers who completed one manuscript review within the due time received 3 points for domestic reviewers and 5 points for international reviewers. When they accumulated 15 points, they could use them to deduct the article processing charge (APC; 400 US dollar) for one manuscript in which they participated as corresponding authors.

From January 1, 2023 to December 11, 2024, 479 manuscripts were submitted. Among the total of 768 reviewers enlisted in the on-line submission system of JKNS, 357 reviewers were assigned and 227 reviewers submitted opinions for at least one manuscript.

The highest number of reviewers who reviewed one manuscript was 96, and the number of reviewers who reviewed more manuscripts decreased as the number of manuscripts increased. The median number of reviewed manuscripts was 2 (Fig. 1).

The reviewers were divided into two groups. Based on the data in Fig. 1, reviewers who completed four or fewer manuscript reviews were assigned to group 1, while those who reviewed five or more manuscripts were assigned to group 2. This grouping was made in order to identify differences between the two groups. Most reviewers (196, 86.3%) were in group 1, while only a small portion (31, 13.7%) were in group 2. About two-thirds of the reviews were done by group 1, while group 2 reviewed three times more manuscripts per reviewer (Table 1). Among the 31 reviewers in group 2, only one was an international reviewer (Table 2).

Although the review period was shorter in group 2, the difference was not statistically significant. The time from the review request to acceptance was significantly shorter in group 2. The review scores evaluated by the editors showed no significant difference between the two groups (Table 3).

The JKNS best reviewers in 2023 and 2024 were selected the on the number of completed reviews, the period of review, and the content of the reviews, consistent with previous years [2,6]. We have introduced the reviewers below with our honor (Fig. 2).

The mean value of points in the reviewer merit system was 9.2±7.1 (mean±standard deviation) for domestic reviewers, 6.8±4.7 for international reviewers. Four manuscripts were exempted from the APC by the reviewer merit system, all of them from Korea. Although it was found that four international reviewers accumulated more than 12 points, none of them used it for APC exemption.

Reviewers who received at least one review request in the past 2 years can be defined as active reviewers. Applying this criterion, only 46% of the total reviewers (357 out of 768) are active reviewers. Another concern is the rate of completed reviews; about 37% (130 out of 357) of review requests are declined by reviewers or canceled by editors due to delayed responses. The low proportion of active reviewers is an issue that needs to be addressed in the future. It is desirable to exclude reviewers who have not been active for a long time and to fill the positions with newly active reviewers. Another concern is the considerable proportion of incomplete review requests. This indicator deteriorated from 75% to 63% compared to the period 2021-2022 [6]. This deterioration is likely due to the increased workload of the specialists resulting from the mass resignation of residents in 2024.

Approximately 250 manuscripts are submitted to the JKNS annually. Usually, two to three reviewers are selected for each paper, assuming 2.5 reviewers, requiring 630 reviews. If each reviewer reviews three papers per year, there should be no significant issues in managing submitted manuscripts with 250 reviewers who promptly accept review requests and complete reviews within the period. Although many of these numbers are based on assumptions, it is expected that there will be no significant difficulties in managing the review process with the current number of active reviewers even if the total number of reviewers decreases.

The evaluation system for reviewers was launched to select the best reviewers based on the quality of reviews, rather than just the quantity or promptness of reviews. It began in 2023, but initially, the number of editors who completed the assessment was very low. Since the evaluation system became mandatory in 2024, data has been accumulated, so it did not have a significant impact on the selection of best reviewers this time. In order to evaluate the quality of reviews more objectively, it is worth considering presenting more specific criteria for each evaluation score. The lack of difference in review score between group 1 and group 2 needs to be verified with additional data. It might be a real feature or masking of differences due to insufficient valid data. There have been reports on the characteristics of reviewers who produce good-quality reviews [3]. JKNS should also make efforts to find such characteristics when sufficient data from reviewer evaluation system is accumulated.

It is noteworthy that the reviewers who completed more manuscript reviews had a significantly shorter time to accept review requests. The time period between sending a review request and its acceptance is wasted time, as the review period is counted from the acceptance of the request. It is necessary to consider ways to further improve this value. Another concern is the considerable rate of declined review requests. The high rate of decline might hinder rapid processing due to the time consumed in assigning other reviewers. It is necessary to establish new criteria for selecting the best reviewers, including stratification of the review period, a low rate of decline, and high reviewer scores, referring to those of other journals [1].

The reviewer merit system was created to reward reviewers, enhance the participation of international reviewers and promote them as potential authors. Considering the small number of articles exempted from the APC and the paucity of international reviewers in group 2, the reviewer merit system has not proved its usefulness. Recruiting more active international reviewers is one of the concerns to be addressed.

The progress of an academic journals requires continuous and unceasing efforts. In a competitive environment where all academic journals strive for better reputations, stagnation means regression. We should not be satisfied with the achievements made so far and stop at maintaining the current system, but seek new ways to further progress. Recruiting good reviewers, fairly evaluating their efforts, and providing appropriate rewards for their contributions are very important factors in achieving the progress of JKNS.