Enhancing the Evaluation System of Training Hospitals for Neurosurgical Residency Training and Education in South Korea : Striving for Balanced Participation and Differentiation

Article information

J Korean Neurosurg Soc. 2024;67(6):595-601
Publication date (electronic) : 2024 June 25
doi : https://doi.org/10.3340/jkns.2024.0067
1Department of Neurosurgery, Chungnam National University College of Medicine, Daejeon, Korea
2Department of Neurosurgery, Samsung Medical Center, Sungkyunkwan University School of Medicine, Seoul, Korea
3Department of Neurosurgery, Daegu Catholic University School of Medicine, Daegu, Korea
4Department of Neurosurgery, Hanyang University College of Medicine, Seoul, Korea
5Department of Neurosurgery, Korea University College of Medicine, Seoul, Korea
Address for reprints : Dae-Hyun Kim Department of Neurosurgery, Daegu Catholic University School of Medicine, 33 Duryugongwon-ro 17-gil, Nam-gu, Daegu 42472, Korea Tel : +82-53-650-4251, Fax : +82-53-650-4932, E-mail : daehkim@cu.ac.kr
Received 2024 March 18; Accepted 2024 June 24.

Abstract

The Korean Neurosurgical Society, with its 62 years of history, has witnessed substantial growth in the field of neurosurgery, producing over 3400 neurosurgeons, establishing 12 divisions and nine regional branches, and advancing in clinical management, diagnostic methods and academic research. Despite these developments, the regulations governing neurosurgical training and evaluation methods for training hospitals have remained largely unchanged, necessitating comprehensive revisions in response to evolving medical environments. To provide balanced participation opportunities for neurosurgery residents, the Korean Neurosurgical Society formed the Training Status Investigation Standard Change Task Force (TF team) under the Training Education Committee. This paper presents the TF team’s findings and proposals for revising training status investigation standards and evaluation criteria. Through the processes including a lot of team meetings, workshops, education programs, official communications with 12 division societies, benchmarking from other societies and analysis of encrypted data from the past 5 years for neurosurgical training hospitals, the TF team created a revised training status investigation proposal, supplemented main surgery criteria. And we applied this revised proposal to the training status investigation data collected from training hospitals in 2022 for simulation. We reduced the score for main surgeries to 10 points, introduced core competency surgery standards, allocating 5 points each for brain core competency surgery and spine and peripheral core competency surgery, for a total of 10 points. We also adjusted the major surgery score to 13 points, expanding the total surgery index score to 33 points. We introduced additional definitions for main surgeries in the areas of spine, pediatrics, and functional surgery. The equipment score was reduced from 17 to 9 points. We specified minimum requirements for resident allocation eligibility, and if a hospital meets all of these criteria, they become eligible to apply for resident allocation. We introduced a new bonus point system for hospitals performing mechanical thrombectomy or stenting and surgery for peripheral nerve diseases. The proposed revisions aim to improve the training and education of neurosurgical residents and overall neurosurgical care in Korea by creating a balanced and differentiated evaluation system for training hospitals. Further monitoring, communication, and adjustments are crucial for successful implementation.

INTRODUCTION

This year marks the 63rd anniversary of the Korean Neurosurgical Society (the society). Over the years, our society has produced around 3400 neurosurgeons, established 12 divisions, and eight regional branches. Not only have we seen quantitative growth, but also diverse surgical techniques and the development of new diagnostic equipment have led to the exploration of new clinical areas. The quality of academic research has also significantly improved. Furthermore, with nearly 90 training hospitals entrusted with the training and education of residency in neurosurgery, we are shaping the future of neurosurgery.

Despite the significant progress and changes our society has undergone in the past 60 years, the regulations for neurosurgical training and the evaluation methods for training hospitals have remained largely unchanged for a long time. Especially since the policies of reducing the number of residents that have been in effect since 2013 and the enactment of the Special Act for Resident Training in 2017, the training and medical environments have undergone significant changes. This has raised the need for a comprehensive revision of the regulations, guidelines, and evaluation criteria for neurosurgical training and training hospitals.

We must no longer recognize residents as cheap laborer but as learners who need education. We should create an environment in training hospitals where residents can participate in surgery, education, and research in a balanced manner. In a survey conducted among neurosurgery residents in Korea, this demand for change and innovation can also be confirmed [2].

To achieve this goal, it is necessary for the society to establish minimum competency standards in various areas of training hospitals. For this purpose, the Korean Neurosurgical Society has formed a team, known as the Training Status Investigation Standard Change Task Force (TF team), under the Training Education Committee. We aim to report the results of approximately 2 years of activities, from preliminary investigations to the revision of resident training regulations and changes in training status investigation items.

METHODS

Issues with the current training status investigation

The current training status investigation standards are categorized into clinical practice, education, and research, with 52 points allocated to clinical practice, 30 points to education, and 18 points to research out of a total of 100 points. Within the clinical practice category, apart from inpatient and surgical standards, the remaining 25 points are allocated to facilities, equipment, and medical records. Most hospitals receive the full 25 points in these areas, resulting in very weak discriminatory power in evaluations among hospitals. Therefore, adjustments in the allocation of points in the clinical practice category are necessary.

The allocation of 15 points for inpatient is excessively high, and main surgeries only include vascular and tumor surgeries, making them relatively overvalued compared to other types of surgeries. Additional definitions for main surgeries in areas other than vascular and tumor surgeries are needed.

Although the society has revised the annual resident training curriculum to ensure high-quality training in a rapidly changing training environment, the goals and final competencies of the annual resident training curriculum are not reflected in the evaluation of training hospitals (Table 1). Therefore, new evaluation criteria that reflect the annual resident training curriculum and final competencies are needed.

Final competencies for neurosurgery residency in Korea

Activities of the Training Status Investigation Standard Change Task Force (TF team)

To address these issues, the society established the Training Status Investigation Standard Change Task Force (TF team) in January 2021. We held a total of nine team meetings, three workshops for all members of the Training Education Committee of the society, and one joint workshop with responsible supervising neurosurgical specialist at training hospitals. Additionally, we conducted three education sessions for supervising neurosurgery specialists to gather input from society members.

We also sent official documents to 12 division societies to collect diverse opinions, benchmarked training status investigation items from other societies. We also analyzed encrypted data from the past 5 years of training status investigations and the past 3 years of surgical statistics for neurosurgical training hospitals.

Through these processes, we created a revised training status investigation proposal [4], supplemented surgery criteria (Table 2), and applied this revised proposal to the training status investigation data collected from training hospitals in 2022 for simulation.

Definitions of main surgery and core competency surgery

RESULTS

The direction of the new revisions includes 1) reflecting the goals and final competencies of the residency annual curriculum, 2) defining core competency surgeries in various areas such as stroke, head injury, spine, and peripheral nerve areas for evaluation, 3) adjusting the scoring system by redistributing points for criteria with insufficient differentiation, 4) specifying the minimum requirements for obtaining eligibility to apply for resident allocation, and 5) introducing a bonus point system reflecting recent trends in the medical field. Ultimately, our aim is to create an evaluation system that encourages balanced participation in clinical practice, education, and research by offering training in various areas.

Revisions

Clinical practice

The existing surgical index score consisted of main surgeries (15 points) and major surgeries (10 points) for a total of 25 points. We reduced the score for main surgeries to 10 points, introduced core competency surgery standards, allocating 5 points each for brain core competency surgery and spine and peripheral core competency surgery, for a total of 10 points. We also adjusted the major surgery score to 13 points, expanding the total surgery index score to 33 points. Previously, only vascular and tumor surgeries were considered main surgeries, but we introduced additional definitions for main surgeries in the areas of spine, pediatrics, and functional surgery. The previous equipment score was 17 points, but as most training hospitals achieved this score, we reduced it to 9 points.

We specified minimum requirements for resident allocation eligibility, and if a hospital meets all of these criteria, they become eligible to apply for resident allocation.

We introduced a new bonus point system. Hospitals performing mechanical thrombectomy or stenting more than 50 cases annually receive 1 point, and those performing it more than 25 cases receive 0.5 points. Hospitals performing peripheral nerve surgery more than 50 cases annually receive 1 point, and those performing it more than 25 cases receive 0.5 points.

Education

Previously, the results of the residency mid-term exam were differentially reflected in a relative evaluation, but we changed it to an absolute evaluation and set 60 points as the pass/fail threshold. We also considered the ratio of the number of residents subject to the exam and the number of residents who passed when calculating the score. We reduced the academic activity score from 8 to 6 points and, instead, introduced a new standard for evaluating the resident’s training plan and assigned 2 points.

The research category retained its existing evaluation standards.

Application of 2022 training status investigation data

When evaluated based on the existing training status investigation scoring table, the average total score for 85 training hospitals was 89.4 points. However, when evaluated using the revised standards, the average score decreased to 85.3 points, marking a decrease of 4.1 points. It is noteworthy that the total score declined in 91.8% of all training hospitals. Examining the scores by category, the average score for major surgeries across all training hospitals increased from 9.3 to 10.9 points, while the score for main surgeries dropped from 13.6 to 7.8 points. The average scores for newly introduced brain core competency surgeries and spine and peripheral core competency surgery were 4.0 and 3.9 points, respectively.

Among the 85 training hospitals, 46 hospitals obtained bonus points, with 44 hospitals receiving vascular bonus points and three hospitals receiving peripheral bonus points. Although hospitals could obtain a maximum of 2 bonus points, no hospital exceeded 1 point.

Out of the 85 training hospitals, 74 met the minimum requirements to apply for the allocation of residents. Among the hospitals that failed to obtain the qualifications for resident allocation, the most common reason was the failure to meet the minimum criteria for spine and peripheral core competency surgery, with a total of six hospitals falling into this category. Additionally, four hospitals had a total score of less than 70 points, making it the second most common reason for failing to meet the minimum requirements. The reasons for the 11 hospitals that failed to meet the minimum criteria are listed in Table 3.

Reasons for failure to apply for residency allocation, resulting from 2022 training status investigation data

DISCUSSION

When comparing the results of the 2022 Training Status Investigation Data with the virtual assessment based on the revised criteria, the average total score for 85 training hospitals was 89.4 points according to the original criteria. However, when evaluated using the revised criteria, the average score decreased to 85.3 points, representing a decrease of 4.1 points. Remarkably, 91.8% of hospitals experienced a decrease in their total scores. Examining the scores by category, the standard score for major surgeries increased from 10 to 13 points, the overall average score for all training hospitals rose from 9.3 to 10.9 points. Conversely, the standard score for main surgeries decreased from 15 to 10 points causing the average score for training hospitals to drop from 13.6 to 7.8 points. The average scores for newly introduced brain core competency surgeries and spine and peripheral core competency surgery were 4.0 and 3.9 points, respectively.

The decline in the average total score in the revised evaluation was an expected outcome due to the significant reduction in the weighting of equipment scores, which had weak discriminatory power in the existing evaluation criteria. Additionally, the introduction of core competency surgery indicators and adjustments to the weighting of major surgery scores contributed to the overall decrease. The standard deviation of the total score in clinical practice and the standard deviation of the total score increased from 3.07 to 5.09 and 7.03 to 8.56, respectively which means that the discriminatory power of the new revision has improved. When considering the goal of creating a differentiation system for training hospitals, this result can be interpreted as relatively successful.

Among all training hospitals, one hospital experienced the most significant increase in total score, which increased by 13.5 points, while another hospital saw the most substantial decrease, with a decline of 16 points. The hospital with the highest score increase performed a wide range of surgeries, including main surgeries, major surgeries, brain core competency surgeries and spine and peripheral core competency surgeries. In contrast, the hospital with the largest score decrease had fewer inpatient cases and fewer surgeries. This trend aligns with the revised criteria’s aim to encourage diversity in the experiences of residents during their training period.

To reflect recent trends in the medical field, enhance competitiveness with relevant societies, and encourage support and investment, bonus points were introduced. Bonus points, up to a maximum of 1 point for endovascular surgery such as mechanical thrombectomy and stenting and a maximum of 1 point for peripheral nerve surgery, were established, allowing each hospital to obtain a maximum of 2 bonus points. While a score of 2 points may seem small out of a total of 52 points in the category of clinical practice, considering the standard deviation of 3.07 points for the existing clinical practice total score and 5.1 points for the revised criteria’s clinical practice total score among all training hospitals, it can be considered an attractive score. In the simulated analysis of the 2022 Training Status Investigation Data, 46 out of 85 training hospitals received bonus points, with 44 hospitals receiving vascular bonus points and three hospitals receiving peripheral bonus points. Although no hospitals obtained more than 1 point in bonus points, it is anticipated that more training hospitals will be able to acquire bonus points once the system is established.

Through a simulated analysis of the 2022 Training Status Investigation Data, we examined the reasons for non-compliance with the minimum requirements for obtaining eligibility to apply for residency positions in 11 training hospitals. The majority of these hospitals narrowly missed the minimum requirements for residency application, making it likely that they can improve their deficiencies with a little effort during the approximately 2-year grace period. For example, among the reasons for non-compliance, the most common was failure to meeting the annual 50 surgery requirement for core competency surgery in spine and peripheral surgery, with six hospitals falling into this category. However, the average number of core competency surgeries in spine and peripheral surgery for the top three hospitals among these six hospitals was 41, indicating that they can likely meet the minimum requirements during the grace period.

The overall decline in the total scores of the Training Status Investigation was the second most common reason for failing to meet the minimum requirements for applying for residency allocation. This total scores also serve as index criteria designated by the Korean Hospital Society Training Evaluation Committee, encompassing the minimum scores for each subspecialty training hospital and the allocation of residents. Initially, concerns arose that more hospitals would fail to meet these index criteria. Fortunately, the simulated analysis revealed that no hospital failed to reach the minimum score for training hospital designation, which is 60 points. Additionally, only four hospitals failed to obtain 70 points which is the minimum score for eligibility to apply for residency allocation, and all four hospitals received 67 points, indicating that this threshold is sufficiently achievable.

Certain training hospitals, specializing in oncological diseases, were able to achieve high scores when applying the original scoring criteria, primarily due to a high volume of main and major surgeries. However, when applying the newly revised scoring criteria, they fell short of meeting the minimum requirements for core competency surgery in spine and peripheral nerve surgery, thereby losing eligibility for applying for residency positions. Up until now, this issue has been addressed through the rotation training of residents to other hospitals. However, as the Training Status Investigation assesses the capabilities of each training hospital, a focus on enhancing capacity to meet minimum requirements independently is required in the future.

So far, the Korean Neurosurgical Society has assigned many residents to hospitals that perform more surgeries, especially cerebrovascular surgeries and oncology surgeries, regardless of the type of surgery. However, there are not many specialists who actually work in cerebrovascular and oncology field after completing the residency training. Residents are important human resources in hospitals, but we must recognize that they are learners who must be trained to become competent specialists who will take on the future of neurosurgery. It is necessary to create a training hospital environment that encourages residents to develop various core competencies through diverse experiences in various areas, and to establish a performance-based evaluation system that ensures residents receive training in a standardized training environment [3]. It is also necessary to develop a systematic competency diagnosis procedure and tools to establish a training system that can improve the effectiveness of education and training by finding out what individual capabilities are lacking and providing a variety of education and training programs that can supplement them [1]. Our society must continue to strive to create a training system that helps residents to grow into excellent specialists no matter where they train, by changing the training status investigation standards, revising resident training regulations, and establishing a systemized annual training curriculum for residents.

The results of the simulation suggest that the proposed revisions to the training status investigation standards may lead to better discrimination among training hospitals, encouraging more balanced participation in clinical practice, education, and research.

However, several issues still need to be addressed. First, we need to establish a system for providing feedback to training hospitals based on the results of the training status investigation. Second, we need to continue monitoring the situation and make further revisions as necessary based on the actual data collected in 2022 and 2023. Third, we need to ensure that these changes are effectively communicated to all members of the society, including supervising physicians and residents.

CONCLUSION

TF team has proposed revisions to the training status investigation standards with the aim of creating a more balanced and differentiated evaluation system for training hospitals. These changes reflect the goals and final competencies of the residency annual curriculum, introduce core competency surgeries, adjust the scoring system, specify minimum requirements for resident allocation eligibility, and introduce a bonus point system.

The results of the simulation suggest that these changes may lead to better differentiation among training hospitals and encourage balanced participation in clinical practice, education, and research. However, further monitoring and adjustments are necessary, and effective communication of these changes to all members of the society is crucial.

We hope that these revisions will contribute to the improvement of the training and education of neurosurgical residents and the overall quality of neurosurgical care in Korea.

Notes

Conflicts of interest

No potential conflict of interest relevant to this article was reported.

Informed consent

This type of study does not require informed consent.

Author contributions

Conceptualization : DHK; Data curation : KK; Formal analysis : KSC; Funding acquisition : JWH; Methodology : SWC; Project administration : SHL; Visualization : SWC; Writing - original draft : SWC; Writing - review & editing : DHK

Data sharing

None

Preprint

None

Acknowledgements

The Training Status Investigation Item Change Task Force (TF team) would like to thank all members of the Korean Neurosurgical Society, especially the Training Education Committee and responsible supervising specialists at training hospitals, for their valuable input and support in the development of these revisions. We would also like to express our gratitude to the division societies and all those who participated in the workshops and training sessions for their feedback and insights.

This work was supported by the Korean Neurosurgical Society.

References

1. Morcke AM, Dornan T, Eika B. Outcome (competency) based education: an exploration of its origins, theoretical basis, and empirical evidence. Adv Health Sci Educ Theory Pract 18:851–863. 2013;
2. Nasca TJ, Philibert I, Brigham T, Flynn TC. The next GME accreditation system--rationale and benefits. N Engl J Med 366:1051–1056. 2012;
3. Park HR, Park SQ, Park HK, Im SB, Yoon SM, Kim BT. Current training status of neurosurgical residents in South Korea: a nationwide multicenter survey. World Neurosurg 131:e329–e338. 2019;
4. The Korean Neurosurgical Society : Neurosurgical Residency Training Regulations of Korea. Available at : https://www.neurosurgery.or.kr/about/file/rule3_230223.pdf.

Article information Continued

Table 1.

Final competencies for neurosurgery residency in Korea

A) Systemic management and treatment of neurosurgical (intermediate) patients
B) Diagnosis and treatment of cerebrovascular diseases
C) Diagnosis and treatment of brain and spinal cord tumors
D) Diagnosis and treatment of various spinal diseases, including congenital, degenerative, and traumatic
E) Diagnosis and treatment of traumatic brain injuries
F) Diagnosis and treatment of functional central nervous system disorders
G) Diagnosis and treatment of degenerative central nervous system diseases
H) Diagnosis and treatment of pain
I) Diagnosis and treatment of peripheral nervous system disorders and associated musculoskeletal diseases

Table 2.

Definitions of main surgery and core competency surgery

Category Subcategory Type of operation Disease or operation
Main surgery Vascular Open surgery For aneurysm, AVM, Moyamoya disease, bypass surgery, endarterectomy, spinal AVM
Endovascular For aneurysm, AVM, spinal AVM
Radiosurgery For vascular disease
Tumor Tumor surgery Craniotomy, TSA, removal of spinal cord (intradural) tumor
Radiosurgery For tumor
Spine Corpectomy For OPLL, spinal tumor, deformity or fracture
Pediatric Cranio-spinal dysraphism For encephalocele, myelocele, meningomyelocele, lipomyelomeningocele, tethered cord syndrome
Functional Open epilepsy surgery, DBS, MVD
Core competency surgery Brain Craniotomy, craniectomy For hematoma, depressed fracture, infartion, infection
Stereotaxic ICH removal
Trephination, EVD
Shunt Ventriculoperitoneal, ventriculoatrial, cystoperitoneal, thecoperitoneal, etc.
Endovascular Thrombectomy, stent
Spine/periphral Laminecotmy, laminoplasty, foraminotomy
Discectomy Microscopic, endoscopic
Spine tumor Extradural
Instrumentation, fusion
Operations for entrapment syndrome, peripheral nerve tumor, neurorrhaphy, nerve graft

AVM : arteriovenous malformation, TSA : transsphenoidal approach, OPLL : ossification of posterior longitudinal ligament, DBS : deep brain stimulation, MVD : microvascular decompression, ICH : intracerebral hemorrhage, EVD : external ventricular drainage

Table 3.

Reasons for failure to apply for residency allocation, resulting from 2022 training status investigation data

Reason for failure No. of hospital
Less than 70 points of total scores 4
Less than 300 major surgeries 3
Less than 50 main surgeries 3
Less than 50 brain core competency surgeries 2
Less than 50 spine or peripheral core competency surgeries 6
Less than 400 hospitalized patients 1
Total No. of hospitals that failed to qualify for residency allocation 11