search for




 

Comparative accuracy of four guidelines to predict high-grade dysplasia or malignancy in surgically resected pancreatic intraductal papillary mucinous neoplasms: Small nuances between guidelines lead to vastly different results
Ann Hepatobiliary Pancreat Surg 2024 Nov;28(4):483-93
Published online November 30, 2024;  https://doi.org/10.14701/ahbps.24-049
Copyright © 2024 The Korean Association of Hepato-Biliary-Pancreatic Surgery.

Irene C. Perez1, Andrew Bigelow2, Vanessa M. Shami2, Bryan G. Sauer2, Andrew Y. Wang2, Daniel S. Strand2, Alexander J. Podboy2, Todd W. Bauer3, Victor M. Zaydfudim3, Allan Tsung3, Ross C. D. Buerlein2

1Department of Internal Medicine, University of Virginia Health System, Charlottesville, VA, USA,
2Division of Gastroenterology and Hepatology, University of Virginia Health System, Charlottesville, VA, USA,
3Division of Surgical Oncology, University of Virginia Health System, Charlottesville, VA, USA
Correspondence to: Ross C. D. Buerlein, MD
Division of Gastroenterology and Hepatology, Digestive Health Center, University of Virginia Health System, Main Floor 1215 Lee St, Charlottesville, VA 22903, USA
Tel: +1-434-243-2090, Fax: +1-434-244-9445, E-mail: rcb9n@uvahealth.org, ORCID: https://orcid.org/0000-0002-1033-9783

Irene C. Perez’s current affiliation: Division of Gastroenterology and Hepatology, University of Colorado, Anschutz Medical Campus, Aurora, CO, USA.
Received February 27, 2024; Revised April 28, 2024; Accepted April 30, 2024.
This is an Open Access article distributed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/4.0) which permits unrestricted non-commercial use, distribution, and reproduction in any medium, provided the original work is properly cited.
 Abstract
Backgrounds/Aims: The guidelines regarding the management of intraductal papillary mucinous neoplasms (IPMNs) all have slightly different surgical indications for high-risk lesions. We aim to retrospectively compare the accuracy of four guidelines in recommending surgery for high-risk IPMNs, and assess the accuracy of elevated CA-19-9 levels and imaging characteristics of IPMNs considered high-risk in predicting malignancy or high-grade dysplasia (HGD).
Methods: The final histopathological diagnosis of surgically resected high-risk IPMNs during 2013−2020 were compared to preoperative surgical indications, as enumerated in four guidelines: the 2015 American Gastroenterological Association (AGA), 2017 International Consensus, 2018 European Study Group, and 2018 American College of Gastroenterology (ACG). Surgery was considered “justified” if histopathology of the surgical specimen showed HGD/malignancy, or there was postoperative symptomatic improvement.
Results: Surgery was postoperatively justified in 26/65 (40.0%) cases. All IPMNs with HGD/malignancy were detected by the 2018 ACG and the combined (absolute and relative criteria) 2018 European guidelines. The combined (“high-risk stigmata” and “worrisome features”) 2017 International guideline missed 1/19 (5.3%) IPMNs with HGD/malignancy. The 2015 AGA guideline missed the most cases (11/19, 57.9%) of IPMNs with HGD/malignancy. We found the features most-associated with HGD/malignancy were pancreatic ductal dilation, and elevated CA-19-9 levels.
Conclusions: Following the 2015 AGA guideline results in the highest rate of missed HGD/malignancy, but the lowest rate of operating on IPMNs without these features; meanwhile, the 2018 ACG and the combined (absolute and relative criteria) 2018 European guidelines result in more operations for IPMNs without HGD/malignancy, but the lowest rates of missed HGD/malignancy in IPMNs.
Keywords : Pancreatic intraductal neoplasms; Pancreatic cyst; CA-19-9 antigen
INTRODUCTION

Intraductal papillary mucinous neoplasms (IPMNs) are pre- malignant pancreatic cysts with varying degrees of malignant potential [1-3]. These lesions are frequently found incidentally in asymptomatic patients, and partly due to the increasing utilization of cross-sectional imaging with improved resolution and an increasingly aging population, their prevalence is thought to be increasing over time (upwards of 70% in some studies) [4-6]. Numerous guidelines exist regarding pancreatic cysts, and when focusing on IPMNs, their main goal is to help determine if a lesion should enter a surveillance program or undergo surgical resection, given the presence or absence of features considered high-risk for containing underlying high-grade dysplasia (HGD) or malignancy [7-9]. Despite extensive research in this area [10-12], the optimal management of IPMNs remains controversial, while the efficacy of the various guidelines remains unclear [13].

The ideal set of guidelines would identify all patients with IPMNs that contain HGD/malignancy, while simultaneously avoiding surgery on patients with IPMNs that do not [14]. The nuances and variations among the different guidelines is a source of confusion for clinicians, which can lead to disparate outcomes that include costs, mortality (from both malignant transformation and operative complications), sensitivity regarding the detection of malignancy, and the rate at which surgery is recommended [15-17]. As such, the clinical implications of following one guideline over another remain unclear.

In this current study, we aim to 1) retrospectively compare the clinical utility of four major guidelines: the 2015 American Gastroenterological Association (AGA) [18], 2017 International Consensus [19], 2018 American College of Gastroenterology (ACG) [3], and 2018 European Group [20], in recommending surgical resection of IPMNs based on the histopathology of surgically resected lesions and postoperative symptom improvement, and 2) assess the accuracy of CA-19-9 levels and the preoperative imaging characteristics of IPMNs considered high-risk in predicting malignancy or HGD.

MATERIALS AND METHODS

This study (16396) was approved by the Institutional Review Board (IRB) for Health Sciences Research at the University of Virginia, and deemed exempt from IRB review. Informed consent was not obtained from subjects. All patients who underwent surgical resection for IPMNs in the period 2013−2020 were identified in the prospectively maintained institutional database. Preoperative diagnosis of IPMN and cyst size and features were established based on cross-sectional imaging and/or endoscopic ultrasound (EUS) (if multiple cysts were present in one patient, the largest cyst was used for the purposes of this study). Surgery was performed by one of three pancreatic surgeons, all of whom used the 2012 or 2017 International Consensus guidelines in clinical practice. Indications for surgery based on these guidelines included the presence of either an enhancing mural nodule ≥ 5 mm in size, main pancreatic ductal dilation ≥ 10 mm in size, or obstructive jaundice due to cyst. In the cases of “worrisome features”—like pancreatitis, cyst size ≥ 3 cm, main pancreatic duct size 5−9 mm, abrupt change in the caliber of the pancreatic duct with distal parenchymal atrophy, cyst growth rate ≥ 5 mm per 2 years—the cases were preoperatively reviewed in a multidisciplinary hepatopancreatobiliary tumor board conference (which is comprised of radiologists, pathologists, oncologists, surgical oncologists, interventional endoscopists, and radiation oncologists), and a determination to operate was made based on a combination of the above considerations and the patient’s individual surgical candidacy. Patients with pancreatic cystic neoplasms other than IPMNs were excluded from the study.

For the purposes of this study, a surgery was considered “justified” if surgical pathology demonstrated HGD (also referred to as carcinoma in situ) or malignancy (invasive adenocarcinoma), or if there was symptomatic improvement (abdominal pain secondary to pancreatitis or obstructive jaundice resolved, or no more episodes of pancreatitis or obstructive jaundice during the timeframe of the study) post-surgery. Symptomatic improvement was determined by postoperative clinic visit documentation. In the cases in which the IPMN was considered the etiology of pancreatitis, all other potential etiologies of pancreatitis had been clinically excluded.

Table 1 lists all the criteria for recommending surgical resection of each of the four guidelines. The 2015 AGA guideline recommends surgery for asymptomatic patients if either of the two listed criteria are met, and for the purposes of this study the guideline was applied to all patients. The 2017 International Consensus guideline recommends surgery if any of the “high-risk stigmata” are present. This guideline also provides multiple “worrisome features” that, if present in combination with any of the specifically mentioned EUS features (mural nodule ≥ 5 mm, obvious main duct involvement, or fine-needle aspiration (FNA) with HGD/malignancy), lead to a recommendation for surgery. For the purposes of this study, the “high-risk stigmata” and “worrisome features” were assessed both separately, and then combined. The 2018 ACG guideline provides a list of IPMN features for which the patient should be referred to a multidisciplinary group, and for the purposes of this study, these were all considered surgical indications. Lastly, the 2018 European guideline has a list of absolute criteria for which surgery is recommended. This guideline also provides a list of relative criteria for which surgery is recommended in surgically-fit patients. For the purposes of this study, the absolute and relative criteria were assessed both separately, and then combined.

Table 1 . Criteria for surgical resection of intraductal papillary mucinous neoplasm (IPMN) per guideline

2015 American Gastroenterological Association
If any of the following criteria is present (intended for asymptomatic patient), surgical referral recommended:
1. Both a solid component AND a dilated pancreatic duct
2. FNA with HGD/malignancy
2017 International Consensus
“High-risk stigmata”
If any of the following criteria is present, surgical referral recommended:
1. Obstructive jaundice due to cyst
2. Enhancing mural nodule (≥ 5 mm)
3. Main pancreatic duct dilation ≥ 10 mm
“Worrisome features”
1. Pancreatitis
2. Cyst ≥ 3 cm
3. Enhancing mural nodule < 5 mm
4. Thickened/enhancing cysts walls
5. Main pancreatic duct size 5−9 mm
6. Abrupt change in caliber of the pancreatic duct with distal pancreatic atrophy
7. Lymphadenopathy
8. Increased serum level of CA-19-9 (presumed to be > 37 U/mL)
9. Cyst growth rate ≥ 5 mm/2 yr
If any of the above “worrisome features” and any of the features below are seen on EUS, then surgery should be considered (otherwise, radiographic surveillance recommended):
a) EUS with definite mural nodule (≥ 5 mm). Definite nodule has lack of mobility, presence of Doppler flow, and FNA of nodule showing tumor tissue.
b) EUS with main duct features suspicious for involvement that include the presence of any one of thickened walls, intraductal mucin, or mural nodules
c) FNA with HGD/malignancy
2018 American College of Gastroenterology
If any of the following criteria is present, referral to a multidisciplinary group for surgical consideration is recommended:
1. Jaundice secondary to cyst
2. Acute pancreatitis secondary to cyst
3. Significantly elevated CA-19-9 (presumed to be > 37 U/mL)
4. Mural nodule or solid component either within the cyst or pancreatic parenchyma (no size for mural nodule is given)
5. Main pancreatic duct dilation > 5 mm
6. Focal dilation of pancreatic duct concerning for main-duct IPMN or an obstructing lesion
7. Cyst ≥ 3 cm
8. FNA with HGD/malignancy
9. Rapid increase in cyst size (> 3 mm per annum)
2018 European Study Group
”Absolute criteria”
If any of the following criteria is present, surgical referral recommended:
1. FNA with HGD/malignancy
2. Solid component
3. Jaundice secondary to tumor
4. Enhancing mural nodule (≥ 5 mm)
5. Main pancreatic duct dilation ≥ 10 mm
“Relative criteria”
If any of the following criteria is present, surgical referral should be considered:
1. Grow-rate ≥ 5 mm per annum
2. Increased levels of CA-19-9 > 37 U/mL in the absence of jaundice
3. Main pancreatic duct dilation between 5−9.9 mm
4. Cyst diameter ≥ 40 mm
5. New-onset diabetes mellitus
6. Acute pancreatitis (caused by IPMN)
7. Enhancing mural nodule (< 5 mm)

FNA, fine-needle aspiration; HGD, high-grade dysplasia; EUS, endoscopic ultrasound.



Statistical analysis

Continuous variables and categorical data were reported as the median, frequency, or percentage, where appropriate. Two-tailed Fisher’s exact test (with p ≤ 0.05 indicating statistical significance) was used for statistical analysis with the SPSS Statistics software (ver. 29.0.1.0; IBM Corp.). Standard formulaic calculations were performed to determine the sensitivity, specificity, positive predictive value, and negative predictive value for each of the guidelines.

RESULTS

A total of 65 patients were included in the study: 26 (40.0%) were considered “justified” with postoperative pathology showing HGD/carcinoma or symptom improvement, while 39 (60.0%) without these features were considered “unjustified”. Table 2 presents the baseline clinical characteristics. The median age of patients that were considered “justified” and those “unjustified” measured as a cumulative median from all the guidelines was 67 ± 0.96 years (“justified”), and 72 ± 0.73 years (“unjustified”), respectively. Forty-two patients (64.6%) had branch-duct IPMNs (BD-IPMNs), 8 patients (12.3%) had main-duct IPMNs (MD-IPMNs), and 15 patients (23.1%) had mixed-type IPMNs. Sixteen patients (24.6%) either had pancreatitis (20.0%) attributed to the IPMN (in these cases, other etiologies of pancreatitis had been excluded, and the size/location of the IPMN was clinically felt to be a risk for pancreatitis related to mucous plugging of the duct), or were considered “symptomatic” with jaundice (4.6%) or new-onset diabetes (1.5%).

Table 2 . Patient and intraductal papillary mucinous neoplasm charac­teristics (n = 65)

CharacteristicValue
Age (yr)69 ± 9
Male32 (49)
White58 (89)
Surveillance time (time from the initial image to surgery), mo14 ± 21
Symptomatic
Jaundice2
Acute pancreatitis12
New-onset diabetes mellitus1
Jaundice and acute pancreatitis1
IPMN
Main-duct8
Branch-duct42
Mixed (main- and branch-duct)15
Benign or low-grade dysplasia46
High-grade dysplasia11
Invasive carcinoma or adenocarcinoma8

Values are presented as mean ± standard deviation, number (%) or number only.



A total of 19 resected lesions were consistent with HGD (n = 11) or invasive carcinoma (n = 8). Four patients with adenocarcinoma had involved locoregional lymph nodes.

Primary outcomes

The number of cases retrospectively considered “justified” for surgery was 26 of 65 (40.0%); 19 cases had HGD/malignancy, and 16 cases had resolution of symptoms (resolution of jaundice/pain, no additional episodes of pancreatitis) post-surgery. Of note, 9 of the symptomatic 16 cases also had HGD/malignancy.

Table 3 shows the number of cases (out of 65) for which the guidelines recommended surgery, which are also listed below:

Table 3 . Comparison of cases that met criteria for surgical resection, “justified” cases, and cases of missed HGD/malignant intraductal papillary mucinous neoplasms per guideline

GuidelineNo. of cases (n = 65) where surgery was recommendedNo. of cases in which surgery was “justified”a)No. of missed cases with HGD/malignancy (n = 19)b)
2018 ACG60 (92.3)26 (43.3)0 (0)
2015 AGA20 (30.8)10 (50.0)11 (57.9)
2018 Euro. absolute criteria33 (50.8)19 (57.6)4 (21.1)
2018 Euro. (combined)59 (90.8)26 (44.1)0 (0)
2017 Int. “high-risk stigmata”36 (55.4)19 (52.8)4 (21.1)
2017 Int. (combined)62 (95.4)25 (40.3)1 (5.3)

HGD, high-grade dysplasia; ACG, American College of Gastroenterology; AGA, American Gastroenterological Association; Euro. (combined), European combined absolute and relative criteria; Int. (combined), International combined “high-risk stigmata” and “worrisome features”.

a)Postoperative pathology was consistent with HGD or malignancy or patients had symptomatic resolution.

b)This column indicates the number of cases for which the guidelines would not have recommended surgery but postoperative pathology showed HGD/malignancy, indicating the guideline would have missed this lesion.



- 2018 ACG: 60 (92.3%)

- 2015 AGA: 20 (30.8%)

- 2018 European, absolute criteria: 33 (50.8%)

- 2018 European, absolute and relative (combined) criteria: 59 (90.8%)

- 2017 International, “high-risk stigmata”: 36 (55.4%)

- 2017 International, “high-risk stigmata” and “worrisome features”: 62 (95.4%)

Of the cases for which the guideline recommended surgery, surgery was postoperatively considered “justified” in:

- 2018 ACG: 26 (43.3%)

- 2015 AGA: 10 (50.0%)

- 2018 European, absolute criteria: 19 (57.6%)

- 2018 European, absolute and relative (combined) criteria: 26 (44.1%)

- 2017 International, “high-risk stigmata”: 19 (52.8%)

- 2017 International, “high-risk stigmata” and “worrisome features”: 25 (40.3%)

Table 4 summarizes the rates of surveillance and surgery that were “justified”, while Table 5 summarizes the sensitivity, specificity, positive predictive value, and negative predictive value when each guideline was retrospectively applied to the patients undergoing surgical intervention for IPMNs in this study. All HGD/malignant IPMNs (total of 19) were accurately detected when applying the 2018 ACG guideline, and when applying the combined absolute and relative criteria of the 2018 European guideline. The 2017 International Consensus guideline missed 1 of 19 cases (5.3%) when applying the criteria of “high-risk stigmata” and “worrisome features”, while the 2015 AGA guideline missed 11 of 19 cases (57.9%). Of note, when only applying the AGA guideline strictly to asymptomatic cases with HGD/malignancy (original guideline indication), the cases of missed HGD/malignant lesions comprised 7 of the 10 cases (70.0%), which is ~10% higher than when including symptomatic patients with HGD/malignancy. Furthermore, when only applying the absolute criteria from the 2018 European guideline, or when applying the “high-risk stigmata” only from the 2017 International Consensus guideline, each missed 4 of 19 cases (21.1%) of HGD/malignancy (Fig. 1, Table 3).

Table 4 . Assessment of recommendations for surveillance or surgical resection of intraductal papillary mucinous neoplasms per guideline

GuidelineCorrect recommendation n/total (%)Incorrect recommendation n/total (%)
SurveillanceSurgerySurveillanceSurgery
2018 ACG5/39 (12.8)26/60 (43.3)0/26 (0)34/60 (56.7)
2015 AGA29/39 (74.4)10/20 (50.0)16/26 (61.5)10/20 (50.0)
2018 Euro. (combined)6/39 (15.4)26/59 (44.1)0/26 (0)33/59 (55.9)
2017 Int. (combined)2/39 (5.1)25/62 (40.3)1/26 (3.8)37/62 (59.7)

ACG, American College of Gastroenterology; AGA, American Gastroenterological Association; Euro. (combined), European combined absolute and relative criteria; Int. (combined), International combined “high-risk stigmata” and “worrisome features”.



Table 5 . Sensitivity, specificity, and positive and negative predicative values per guideline

GuidelineSensitivity (%)Specificity (%)PPV (%)NPV (%)
2018 ACG100.012.843.3100.0
2015 AGA38.574.450.064.4
2018 Euro. (combined)100.015.444.1100.0
2017 Int. (combined)96.25.140.366.7

PPV, positive predictive value; NPV, negative predictive value; ACG, American College of Gastroenterology; AGA, American Gastroenterologi­cal Association; Euro. (combined), European combined absolute and re­lative criteria; Int. (combined), International combined “high-risk stigmata” and “worrisome features”.



Fig 1. Missed HGD/malignant IPMNs and overtreatment of IPMNs per guideline. The rate of missed lesions with HGD/malignancy (total of 19 cases with HGD/malignancy) is shown by the dark blue bar graph for each guideline, while the rate of overtreatment per the total number of cases recommended for surgery based on each guideline is shown by the light blue bar graph. HGD, high-grade dysplasia; ACG, American College of Gastroenterology; AGA, American Gastroenterological Association; Euro., European; Euro. (combined), European combined absolute and relative criteria; Int., international; IPMNs, intraductal papillary mucinous neoplasms.

In terms of symptomatic patients (n = 16), the 2018 ACG and the 2018 European guidelines (applying absolute and relative criteria) recommended surgery for all these patients. On the other hand, the 2015 AGA guideline missed 9 of these patients (56.3%), while the 2017 International Consensus guideline (applying “high-risk stigmata” and “worrisome features”) missed 1 patient. Fig. 1 shows the overtreatment per guideline in which surgery was recommended, but postoperative pathology results showed no HGD/malignancy, and/or the patient had no postoperative symptomatic improvement.

Secondary outcomes

Table 6 summarizes the preoperative IPMN characteristics, which were determined by the combination of preoperative cross-sectional imaging studies and/or EUS. CA-19-9 levels were obtained from the serum. There were no significant differences between the pre-malignant (n = 46) and HGD/malignant lesions (n = 19) in terms of the presence of a mural nodule (benign/low-grade 8/46 [17.4%] vs. HGD/malignant 1/19 [5.3%], p = 0.264) and the presence of a solid component (benign/low-grade 4/46 [8.7%] vs. HGD/malignant 5/19 [26.3%], p = 0.108). However, the pre-malignant cysts had fewer cases with pancreatic ductal dilation ≥ 5−9 mm (benign/low-grade 20/46 [43.5%] vs. HGD/malignant 14/19 [73.7%], p = 0.032) and dilation ≥ 10 mm (benign/low-grade 7/46 [15.2%] vs. HGD/malignant 9/19 [47.4%], p = 0.011); and had fewer cases with CA-19-9 > 37 U/mL (benign/low-grade 5/42 [11.9%] vs. HGD/malignant 7/15 [46.7%], p = 0.009). Additionally, in the 16 patients who underwent surgical resection of their IPMN due to the presence of underlying symptoms, there was a statistically higher proportion of patients who were symptomatic and harbored HGD/malignancy (symptomatic with benign/low-grade lesion 7/46 [15.2%] vs. symptomatic with HGD/malignant lesion 9/19 [47.4%], p = 0.011).

Table 6 . Pathological characteristics of intraductal papillary mucinous neoplasms

Benign/low-grade n/total (%)HGD/malignant n/total (%)p-value
Main pancreatic duct ≥ 5−9 mm20/46 (43.5)14/19 (73.7)0.032*
Main pancreatic duct ≥ 10 mm7/46 (15.2)9/19 (47.4)0.011*
Solid component4/46 (8.7)5/19 (26.3)0.108
Mural nodule8/46 (17.4)1/19 (5.3)0.264
Pancreatic cyst size ≥ 3 cm27/46 (58.7)6/19 (31.6)0.059
CA-19-9 > 37 U/mL5/42 (11.9)7/15 (46.7)0.009*
Cyst growth > 3 mm per annum14/46 (30.4)1/19 (5.3)0.049*
Thick cyst wall3/46 (6.5)2/19 (10.5)0.625

HGD, high-grade dysplasia.

*Statistically significant (p ≤ 0.05).



Pancreatoduodenectomy was performed in 42 cases (64.6%), partial pancreatectomy in 7 cases (10.8%), and distal pancreatectomy in 16 cases (24.6%).

There were 4 deaths among patients with HGD/malignancy (two from postoperative complications, and two from disease progression) during the study period. In the patients with pre-malignant IPMNs, there were 3 deaths during the study period from unrelated causes.

DISCUSSION

The nuances and differences between the multiple guidelines for managing IPMNs can lead to dramatically different rates of cancer detection (or rates of missed cancers), rates of mortality (including from malignancy, as well as from operative complications), and health care costs [15]. Additionally, while all IPMNs carry a risk of malignancy, and surgical resection may be of some benefit to patients even prior to malignant transformation, the risks of pancreatic surgery are generally considered to be high, so it is ideal to avoid any pancreatic surgery, unless absolutely necessary. Therefore, clinicians must understand the impact of following one guideline over another.

This study included 65 patients with IPMNs, most of whom had BD-IPMNs, who underwent surgical resection. When retrospectively applying the criteria of HGD/malignancy findings on surgical samples or improvement in preoperative symptoms attributed to the cystic lesion, we found surgery was considered “justified” in 40.0% of cases (26 of 65). The rationale of justifying surgery in patients with resolution of preoperative symptoms after surgery is that symptoms of obstructive jaundice and pancreatitis secondary to the presence of IPMNs have been shown to have a positive correlation with the risk for progression to HGD/malignancy [1,21,22]. Additionally, these symptoms are considered an absolute or relative indication for surgery by all guidelines except the 2015 AGA guideline (intended to be for asymptomatic patients). Our data does show a statistical correlation between symptomatic patients and HGD/malignancy, hence validating the use of these symptoms in the respective guidelines as predictors of HGD/malignancy. Moreover, the percentage of “justified” surgeries in this study is similar to that seen in other studies using prior guidelines [16,17,23]. The 2018 ACG guideline, as well as the absolute and relative indications (combined) from the 2018 European guideline, correctly recommended surgery in all cases of HGD/malignancy. This is also shown in both guidelines having a sensitivity and negative predictive value of 100%, indicating that they are optimal at ruling out a lesion with HGD/malignancy. Park et al. [24] in a similar study previously found the 2018 European guideline to have a 92% negative predictive value for IPMNs, hence these results are consistent with the pre-existing literature. However, these two same sets of guidelines also resulted in high rates of surgery on patients who ultimately were found not to have HGD/malignancy or improvement in symptoms postoperatively, which is represented by their low specificity and positive predictive value, indicating a degree of overtreatment. Similar studies have found a low specificity of 2.0%–11.3% vs. 15.4% in this study and positive predictive value of 35.0%–60.0% vs. 44.1% in this study with the 2018 European guideline [25,26].

Further analysis showed that the 2017 International Consensus guideline (applying “high-risk stigmata” and “worrisome features”) accurately detected 96.2% of cases in which surgery was considered “justified” (represented by its sensitivity of 96.2%). This coincides with the previously reported high sensitivities of 70.0%−98.0% in other studies [25,27,28]. For the one patient with an underlying cancer for whom this guideline did not recommend surgery, the patient did not have a preoperative EUS/FNA. While EUS has been shown to be imperfect, data does support preoperative EUS with sampling to improve diagnostic accuracy (and therefore surgical planning) in cases in which a diagnosis is uncertain [29]. On the other hand, the 2015 AGA guideline was the best at only recommending surveillance rather than surgery in patients who were postoperatively shown to not have HGD/malignancy or symptomatic improvement, represented by its specificity (74.4%), that is within the range of other studies, and negative predictive value (64.4%), that is lower than previously reported [17,25,30]. However, this was at the expense of having the highest rate of not recommending surgery in HGD/malignant lesions in up to 57.9% of patients; 11 cases missed in this study, represented by its low sensitivity (38%), which is also supported by prior data [17,25,30]. Of these 11 cases of missed HGD/malignancy, 9 were HGD, 1 was invasive carcinoma, and 1 was adenocarcinoma. The 2015 AGA guideline was only intended to be applied to asymptomatic patients, and when symptomatic patients with HGD/malignancy in this study were excluded, thus leaving only asymptomatic patients with HGD/malignancy to be assessed, the rate of missed HGD/malignant lesion worsened to 70.0% from 57.9% (missed 7 of the 10 asymptomatic patients with HGD/malignancy).

Finally, when applying the criteria of “high-risk stigmata” of the 2017 International Consensus guideline or the absolute criteria of the 2018 European guideline, 21.1% of HGD/malignant cases (4 of 19 cases) were missed. The 4 cases missed by the 2018 European guideline all contained HGD, while the 4 cases missed by the 2017 International Consensus guideline involved 3 with HGD, and 1 with ductal adenocarcinoma.

When comparing this current study to Lekkerkerker et al. [31], our findings confirm that the 2015 AGA guideline, when compared to the other guidelines, misses the most HGD/malignant lesions; however, we found this rate to be increased from 12.0% in the prior study, to 57.9% in our study. A recent study by van Huijgevoort et al. [23] also noted a higher miss rate of 73.0% with the 2015 AGA guideline. In terms of comparing our study to Lekkerkerker et al. [31], a similar number of patients underwent preoperative EUS in the former study compared to ours (72.0% vs. 66.2%, p = 0.45), so this does not explain the observed difference in the miss rate. This former study also applied the AGA guideline to symptomatic patients; however, it included a smaller portion of patients of 3.0%−4.0% vs. 24.6% in this study. We do not believe having a larger number of symptomatic patients explains the higher rate of missed HGD/malignant lesions in our study, as when the AGA guideline was strictly applied to only asymptomatic patients with HGD/malignancy, the rate of missed HGD/malignant lesions did not improve. Additionally, when the combined criteria from the 2018 European guideline was used, and when the “high-risk stigmata” and “worrisome features” from the 2017 International Consensus guideline were applied [19,20], there were no missed HGD/malignant lesions, similar to that found by Lekkerkerker et al. [31].

In terms of identifying significant pathological features of IPMNs, we demonstrate a statistically significant association between main pancreatic duct dilation ≥ 5−9 mm (p = 0.032) and ≥ 10 mm (p = 0.011) with the presence of underlying HGD/malignancy, which supports surgical resection for IPMNs with this associated finding. We also demonstrate a statistically significant association between CA-19-9 elevation and the presence of underlying HGD/malignancy. This high-risk feature is considered a worrisome feature in the 2017 International Consensus guideline (updated change from the prior guideline) [32], an indication for surgical consideration in the 2018 ACG guideline, a relative indication for surgery in the 2018 European guideline, and is not considered at all by the 2015 AGA guideline [18]. This suggests that guidelines should consider placing more emphasis on elevated CA-19-9 levels to increase their ability to detect HGD/malignant lesions, and hence improve their positive predictive value. Furthermore, there was a statistically significant association between benign/low-grade dysplastic IPMNs and a cyst growth > 3 mm per annum (benign/low-grade 14/46 [30.4%] vs. HGD/malignant 1/19 [5.3%], p = 0.049). In regard to cyst growth rate, there is variability among the guidelines (except the 2015 AGA guideline, which does not include this in their criteria) on the amount of cyst growth rate that is considered a reason for surgical consultation, which varies from ≥ 2.5 to 5 mm per annum. Our finding suggests that a higher cyst growth rate from > 3 mm per annum might be needed to accurately distinguish a benign vs. HGD/malignant IPMN.

This study has many strengths. First, only IPMNs were included. IPMNs are the most common cystic lesion, and all guidelines are designed to focus on IPMNs. Similar prior studies have included cystic neuroendocrine tumors, which most guidelines do not discuss and are managed differently, so applying these guidelines to cystic neuroendocrine tumors of the pancreas would be inappropriate. Additionally, this study included a comparison of four major guidelines that comprised the two U.S. national guidelines from the ACG and AGA, the International Consensus guideline, and the updated European Study Group guideline on the management of IPMNs. All patients in this study also had postoperative surgical pathology, rather than just cytology from EUS-directed FNA, which compared to postoperative specimens, is significantly limited in its sensitivity and accuracy. Lastly, this study is clinically applicable, as it helps clinicians understand the various impacts of applying the different guidelines for IPMNs.

This study also has its limitations. Preoperative EUS with FNA was not performed on all cystic pancreatic lesions, which could have resulted in over-estimation of the number of missed lesions per guideline. However, this would have impacted all guidelines equally, since they all agree on recommending surgery in the presence of HGD/malignancy from the EUS-directed FNA; additionally, cytology alone has a low sensitivity, and often, surgeons must clinically decide on operative management without the results of EUS-directed FNA sampling. Furthermore, this is a retrospective study of seven years, which comes with all the limitations of any retrospective study.

In summary, decision making regarding surgery in IPMNs is challenging and fraught with nuances and subtleties with varying recommendations from multiple different guidelines, especially for cases that do not have features that are universally regarded as indications for surgery. While we know that the natural history of BD-IPMNs is associated with a slower onset to malignancy compared to MD-IPMNs, we are still learning how to define and incorporate high-risk features of these BD-IPMNs into clinical decision making. Additionally, the guidelines generally do not include information from novel diagnostic modalities, like targeted DNA-based next-generation sequencing (NGS), quantitative methylation-specific PCR, quantitative specific microRNAs assays, EUS-guided needle-based confocal laser endomicroscopy (nCLE), and the integration of artificial intelligence with EUS and EUS-guided nCLE [33]. Incorporation of these entities would certainly help improve the diagnostic accuracy and risk stratification of IPMNs.

Studies demonstrating the future potential use of novel diagnostic tools in stratifying IPMNs include a study by Singhi et al. [34] that identified, in a small set of patients (n = 56) who underwent surgical resection of IPMNs, a specific selection criteria of mutant genes (KRAS/GNAS/ TP53/PIK3CA/PTEN) that are typically found in pancreatic cysts, including those with invasive adenocarcinoma, which when present at a specific mutant allele frequency and detected by NGS from preoperative cyst fluid, correlated with advanced neoplasia with 100% sensitivity and 100% specificity. Rosenbaum et al. [35] showed that when incorporating cases that also had non-KRAS/GNAS mutations (TP53, SMAD4, CDKN2A and NOTCH1) in a small cohort of patients with IPMNs (n = 37), the sensitivity of cytology to detect malignancy increased from 75.0% to 79.0%. Additionally, a recent study by Paniccia et al. [36] showed that NGS of 22 pancreatic cyst-associated genes in combination with cytopathologic findings of cyst fluid had a sensitivity and specificity of 88.0% and 96.0%, respectively, for advanced neoplasia in 167 cases of IPMNs, and was superior to the absolute criteria utilized by the 2015 AGA and 2017 International guidelines for recommending surgery. In terms of studies showing the benefits of EUS-guided nCLE for risk assessment of IPMNs, Krishna et al. [37], in a study of 26 patients with branch-duct or mixed-branch IPMNs, showed that increased papillary width (cut-off ≥ 50 μm), reflective of cellular stratification, and darkness (cut-off ≤ 90 pixel intensity), indicative of loss of polarity and increased nuclear-to-cytoplasmic ratio, of the papillary epithelium quantified with dedicated software both had 88% sensitivity and 100% specificity for detecting HGD/malignancy. When these two endomicroscopy markers were combined, the sensitivity for detecting HGD/malignancy increased to 94%, with no effect on specificity [37]. Limitations of EUS-guided nCLE include the need for a costly special processor, nCLE probes that are not infinitely reusable, limited maneuverability of the needle that may not allow full assessment of a cyst, interobserver variability, and the need for additional training [33,37]. Of note, the European guideline does not recommend the use of nCLE for the differential diagnosis of pancreatic cystic neoplasms, and they express a major concern with this diagnostic modality, given the rate 7.0% to 9.0% of adverse events (pancreatitis, intracystic bleeding, and transient abdominal pain) seen in the INSPECT and DETECT studies [20,38,39]. A later study with 47 patients by Feng et al. [40] did not report any episodes of pancreatitis when nCLE was used to develop a new diagnostic criteria for mucinous cystic lesions. Lastly, specific hypermethylated genes (i.e., SOX17, FOXE1, TBX15, and BMP3) and microRNAs associated with HGD and/or malignancy in IPMNs have been identified, and are alternative biomarkers that can be used to risk-stratify IPMNs [41-45].

In the future, more studies are needed to continue to characterize the association between high-risk features of IPMNs and malignancy, and to determine how to best incorporate novel diagnostic tools in a cost-effective manner to help modify evidence-based management algorithms and guidelines to properly identify patients with IPMNs who will benefit from surgery. In the meantime, it remains critical to understand the nuances of the various guidelines, their differences, and how their clinical application will result in significant impacts on patients in the form of variations in cancer detection, surgery recommendations, mortality rates from malignancy, operative complications, and health care costs. We have shown the 2018 ACG and 2018 European (combined relative and absolute criteria) guidelines to be the best at effectively ruling out the presence of HGD/malignancy in a pancreatic IPMN (with the downside of unnecessarily operating on more patients); similarly, we have shown the 2015 AGA guideline has the highest specificity, at the expense of having the lowest sensitivity.

FUNDING

None.

CONFLICT OF INTEREST

No potential conflict of interest relevant to this article was reported.

AUTHOR CONTRIBUTIONS

Conceptualization: ICP, VMS, BGS, AYW, DSS, AJP, TWB, VMZ, AT, RCDB. Data curation: ICP, AB, VMS, BGS, AYW, DSS, AJP, TWB, VMZ, AT, RCDB. Methodology: ICP, VMS, BGS, AYW, DSS, AJP, TWB, VMZ, AT, RCDB. Visualization: ICP, AB, VMS, BGS, AYW, DSS, AJP, TWB, VMZ, AT, RCDB. Writing - original draft: ICP, RCDB. Writing - review & editing: ICP, AB, VMS, BGS, AYW, DSS, AJP, TWB, VMZ, AT, RCDB.

References
  1. Salvia R, Fernández-del Castillo C, Bassi C, Thayer SP, Falconi M, Mantovani W, et al. Main-duct intraductal papillary mucinous neoplasms of the pancreas: clinical predictors of malignancy and long-term survival following resection. Ann Surg 2004;239:678-685; discussion 685-687.
    Pubmed KoreaMed CrossRef
  2. Zaheer A, Pokharel SS, Wolfgang C, Fishman EK, Horton KM. Incidentally detected cystic lesions of the pancreas on CT: review of literature and management suggestions. Abdom Imaging 2013;38:331-341.
    Pubmed CrossRef
  3. Elta GH, Enestvedt BK, Sauer BG, Lennon AM. ACG clinical guideline: diagnosis and management of pancreatic cysts. Am J Gastroenterol 2018;113:464-479.
    Pubmed CrossRef
  4. Buerlein RCD, Shami VM. Management of pancreatic cysts and guidelines: what the gastroenterologist needs to know. Ther Adv Gastrointest Endosc 2021;14:26317745211045769.
    Pubmed KoreaMed CrossRef
  5. Scheiman JM, Hwang JH, Moayyedi P. American gastroenterological association technical review on the diagnosis and management of asymptomatic neoplastic pancreatic cysts. Gastroenterology 2015;148:824-848.e22.
    Pubmed CrossRef
  6. Ferrone CR, Correa-Gallego C, Warshaw AL, Brugge WR, Forcione DG, Thayer SP, et al. Current trends in pancreatic cystic neoplasms. Arch Surg 2009;144:448-454.
    Pubmed KoreaMed CrossRef
  7. Pagliari D, Saviano A, Serricchio ML, Dal Lago AA, Brizi MG, Lanza F, et al. Uptodate in the assessment and management of intraductal papillary mucinous neoplasms of the pancreas. Eur Rev Med Pharmacol Sci 2017;21:2858-2874.
  8. Scheiman JM. Pancreatic cysts - part 1: using the American Gastroenterological Association guidelines for the management of pancreatic cysts-a practical approach. Pancreas 2017;46:742-744.
    Pubmed CrossRef
  9. Anand N, Sampath K, Wu BU. Cyst features and risk of malignancy in intraductal papillary mucinous neoplasms of the pancreas: a meta-analysis. Clin Gastroenterol Hepatol 2013;11:913-921; quiz e59-e60.
    Pubmed CrossRef
  10. Kolb JM, Argiriadi P, Lee K, Liu X, Bagiella E, Gupta S, et al. Higher growth rate of branch duct intraductal papillary mucinous neoplasms associates with worrisome features. Clin Gastroenterol Hepatol 2018;16:1481-1487.
    Pubmed CrossRef
  11. Pergolini I, Sahora K, Ferrone CR, Morales-Oyarvide V, Wolpin BM, Mucci LA, et al. Long-term risk of pancreatic malignancy in patients with branch duct intraductal papillary mucinous neoplasm in a referral center. Gastroenterology 2017;153:1284-1294.e1.
    Pubmed CrossRef
  12. Ridtitid W, DeWitt JM, Schmidt CM, Roch A, Stuart JS, Sherman S, et al. Management of branch-duct intraductal papillary mucinous neoplasms: a large single-center study to assess predictors of malignancy and long-term outcomes. Gastrointest Endosc 2016;84:436-445.
    Pubmed CrossRef
  13. Vege SS, Ziring B, Jain R, Scheiman JM, Hwang JH, Moayyedi P. Optimal strategies for pancreatic cyst surveillance: we need better comparative data, not more case series. Gastrointest Endosc 2017;85:685-686.
    Pubmed CrossRef
  14. Marchegiani G, Pollini T, Andrianello S, Tomasoni G, Biancotto M, Javed AA, et al. Progression vs cyst stability of branch-duct intraductal papillary mucinous neoplasms after observation and surgery. JAMA Surg 2021;156:654-661.
    Pubmed KoreaMed CrossRef
  15. Lobo JM, Scheiman JM, Zaydfudim VM, Shami VM, Sauer BG. Clinical and economic outcomes of patients undergoing guideline-directed management of pancreatic cysts. Am J Gastroenterol 2020;115:1689-1697.
    Pubmed CrossRef
  16. Hsiao CY, Yang CY, Wu JM, Kuo TC, Tien YW. Utility of the 2006 Sendai and 2012 Fukuoka guidelines for the management of intraductal papillary mucinous neoplasm of the pancreas: a single-center experience with 138 surgically treated patients. Medicine (Baltimore) 2016;95:e4922.
    Pubmed KoreaMed CrossRef
  17. Xu MM, Yin S, Siddiqui AA, Salem RR, Schrope B, Sethi A, et al. Comparison of the diagnostic accuracy of three current guidelines for the evaluation of asymptomatic pancreatic cystic neoplasms. Medicine (Baltimore) 2017;96:e7900.
    Pubmed KoreaMed CrossRef
  18. Vege SS, Ziring B, Jain R, Moayyedi P; Clinical Guidelines Committee; American Gastroenterology Association. American gastroenterological association institute guideline on the diagnosis and management of asymptomatic neoplastic pancreatic cysts. Gastroenterology 2015;148:819-822; quize12-13.
    CrossRef
  19. Tanaka M, Fernández-Del Castillo C, Kamisawa T, Jang JY, Levy P, Ohtsuka T, et al. Revisions of international consensus Fukuoka guidelines for the management of IPMN of the pancreas. Pancreatology 2017;17:738-753.
    Pubmed CrossRef
  20. European Study Group on Cystic Tumours of the Pancreas. European evidence-based guidelines on pancreatic cystic neoplasms. Gut 2018;67:789-804.
    Pubmed KoreaMed CrossRef
  21. Moriya T, Hashimoto Y, Traverso LW. The duration of symptoms predicts the presence of malignancy in 210 resected cases of pancreatic intraductal papillary mucinous neoplasms. J Gastrointest Surg 2011;15:762-770; discussion 770-771.
    Pubmed CrossRef
  22. Shin SH, Han DJ, Park KT, Kim YH, Park JB, Kim SC. Validating a simple scoring system to predict malignancy and invasiveness of intraductal papillary mucinous neoplasms of the pancreas. World J Surg 2010;34:776-783.
    Pubmed CrossRef
  23. van Huijgevoort NCM, Hoogenboom SAM, Lekkerkerker SJ, Busch OR, Del Chiaro M, Fockens P, et al. Diagnostic accuracy of the AGA, IAP, and European guidelines for detecting advanced neoplasia in intraductal papillary mucinous neoplasm/neoplasia. Pancreatology 2023;23:251-257.
    Pubmed CrossRef
  24. Park RHS, Lim GRS, Wu JJY, Koh YX, Teo JY, Cheow PC, et al. Validation of the clinical utility of 4 guidelines in the initial triage of mucinous cystic lesions of the pancreas based on cross-sectional imaging: experience with 188 surgically-treated patients. Eur J Surg Oncol 2020;46:2114-2121.
    Pubmed CrossRef
  25. Vanden Bulcke A, Jaekers J, Topal H, Vanbeckevoort D, Vandecaveye V, Roskams T, et al. Evaluating the accuracy of three international guidelines in identifying the risk of malignancy in pancreatic cysts: a retrospective analysis of a surgical treated population. Acta Gastroenterol Belg 2021;84:443-450.
    Pubmed CrossRef
  26. Crippa S, Fogliati A, Valente R, Sadr-Azodi O, Arnelo U, Capurso G, et al. A tug-of-war in intraductal papillary mucinous neoplasms management: comparison between 2017 International and 2018 European guidelines. Dig Liver Dis 2021;53:998-1003.
    Pubmed CrossRef
  27. Sharib JM, Fonseca AL, Swords DS, Jaradeh K, Bracci PM, Firpo MA, et al. Surgical overtreatment of pancreatic intraductal papillary mucinous neoplasms: do the 2017 International Consensus Guidelines improve clinical decision making? Surgery 2018;164:1178-1184.
    Pubmed CrossRef
  28. Watanabe Y, Endo S, Nishihara K, Ueda K, Mine M, Tamiya S, et al. The validity of the surgical indication for intraductal papillary mucinous neoplasm of the pancreas advocated by the 2017 revised International Association of Pancreatology consensus guidelines. Surg Today 2018;48:1011-1019.
    Pubmed CrossRef
  29. Giannone F, Crippa S, Aleotti F, Palumbo D, Belfiori G, Partelli S, et al. Improving diagnostic accuracy and appropriate indications for surgery in pancreatic cystic neoplasms: the role of EUS. Gastrointest Endosc 2022;96:648-656.e2.
    Pubmed CrossRef
  30. Singhi AD, Zeh HJ, Brand RE, Nikiforova MN, Chennat JS, Fasanella KE, et al. American Gastroenterological Association guidelines are inaccurate in detecting pancreatic cysts with advanced neoplasia: a clinicopathologic study of 225 patients with supporting molecular data. Gastrointest Endosc 2016;83:1107-1117.e2.
    Pubmed CrossRef
  31. Lekkerkerker SJ, Besselink MG, Busch OR, Verheij J, Engelbrecht MR, Rauws EA, et al. Comparing 3 guidelines on the management of surgically removed pancreatic cysts with regard to pathological outcome. Gastrointest Endosc 2017;85:1025-1031.
    Pubmed CrossRef
  32. Tanaka M, Fernández-del Castillo C, Adsay V, Chari S, Falconi M, Jang JY, et al. International consensus guidelines 2012 for the management of IPMN and MCN of the pancreas. Pancreatology 2012;12:183-197.
    Pubmed CrossRef
  33. Krishna S, Abdelbaki A, Hart PA, Machicado JD. Endoscopic ultrasound-guided needle-based confocal endomicroscopy as a diagnostic imaging biomarker for intraductal papillary mucinous neoplasms. Cancers (Basel) 2024;16:1238.
    Pubmed KoreaMed CrossRef
  34. Singhi AD, McGrath K, Brand RE, Khalid A, Zeh HJ, Chennat JS, et al. Preoperative next-generation sequencing of pancreatic cyst fluid is highly accurate in cyst classification and detection of advanced neoplasia. Gut 2018;67:2131-2141.
    Pubmed KoreaMed CrossRef
  35. Rosenbaum MW, Jones M, Dudley JC, Le LP, Iafrate AJ, Pitman MB. Next-generation sequencing adds value to the preoperative diagnosis of pancreatic cysts. Cancer Cytopathol 2017;125:41-47.
    Pubmed CrossRef
  36. Paniccia A, Polanco PM, Boone BA, Wald AI, McGrath K, Brand RE, et al. Prospective, multi-institutional, real-time next-generation sequencing of pancreatic cyst fluid reveals diverse genomic alterations that improve the clinical management of pancreatic cysts. Gastroenterology 2023;164:117-133.e7.
  37. Krishna SG, Hart PA, DeWitt JM, DiMaio CJ, Kongkam P, Napoleon B, et al. EUS-guided confocal laser endomicroscopy: prediction of dysplasia in intraductal papillary mucinous neoplasms (with video). Gastrointest Endosc 2020;91:551-563.e5.
    Pubmed CrossRef
  38. Konda VJ, Meining A, Jamil LH, Giovannini M, Hwang JH, Wallace MB, et al. A pilot study of in vivo identification of pancreatic cystic neoplasms with needle-based confocal laser endomicroscopy under endosonographic guidance. Endoscopy 2013;45:1006-1013.
    Pubmed CrossRef
  39. Nakai Y, Iwashita T, Park DH, Samarasena JB, Lee JG, Chang KJ. Diagnosis of pancreatic cysts: EUS-guided, through-the-needle confocal laser-induced endomicroscopy and cystoscopy trial: DETECT study. Gastrointest Endosc 2015;81:1204-1214.
    Pubmed CrossRef
  40. Feng Y, Chang X, Zhao Y, Wu D, Meng Z, Wu X, et al. A new needle-based confocal laser endomicroscopy pattern of malignant pancreatic mucinous cystic lesions (with video). Endosc Ultrasound 2021;10:200-206.
    Pubmed KoreaMed CrossRef
  41. Hata T, Dal Molin M, Hong SM, Tamura K, Suenaga M, Yu J, et al. Predicting the grade of dysplasia of pancreatic cystic neoplasms using cyst fluid DNA methylation markers. Clin Cancer Res 2017;23:3935-3944.
    Pubmed KoreaMed CrossRef
  42. Hong SM, Omura N, Vincent A, Li A, Knight S, Yu J, et al. Genome-wide CpG island profiling of intraductal papillary mucinous neoplasms of the pancreas. Clin Cancer Res 2012;18:700-712.
    Pubmed KoreaMed CrossRef
  43. Majumder S, Taylor WR, Yab TC, Berger CK, Dukek BA, Cao X, et al. Novel methylated DNA markers discriminate advanced neoplasia in pancreatic cysts: marker discovery, tissue validation, and cyst fluid testing. Am J Gastroenterol 2019;114:1539-1549.
    Pubmed KoreaMed CrossRef
  44. Shirakami Y, Iwashita T, Uemura S, Imai H, Murase K, Shimizu M. Micro-RNA analysis of pancreatic cyst fluid for diagnosing malignant transformation of intraductal papillary mucinous neoplasm by comparing intraductal papillary mucinous adenoma and carcinoma. J Clin Med 2021;10:2249.
    Pubmed KoreaMed CrossRef
  45. Sato Y, Suzuki R, Takagi T, Sugimoto M, Ohira H. Circulating extracellular vesicle-encapsulated microRNA as screening biomarkers for intraductal papillary mucinous neoplasm. Oncol Lett 2020;20:315. Erratum in: Oncol Lett 2021;22:611.
    Pubmed KoreaMed CrossRef

 

November 2024, 28 (4)