Distal pancreatectomy (DP) involves a surgical resection of distal pancreas due to malignant or benign lesions located to the left of superior mesenteric vessels [1]. Depending on the nature and anatomical features of the pancreatic lesion, DP can be carried out with or without splenic preservation. The former is usually required for malignant or large-sized lesions while the latter might be carried out for small-sized benign lesions to reduce procedure-related morbidities [2].

Despite advancement of surgical techniques including minimally invasive approaches in DP and more recent compliance to enhanced recovery principles, overall postoperative morbidity after DP remains high, most commonly due to occurrence of pancreatic stump leak. This can subsequently result in postoperative pancreatic fistula (POPF) [3]. The International Study Group on Pancreatic Fistula (ISGPF) defines POPF grades B and C [4,5] as clinically relevant fistula which complicate between 15% and 26% of DPs [6,7]. Clinically relevant POPF has been demonstrated to be predictive of serious haemorrhagic and infective complications which can significantly prolong length of hospital stay and increase cost [8-10].

Controversies exist regarding the best strategy for closure of pancreatic stump in DP. Several techniques have been introduced to decrease the risk of POPF following DP, including linear stapler closure, hand-sewn suture closure, reinforced stapler closures, use of ultrasonic dissection devices, independent main pancreatic duct ligation, pancreatico-enteric anastomosis, sealing by fibrin glue, use of mesh, seromuscular patch, use of falciform ligament or omental plug, and different combinations of the aforementioned techniques [11]. Nevertheless, most comparisons have been evaluated by small number of rather low-level evidence research. Moreover, available evidence syntheses via meta-analyses are either non-specific with inclusions of several comparisons or lack outcome syntheses with respect to clinically relevant POPF. We believe each comparison in this context deserves an independent evidence synthesis with the main focus on clinically relevant POPF to demonstrate the real advantage of one approach over the other with respect to the best available evidence.

The objective of the current study was to carry out a comprehensive meta-analysis of all existing comparative studies to evaluate the risk of clinically relevant POPF associated with two commonly performed approaches (hand-sewn and stapler closure of pancreatic stump) in patients undergoing DP. This study aimed to investigate the impact of stapler line reinforcement on development of clinically relevant POPF. This study also aimed to conduct a trial sequential analysis to assess whether findings of the conducted meta-analysis of randomized controlled trials (RCTs) (level 1a evidence) were conclusive following evaluation of risk of type 1 or 2 errors.

Study design and eligibility criteria

Our methodology is outlined in a review protocol. The methodology of this study followed Standards of Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement [12]. All existing randomized and observational studies that compared hand-sewn closure technique and stapler closure technique for the management of pancreatic stump in patients undergoing DP were considered. Single-arm studies, expert opinions, letters to editors, case reports, and case series were not considered.

Population of interest

All male or female adult patients (age more than 18 years) who underwent DP for benign, premalignant, or malignant lesions of pancreas were included.

Intervention and comparison of interest

In this study, two comparisons were evaluated: 1) Hand-sewn closure versus stapler closure: Interested intervention was hand-sewn pancreatic stump closure with any suture types. It was compared with stapler closure of pancreatic stump using any types of stapler device. 2) Reinforced stapler closure versus standard stapler closure. The interested intervention was reinforced stapler closure with use of polyglycolic acid (PGA) mesh. It was compared with standard stapler closure of pancreatic stump without any reinforcement.

Outcomes

The primary outcome was POPF grades B and C according to the definition of ISGPF [4,5]. Because a POPF grade A is an asymptomatic biochemical leakage without clinical relevance [5], it was not considered as a complication.

Literature search strategy

A strategy for literature search was formulated and run via MEDLINE, CENTRAL, CINAHL, EMBASE, and Web of Sciences (Appendix 1). Moreover, an independent evaluation of reference lists of identified studies or reviews was carried out by two independent authors. The most recent literature search was performed on December 18th, 2023.

Study selection

An independent evaluation of identified articles was performed by two reviewers. When required, their full-texts were accessed and carefully investigated against our inclusion and exclusion criteria. Studies deemed eligible were selected for inclusion. Discrepancies during this stage were addressed via detailed discussion among assessors. If such disagreements remained unresolved, an independent assessor was involved.

Extraction and management of data

A spreadsheet for data extraction was developed. Information about included studies and outcome measures were collected from all eligible studies by two assessors. Disagreements during this stage were also addressed by consultation with an independent assessor.

Evaluation of risk of bias

An independent assessment of the methodology and risk of bias of our eligible studies were performed by two assessors upon criteria outlined by the Cochrane’s tool [13] for RCTs and the Risk Of Bias In Non-randomized Studies of Interventions (ROBINS-I) assessment tool [14] for observational studies. Disagreements following such assessments were addressed via discussion between the assessors. If disagreements persisted, an additional author was involved.

Summary measures and synthesis

Considering that POPF is a dichotomous outcome measure, odds ratio (OR) was calculated as the estimate of effect size. In the comparison of hand-sewn versus stapler closure, the OR was the odds of POPF associated with hand-sewn technique compared to the stapler technique. An OR < 1 would favour the hand-sewn group. In the comparison of reinforced stapler closure versus standard stapler closure, the OR was the odds of POPF associated with the reinforced stapler compared to the standard stapler. An OR < 1 would favour the reinforced group.

Review Manager 5.4 software [13] was utilised for analyses which involved the use of random-effects modelling. Results of data synthesis for POPF are presented in a forest plot demonstrating 95% confidence intervals (CIs).

We evaluated heterogeneity by calculating I² using the Cochran Q test (χ²). Heterogeneity was subsequently interpreted as follows: I² = 0%–25%, mild heterogeneity; I² = 26%–75%, moderate heterogeneity; and I² = 76%–100%, considerable heterogeneity. Funnel plots were also constructed to investigate publication bias.

We conducted sub-group analysis with respect to randomized and non-randomized studies. To identify contributing factors to heterogeneity, sensitivity analyses were performed. Individual effect of each RCT on outcomes was evaluated using a leave-one-out sensitivity analysis (repeating of outcome synthesis after exclusion of one study at a time).

Trial sequential analysis

We conducted a trial sequential analysis for any outcome measure investigated by at least 5 RCTs via the trial sequential analysis software 0.9.5.5 Beta (Copenhagen Trial Unit). Using O’Brien-Fleming α-spending function, thresholds for Z-values were adjusted to allow the risk of type I error to be restored to the desired maximum risk. A statistical significance would be achieved if O’Brien-Fleming α-spending boundaries were crossed by a Z-curve. Furthermore, Z values were penalised based on the strength of the available evidence and the number of repeated significance tests. We controlled type 2 error risk using the β-spending function and futility boundaries. Crossing futility boundaries by a Z-curve would suggest that the two approaches did not differ more than the anticipated intervention effect. We considered power of 80% to estimate the information size.

Summary table of findings

A table was made to compile and summarise existing evidence on associated risk of POPF in hand-sewn versus stapler closure of pancreatic stump and reinforced versus standard stapler groups. POPF was deemed the most important outcome parameter necessary for making a decision on inclusion in the summary table of findings. In line with standards highlighted in the system created by the Grading of Recommendation, Assessment, Development and Evaluation Working Group (GRADE Working Group), we graded the quality of evidence as high, moderate, low, and very low (Supplementary Table 1).

A total of 5,431 articles were detected following literature search, of which 78 studies were short-listed for further assessment. Another 46 articles were excluded as 24 did not have any comparison group, 9 used other approaches for reinforcement, 8 did not report clinically relevant POPF, and the remaining 5 studies compared pancreatico-enteric anastomosis with either hand-sewn or stapler techniques. Finally, 7 randomized [15-21] and 25 observational studies [22-47] with 4,022 patients were included (1,184 patients had hand-sewn closure of pancreatic stump and the remaining 2,838 patients had stapler closure of pancreatic stump following DP) (Fig. 1). Among those who had stapler closure of pancreatic stump, 914 patients had reinforcement of the stapler line The remaining 1,924 participants did not have reinforcement of the stapler line. Table 1 outlines included studies and related data.

Table 1 . Included studies and related data

Author	Year	Country	Journal	Study design	Comparison
Bassi et al. [15]	1999	Italy	HPB	RCT	Hand-sewn closure vs. Stapler closure
Bilimoria et al. [22]	2003	USA	Br J Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure, Reinforced stapler closure vs. Standard stapler closure
Balzano et al. [25]	2005	Italy	J Gastrointest Surg	Retrospective observational study	Reinforced stapler closure vs. Standard stapler closure
Goh et al. [23]	2008	Singapore	Arch Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure, Reinforced stapler closure vs. Standard stapler closure
Ferrone et al. [24]	2008	USA	J Gastrointest Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Finan et al. [26]	2009	USA	Am Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure Reinforced stapler closure vs. Standard stapler closure
Nathan et al. [28]	2009	USA	Ann Surg	Retrospective observational study	Reinforced stapler closure vs. Standard stapler closure
Harris et al. [27]	2010	USA	J Gastrointest Surg	Retrospective observational study	Reinforced stapler closure vs. Standard stapler closure
Frozanpor et al. [37]	2010	Sweden	JOP	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Kah Heng et al. [39]	2010	Singapore	ANZ J Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Yoshioka et al. [44]	2010	Japan	World J Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Ochia et al. [33]	2010	Japan	J Gastrointest Surg	Retrospective observational study	Reinforced stapler closure vs. Standard stapler closure
Diener et al. [16]	2011	Germany	Lancet	RCT	Hand-sewn closure vs. Stapler closure
Eguchi et al. [36]	2011	Japan	Dig Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Hamilton et al. [17]	2012	USA	Ann Surg	RCT	Reinforced stapler closure vs. Standard stapler closure
Ban et al. [45]	2012	Japan	World J Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Sepesi et al. [31]	2012	USA	J Gastrointest Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Wellner et al. [29]	2012	Germany	World J Gastrointest Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Kawai et al. [38]	2013	Japan	Am J Sur	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Ceppa et al. [30]	2015	USA	J Gastrointest Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Jang et al. [18]	2017	South Korea	JAMA Surg	RCT	Reinforced stapler closure vs. Standard stapler closure
Futagawa et al. [43]	2017	Japan	Anticancer Res	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Tieftrunk et al. [46]	2018	Germany	PLoS One	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Hayashibe and Ogino [32]	2018	Japan	Asian J Surg	Retrospective observational study	Reinforced stapler closure vs. Standard stapler closure
Kondo et al. [19]	2019	Japan	Ann Surg Oncol	RCT	Reinforced stapler closure vs. Standard stapler closure
Kawaida et al. [34]	2019	Japan	Anticancer Res	Retrospective observational study	Reinforced stapler closure vs. Standard stapler closure
Chikhladze et al. [42]	2020	Germany	Asian J Surg	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Wennerblom et al. [20]	2021	Sweden	Br J Surg	RCT	Reinforced stapler closure vs. Standard stapler closure
Merdrignac et al. [21]	2022	France	Ann Surg	RCT	Reinforced stapler closure vs. Standard stapler closure
Tian et al. [35]	2022	China	Front Oncol	Retrospective observational study	Reinforced stapler closure vs. Standard stapler closure
Palmeri et al. [41]	2023	Italy	Int J Med Robot	Retrospective observational study	Hand-sewn closure vs. Stapler closure
Murata et al. [40]	2023	Japan	Surg Laparosc Endosc Percutan Tech	Retrospective observational study	Hand-sewn closure vs. Stapler closure

RCT, randomized controlled trial.

Fig 1. Study flow diagram.

Methodological appraisal

The risk of bias assessment of eligible RCTs is outlined in Fig. 2A. Six studies were associated with low risk of selection bias. One RCT had an unclear risk of selection bias because of the lack of data about allocation concealment or random sequence generation. Six RCTs were associated with a low risk of performance bias. One RCT was associated with a high risk of performance bias because of lack of blinding of their patients. Two RCTs were associated with a low risk of detection bias. Four RCTs were associated with a high risk of detection bias because of the lack of blinding of assessors. One RCT was associated with an unclear risk of detection bias. All included RCTs were associated with low risks of attrition, reporting, or other bias types.

Fig 2. Risk of bias summary and graph showing authors’ judgments about each risk of bias item for (A) randomized controlled trials, and (B) observational studies.

Fig. 2B outlines the outcome of the risk of bias assessment of included observational studies. Eight studies were associated with an unclear risk of bias due to confounding. The rest of included studies were associated with a low risk of bias due to confounding. The risk of bias due to patient selection was unclear in 9 studies and low in the rest of included studies. Risks of other types of bias were low in all included observational studies.

Synthesis of outcomes

Outcomes are presented in Fig. 3, 4, Supplementary Fig. 1, Supplementary Table 1.

Fig 3. Forest plots of comparison of hand-sewn versus stapler pancreatic stump on clinically relevant postoperative pancreatic fistula. Solid squares denote odds ratio. Horizontal lines represent 95% CIs. Diamond denotes pooled effect size. M-H, Mantel Haenszel test; CI, confidence interval.

Fig 4. Forest plots of comparison of reinforced versus standard pancreatic stump on clinically relevant postoperative pancreatic fistula. Solid squares denote odds ratio. Horizontal lines represent 95% CIs. Diamond denotes pooled effect size. M-H, Mantel Haenszel test; CI, confidence interval.

Hand-sewn closure versus stapler closure

Clinically relevant postoperative pancreatic fistula

Twenty studies (2,435 patients) provided information on clinically relevant POPF. Rates of clinically relevant POPF in hand-sewn and stapler groups were 27.6% and 19.3%, respectively. Hand-sewn closure significantly increased the risk of clinically relevant POPF compared to stapler closure (OR: 1.56, 95% CI: 1.07–2.28, p = 0.02). There was a moderate degree of heterogeneity among studies (I²: 57%, p = 0.001). There was a moderate certainty of the evidence.

Randomized controlled trials

Two studies (381 patients) provided information on clinically relevant POPF. Rates of clinically relevant POPF in hand-sewn and stapler groups were 21.6% and 19.9%, respectively. No significant difference in risk of POPF was detected between the two cohorts (OR: 1.20, 95% CI: 0.56–2.58, p = 0.64). There was a moderate degree of heterogeneity among studies (I²: 19%, p = 0.27). There was a moderate certainty of the evidence.

Observational studies

Eighteen studies (2,054 patients) provided information on clinically relevant POPF. Rates of clinically relevant POPF in hand-sewn and stapler groups were 29.1% and 19.2%, respectively. Hand-sewn closure significantly increased the risk of clinically relevant POPF compared to stapler closure (OR: 1.59, 95% CI: 1.06–2.41, p = 0.03). There was a moderate degree of heterogeneity among studies (I²: 56%, p = 0.003). There was a moderate certainty of the evidence.

Reinforced stapler closure versus standard stapler closure

Clinically relevant postoperative pancreatic fistula

Fifteen studies (1,587 patients) provided information on clinically relevant POPF. Rates of clinically relevant POPF in hand-sewn and stapler groups were 12.6% and 20.8%, respectively. Use of reinforced stapler closure was associated with a significantly reduced rate of POPF when compared with standard stapler closure (OR: 0.54, 95% CI: 0.36–0.80, p = 0.002). There was a moderate degree of heterogeneity among studies (I²: 35%, p = 0.09). There was a moderate certainty of the evidence.

Randomized controlled trials

Five studies (622 patients) provided information on clinically relevant POPF. Rates of clinically relevant POPF in hand-sewn and stapler groups were 11.4% and 20.4%, respectively. Use of reinforced stapler closure was associated with a non-significant reduction in the rate of POPF when compared with standard stapler closure (OR: 0.50, 95% CI: 0.24–1.08, p = 0.08). There was a moderate degree of heterogeneity among studies (I²: 57%, p = 0.05). There was a moderate certainty of the evidence.

Observational studies

Ten studies (965 patients) provided information on clinically relevant POPF. Rates of clinically relevant POPF in hand-sewn and stapler groups were 13.3% and 21.6%, respectively. Use of reinforced stapler closure was associated with a significantly reduced rate of POPF when compared with standard stapler closure (OR: 0.55, 95% CI: 0.34–0.90, p = 0.02). There was a moderate degree of heterogeneity among studies (I²: 26%, p = 0.29). There was a moderate certainty of the evidence.

Sensitivity analysis

In the analysis of reinforced stapler versus standard stapler closure, when RCTs were considered, removal of study of Merdrignac et al. [21] changed findings significantly in favour of reinforced stapler (p = 0.01) and decreased heterogeneity from 57% to 32%.

Trial sequential analysis

Hand-sewn closure versus stapler closure

Clinically relevant POPF: Since only two RCTs were considered in the analysis of hand-sewn versus stapler closure of pancreatic stump, no trial sequential analysis was possible.

Reinforced stapler closure versus standard stapler closure

Clinically relevant POPF: The calculated information size was 833 patients. There was no crossing of α-spending boundaries by Z-curve. Futility boundaries were reached before the information size. Penalized Z-value remained < 1.96. This indicated that the meta-analysis was not conclusive and that findings on this outcome measure were associated with type 2 error (Fig. 5).

Fig 5. Results of trial sequential analysis for clinically relevant postoperative pancreatic fistula. Red inward-sloping dashed lines make up trial sequential monitoring boundaries. To the right, outward sloping red dashed lines make up the futility region. Solid blue line is cumulative Z curve. Solid green line presents penalised Z-value.

Considering the ongoing debate about the most appropriate approach in closure of pancreatic stump following DP, we performed this comprehensive meta-analysis of all existing comparative research to investigate outcomes associated with hand-sewn and stapler closure of pancreatic stump. We included 7 randomized and 25 observational studies enrolling 4,022 participants undergoing DP, of whom 1,184 underwent hand-sewn closure of pancreatic stump and the remaining 2,838 patients had stapler closure of pancreatic stump. Hand-sewn closure was associated with a significantly higher rate of clinically relevant POPF than stapler closure. Moreover, when stapler closure was considered, staple line reinforcement with PGA mesh significantly reduced formation of such POPF.

When considering only RCTs, no significant difference was found in risk of clinically relevant POPF between the two cohorts. The reduction in POPF rate in favour reinforced stapler closure did not reach statistical significance. When considering observational studies, hand-sewn closure significantly increased the rate of clinically relevant POPF compared to stapler closure. Moreover, when stapler closure was considered, staple line reinforcement significantly decreased the formation of clinically relevant POPF.

For the comparison of hand-sewn versus stapler closure of pancreatic stump, no trial sequential analysis was possible as the minimum number of 5 RCTs was unavailable. Nevertheless, for the comparison of reinforced versus standard stapler closure, the trial sequential analysis showed that results of the conducted meta-analysis of RCTs were associated with type 2 error because the minimal population size of 833 randomized patients to eliminate such risk could not be achieved. This indicates the need for future adequately powered RCTs to provide stronger evidence in this context.

The reported heterogeneity was moderate in all our outcome syntheses, indicating robustness of our findings regarding most outcomes. Previous meta-analyses have studied outcomes of pancreatic stump closure using variety of techniques. Nevertheless, they were either non-specific with inclusions of several comparisons or lacking outcome syntheses with respect to clinically relevant POPF. Ratnayake et al. [48] have conducted a network meta-analysis of several techniques and concluded that patch coverage following suture or stapler closure is associated with the lowest POPF risk and best results among the existing stump closure techniques after DP. However, the authors only included two studies with low sample sizes for the comparison of suture versus hand-sewn techniques. They missed 5 RCTs that evaluated reinforced stapler technique with PGA mesh. Most importantly, all comparisons in the aforementioned study had less than 5 studies, which questioned the validity and indication for a network-meta-analysis in this context. Tieftrunk et al. [46] have conducted a meta-analysis of all available pancreatic stump closure techniques and concluded that following DP, stapler closure, pancreatico-enteric anastomosis, and seromuscular patches/falciform were associated with lower incidence of POPF in comparison with suture closure alone. Although their meta-analysis was comprehensive, the main conclusion was made based on all types of POPF including type A. Moreover, several more studies have been published after their meta-analysis, which should now be considered for any meta-analysis in this context.

In terms of comparative evidence of hand-sewn and stapler pancreatic stump closure in DP, there were only two available RCTs, of which one was very old. Although randomized trials are considered as highest standard study design for comparative research, in this context, considering that majority of participants are not randomly allocated within trials, some may argue that RCT findings may not present the ‘real-world’ practice. Nevertheless, there was a robust evidence from a large number of observational studies with a pooled population size of more than 2,000 patients which indicated that stapler closure of pancreatic stump significantly reduced the rate of clinically relevant POPF compared to hand-sewn closure technique.

In view of comparative evidence of reinforced and standard stapler closure technique, although meta-analysis of RCTs demonstrated no statistical significance between two groups in terms of risk development of clinically relevant POPF, we believe that the risk reduction of almost 10% in favour of reinforced stapler is clinically significant. Interestingly, during our one-leave-out sensitivity analysis, removal of one study from the analysis changed findings significantly in favour of reinforced stapler and reduced heterogeneity from 57% to 32%. Moreover, findings of robust evidence from observational studies further confirmed this conclusion. Furthermore, detection of risk of type 2 error from the conducted trial sequential analysis further supported our conclusion that reinforced stapler closure with PGA mesh had a lower risk of clinically relevant POPF in comparison with standard stapler closure. On the other hand, our conclusions might have overstated such benefits in favour of reinforced stapler closure technique as 3 out of 5 included RCTs reported that reinforced stapler during DP did not reduce the incidence of clinically relevant pancreatic fistula compared to stapler without reinforcement. This dilemma further indicates the need for future RCTs in this context to provide more robust evidence in favour of either treatment strategy.

In order to address the risk of type 2 error detected in our trial sequential analysis, there is a need for further high quality RCTs adequately powered in this context. Elimination of risk of type 2 error can only be achieved by having larger sample sizes. In the analysis of reinforced stapler closure versus standard stapler closure, the minimum pooled population requirement has been estimated to be 833 patients while available studies provided a pooled population size of 622 patients. We do not hesitate to recommend a repeat evidence synthesis attempt in the literature after achieving a total pooled population size of 833 patients provided by future RCTs. With respect to the comparison of hand-sewn closure versus stapler closure techniques, there is a need for at least 3 more RCTs to be able to conduct a trial sequential analysis and estimate a minimum population size and subsequently evaluate the risk of type 2 error.

Considering the existence of several available techniques in pancreatic stump closure after DP, we do not hesitate to discourage network comparison between treatments when evidence synthesis is considered due to the presence of heterogenous best available level of evidence surrounding existing techniques. We believe that each comparison in this context deserves an independent evidence synthesis with the main focus on clinically relevant POPF to investigate the real advantage of one approach over the other.

Readers of this study should consider limitations of this study when interpreting it findings. Despite the existence of robust level 2 evidence, the number of included RCTs was limited, particularly in the comparison of hand-sewn versus stapler closure technique. Type 2 error risk was evident in both comparisons considered in this meta-analysis. The risk of detection bias and the performance bias were high in one and four studies due to lack of blinding of outcome assessors and patients, respectively. Finally, we were not able to report outcomes with respect to baseline characteristics of included patients, which might have biased our results, particularly when observational studies were considered.

The best available evidence indicates that reinforced stapler closure of pancreatic remnant after DP may reduce risk of clinically relevant POPF compared to hand-sewn closure or stapler closure without reinforcement. Although the available level 2a evidence is robust, the level 1a evidence is subject to type 2 error. Future adequately powered randomized research is needed to provide stronger evidence.

Supplementary data related to this article can be found at https://doi.org/10.14701/ahbps.24-015.

None.

No potential conflict of interest relevant to this article was reported.

Conceptualization: Shahin H. Data curation: Shahin H, Shahab H. Methodology: Shahin H, Shahab H. Visualization: All authors. Writing - original draft: Shahin H, Shahab H. Writing - review & editing: All authors.