ABSTRACT
There is a balance between procedural success and patient acceptability in outpatient hysteroscopy. Reporting either outcome alone inadequately reflects how well the procedure was conducted. A new metric is proposed: the “Clark ratio”, defined as patient acceptability divided by procedural failure, to capture this trade-off. Higher ratios indicate better performance. The limitations of data from randomised controlled trials and need for real-world evidence and standardised outcome measures to validate feasibility and patient experience is discussed. The Clark ratio is a new concept and could potentially be adopted, if found to be valid and useful, as a simple, easily interpretable, pragmatic benchmark for quality assurance and outcome in research for outpatient hysteroscopic procedures.
Introduction
There is a balance to be struck between successful completion of an outpatient/office procedure1 and patient experience. Clinicians are striving for high procedural success rates with high corresponding rates of patient acceptability. However, it may be that those clinicians with relatively higher rates of procedural success may achieve this at the expense of poorer patient acceptability. The converse may be true, those clinicians with lower rates of procedural success may not persevere with difficult procedures, be less likely to undertake prolonged procedures and have a lower threshold to abandon procedures where the patient is experiencing pain. As a consequence, patient acceptability may be higher. Thus, simply reporting procedural success or patient acceptability in isolation limits a full interpretation about the quality of how the procedure was conducted. It is important that where the rates of procedural success, namely a successfully completed procedure devoid of any adverse events, are reported that some measure of patient experience such as acceptability, is also reported.
I propose a simple and novel metric to account for the relationship between procedural acceptability and success is proposed. The Clark ratio I suggest is a conceptual framework that could aid quality-improvement but the metric would need validating in prospective studies before potential use in day-to-day hysteroscopic practice.
Proposal for a Novel Metric: The CLARK Ratio
Individual patients and clinicians will differ in their opinions on the minimal satisfactory trade-off between procedural success and acceptability. A simple ratio can be derived to take account of this trade-off between acceptability and success (or conversely failure): the Clark ratio (Box 1). The ratio is derived by dividing rates of patient acceptability by procedural failure (or 1-procedural success) for outpatient hysteroscopic procedures. It follows that the higher the Clark ratio, the better the conduct of the procedures (Figure 1). This simple metric can potentially be used to provide a more informative summary of procedural performance and be used for quality assurance at both individual and broader service levels.
Patient Acceptability and Procedural Success Data
The Royal College of Obstetricians and Gynaecologists (RCOG) and British Society for Gynaecological Endoscopy jointly published an evidence-based guideline in 20242 that included an audit tool providing a summary of the best randomised controlled trials (RCTs)2-7 for procedural and patient outcomes. Table 1 provides a summary of the benchmark data from these trials for patient acceptability and procedural success from which the Clark ratio can be derived. However, questions arise about the validity and generalisability of patient experience, namely acceptability, and procedural feasibility to be able to generate a meaningful, widely applicable metric balancing acceptability and success. Whilst acceptability and procedural success are broadly understood concepts, the definition and measurement of these outcomes lack standardisation.8, 9 Moreover, the most robust data we have pertaining to procedural acceptability and success for diagnostic and operative hysteroscopy come from RCTs2-7 that have in turn informed, evidence-based clinical guidelines.2 Although these RCTs are well conducted, large and often multi-centre, generalisability from these rarified experimental research environments to “real life”, day to day, clinical settings may be questionable. It is well recognised that RCTs often obtain data from narrower, more homogeneous patient populations treated by “expert” practitioners/centres from within more controlled, well-equipped settings, leading to over optimistic estimates of feasibility, efficacy and safety.10-13
Pragmatic use of Outcome Data
The data in Table 1 come from high quality published studies and so, as argued earlier, are likely to represent a higher standard to normal care. Thus, for the observed Clark ratios to be meaningful then a pragmatic approach is needed. This involves reducing the acceptability and success rates to account for some deterioration in outcomes in real life practice to produce rounded estimates for acceptability and success in the same “ball park” as the precise, but less generalisable data acquired from RCTs. Suggested ranges of patient acceptability and procedural success (failure) with observed Clark ratios are shown in Table 1.
It should be noted that the Clark ratio metric applies only for the commonest outpatient hysteroscopic procedures (diagnostic +/- blind / directed biopsies +/- IUCD fitting and polypectomies) with the similar associated pain scores.14, 15 For more invasive hysteroscopic / intrauterine procedures like endometrial ablation, myomectomy and retained pregnancy tissue extraction, the limited data available preclude the derivation of a reliable and reproducible Clark ratio at present.
Where sample sizes are small ratios can be unstable. The Clark ratio should only be calculated if there are 30 or more procedures. Also, with small numbers, confidence intervals around acceptability and failure rates can be used to derive a plausible range for the Clark ratio. This approach accounts for statistical uncertainty and improves comparability across studies and clinical settings.
Acquiring Real World Data
I have stated in the preceding sections that the published acceptability and feasibility data come from controlled trials rather than real-life settings. Attempts have been made to provide “real life” generalisable data; most notably the BSGE Ambulatory Care Network prospective cohort of over 5000 consecutive outpatient hysteroscopic procedures. However, whilst this study provided much information from patients about their experiences and views of outpatient hysteroscopy, including precise estimates of pain scores according to the specific hysteroscopic procedure, it did not provide data about patient acceptability or procedural success/failure.14, 15 Thus, there is an urgent need to provide real-life data to validate the suggested benchmarking norms shown in Table 1. To this end, the BSGE plans a large, real life, multi-centre cohort study to provide both procedural and patient acceptability and wider experience data to inform practice.
The outpatient hysteroscopy community also needs a core outcome set so that a minimum set of data are collected in a standardised fashion, which should include acceptability and success data, defined and measured in the same way e.g. a 10-point visual analogue scale. We have registered this project on the COMET Core Outcome Measures in Effectiveness Trials website (https://www.comet-initiative.org/Studies/Details/3049). Published studies, presenting a minimum set of routine core outcome data, will allow the suggested Clark ratio norms to be validated and amended accordingly. Importantly, uniform data collection, agreed by consensus, can then provide data for other, more invasive hysteroscopic/intrauterine procedures like endometrial ablation, myomectomy and retained pregnancy tissue extraction. The data for such hysteroscopic/intrauterine interventions are currently lacking preventing the derivation of benchmarking Clark ratios.
Clinical Use of the Clark Ratio
The Clark ratio is a simple metric, and is anticipated to be easily interpretable, providing more granularity about the conduct of common hysteroscopic procedures.
Clinicians or the wider service can use this simple metric to evaluate the quality of their procedures by comparing their local data with the suggested norms shown in Table 1. Thus, generation of the Clark ratio can contribute to quality assuring outpatient hysteroscopic services. The metric is generalisable to individual settings and practices and should be used to act as a baseline benchmark. Clark ratios below these recommended standards should trigger clinicians to reflect on their practice, examine potential areas for improvement and put into place remedial measures. A hierarchy could be adopted to guide practice, based upon the point estimate and derived ranges, and appropriate actions implemented (Table 2). To make the Clark ratios simpler to understand the categorical ranges can be converted into numerical values (Clark grade) from 1 (unacceptable) to 4 (excellent) (Table 2). The selected cut-offs to rate the quality of performance and inform clinical actions, whilst rounded up and simplified are broadly based upon the 95% confidence limits derived from the available RCT evidence for success/failure and acceptability presented in Table 1. However, the proposed thresholds will need testing and validating.
When applying the Clark ratio thresholds there must be a global overview of rates of acceptability and failure. If rates of failure (denominator) exceed 20% then the Clark ratio will always fall below 5 and be graded as unsatisfactory. However, in an extreme (and probably unrealistic) situation where rates of acceptability (numerator) fall below 50%, a borderline Clark grade could be achieved if the failure rate is below 10% (equating to a Clark ratio ≥5). Most clinicians and patients would consider an acceptability rate below 50% to be undesirable regardless of the rates of procedural success. Thus, when reporting an individual clinician or overall units performance using the Clark ratio, the mean values for acceptability and failure rates should also be presented to aid interpretation. Acceptability rates below 50% should be considered insupportable and rates below 80% considered undesirable even if the Clark ratio falls within a satisfactory range (Table 2).
Discussion
The presentation of procedural acceptability or success rates in isolation limits understanding of the quality of outpatient/office procedures, particularly those that are potentially unpleasant and painful in conscious patients. The use of the CLARK ratio allows the potential trade-off between patient acceptability and successful completion of the procedure to be accounted for. This metric can therefore enhance interpretation about the appropriateness and/or conduct of a procedure in a conscious patient.
Agreeing a core set of outcomes and outcome measures for outpatient hysteroscopy is in process. Whilst this is awaited, we will need to be practical. Procedural “success” is intuitive and generally well understood; an adequately completed procedure, achieving its intended purpose, and without serious complications. A failed procedure (the denominator of the Clark ratio) is in turn one that is not completed satisfactorily (e.g., an inadequate view within the uterine cavity to make a diagnosis; failure to completely remove an endometrial polyp; uterine perforation complicating an endometrial ablation). The concept of “acceptability”, however, is less clearly defined and is multifaceted.8, 9 Patients may consider their overall care adequate e.g., the attitude of the staff, the facilities etc. but we are primarily interested in the acceptability of the procedure itself, namely was the overall experience of the procedure (e.g., exposure, positioning, level of anxiety/distress etc.) positive and maybe most importantly, the degree of pain or discomfort, considered reasonable. The patient should feel satisfied with the outcome and the overall experience of the procedure. The RCTs identified as the highest quality and most relevant to inform the estimates of acceptability in the RCOG guideline for “Best Practice in Outpatient Hysteroscopy”,2 used dichotomous3 and ordinal scales to assess acceptability (“yes” vs. “no”; 4-point ordinal scale–“totally acceptable”; “generally acceptable”; “fairly acceptable” vs. “unacceptable.”).4-6
In the absence of a core outcome set, it is proposed that a pragmatic approach to evaluating procedural “acceptability” is used when auditing practice, and when reporting in a research study, but the method of measurement and assessment is clearly reported. For instance, any measure or scale assessing the patient experience during the procedure (e.g., satisfaction, acceptability) should be considered appropriate. However, where a hierarchical scale is used it is recommended that a neutral response (e.g., “neither acceptable or unacceptable”) is considered an “unacceptable” response.
Study Limitations
The utility of the Clark ratio may be compromised if it is only meaningful and therefore applicable if the rates of acceptability have to be 80% or more and rates of failure 20% or less. However, external testing and validation studies within independent data sets are likely to show such extremes of success and experience are improbable making the metric widely applicable. The Clark ratio concept implies that an association between procedural acceptability and success is linear. Future, prospective observational series using “real-life” settings like those proposed in this opinion piece, rather than extrapolating rarified data from RCTs, can explore the nature of the relationship, if any, to see whether indeed the change in one variable causes a constant, proportional change in the other, or whether threshold effects exist for example.
Another limitation of the Clark ratio concept relates to the selection of cut-offs to rate the quality of performance and inform clinical actions. These thresholds, whilst rounded up and simplified, however, are broadly based upon the 95% confidence limits derived from the best available RCTs evaluating specific diagnostic and operative outpatient hysteroscopic procedures for success/failure and acceptability (presented in Table 1). However, the proposed thresholds will need testing and validating in future studies.
Conclusion
Simple metrics are straightforward and can give fast insights without complex analysis. They can be easy to understand, be good for communication and arguably allow timely decision-making. However, they lack depth and can be misleading without context, e.g., details about populations, giving a false impression. The ideal quality assurance should use simple metrics like the Clark ratio for quick snapshots when human resources are limited, but where resources allow deeper analysis will provide more context. In addition to quality assurance, the Clark ratio can be presented as a standard outcome within research studies facilitating meaningful comparisons and helping identify effective interventions, such as innovations in patient information provision, adoption of novel analgesics, development of multifaceted management protocols and introduction of new technologies.
The derivation and use of this novel, simple metric, the Clark ratio, is proposed for quality assurance and as an outcome in research studies involving outpatient hysteroscopic procedures where patient experience is evaluated. The data provided here are for common hysteroscopic and intrauterine procedures. However, this metric can also be applied to a wider range of hysteroscopic, gynaecological and non-gynaecological procedures conducted on conscious patients where reliable data for patient acceptability and procedural success exist. However, the reader should be aware that the Clark ratio, that the potential trade-off exists between patient experience and procedural feasibility, remains theoretical in nature and requires further exploration and validation in future observational studies. Moreover, the stability and utility of the proposed thresholds, including the mean (standard deviation/ranges) of the numerator and denominator derived from many Units / practitioners, will require prospective analysis as part of a validation exercise. In this way, the acquired data can be used to accept or reject the validity and utility of the proposed metric and its suitability to be used in day-to-day clinical practice to help benchmark hysteroscopic performance and services.
