An Environmental Scan and Evaluation of Quality Indicators Across Canadian Kidney Transplant Centers

Background: Kidney transplantation is the optimal treatment for an individual requiring kidney replacement therapy, resulting in improved survival and quality of life while costing the health care system less than maintenance dialysis. Achieving and maintaining a kidney transplant requires extensive coordination of several different health care services. To improve the quality of kidney transplant care, quality metrics or indicators that encompass all aspects of the individual’s journey to transplant should be measured in a standardized fashion. Objective: To identify, categorize, and evaluate strengths and weaknesses of kidney transplant quality indicators currently being used across Canada. Design: An environmental scan of quality indicators being used by kidney organizations and programs. Setting: A 16-member volunteer pan-Canadian panel with expertise in nephrology, transplant, and quality improvement. Sample: Transplant programs, as well as provincial transplant and kidney agencies across Canada. Methods: Indicators were first categorized based on the period of transplant care and then using the Institute of Medicine and Donabedian frameworks. A 4-member subcommittee rated each indicator using a modified version of the Delphi consensus technique based on the American College of Physician/Agency for Healthcare Research and Quality criteria. Consensus ratings were subsequently shared with the entire 16-member panel for additional comments. Results: We identified 46 measures related to transplant care across 7 Canadian provinces (9 referral and evaluation, 9 waitlist activity and outcomes, 6 hospitalization for transplant surgery, 12 posttransplant care, 6 organ utilization, 4 living donor). We rated 24 indicators (52%) as necessary to distinguish high-quality from low-quality care, most of which measured effective (n = 10) or efficient (n = 6) care. Only 7 (15%) of 46 indicators evaluated person-centered or equitable care. Fourteen common indicators were measured by 5 of 7 provinces, 10 of which were deemed “necessary,” measuring safe (n = 2), effective (n = 5), efficient (n = 2), and equitable (n = 1) care. Limitations: The panel lacked patient and allied health representation. Conclusions: There are a large number of kidney transplant quality indicators currently being used in Canada, some of which are common across provinces and focus primarily on measuring effective care. Person-centered and equitable care indicators were lacking, and only half of these indicators were deemed “necessary” for quality improvement. Our results should complement ongoing work to achieve national consensus on the standardization of quality indicators in kidney transplantation.


Introduction
Kidney transplantation is the optimal treatment for individuals requiring kidney replacement therapy (KRT) as it results in improved survival, enhanced quality of life, and reduced health care costs compared with maintenance dialysis, [1][2][3][4] For an individual with kidney failure, achieving and maintaining a kidney transplant long-term require extensive oversight, expertise, and coordination of health care services across various health care sectors (eg, hospital transplant programs, organ procurement organizations, chronic kidney disease programs, primary care). Many challenges to providing high-quality kidney transplant care exist, including lack of access demonstrated by variability in referral and transplant rates across programs, prolonged recipient and donor evaluation wait times, and stagnant living donor numbers. [5][6][7][8] To understand whether transplant programs are providing high-quality care and to facilitate improvement activities, a comprehensive view of the individual's journey to transplant and beyond must be translated into measurable quality-of-care metrics (or quality indicators). However, few examples of how to measure the quality of kidney transplant care exist in Canada. 9,10 Quality indicators are quantitative or qualitative measures that can be described with different frameworks. The Institute of Medicine (IOM) domains determine whether the care provided is safe (free from harm), effective (evidence-based), efficient (limits waste), timely (available when needed), person-centered (focused on the individual), and equitable (equally available). 11 Quality indicators can also be categorized into 3 aspects of quality described in the Donabedian framework: structure measures (the setting in which kidney transplantation occurs, for example, the presence of a multidisciplinary team), process measures (the steps to deliver care, for example, the proportion of recipients on pneumocystis jiroveci [PJP] prophylaxis), and outcome measures (how that care will impact the individual, for example, graft survival). 12 Not only can quality indicators determine whether predefined benchmarks are met (ie, quality assurance), but they can also be used by frontline health care providers for microsystem quality improvement (ie, small-scale projects that test iterative changes aiming to improve local performance). 13 In Canada, transplant programs are somewhat siloed, and little is known about the provincial scope and overlap of existing quality indicators and the different domains of health care quality currently being measured in kidney transplantation. The objective of this study was to identify and describe the characteristics of kidney transplant quality indicators currently being used in Canada, as well as to highlight their strengths and weaknesses based on the American College of Physicians/ Agency for Healthcare Research Quality criteria. 14 We anticipate this work will provide a pan-Canadian resource of existing quality indicators and complement future studies that aim to standardize the use and operationalize definitions for quality indicators used in kidney transplant care.

Environmental Scan of Quality Indicators
We collected kidney transplant quality indicators currently in use by provincial agencies and transplant programs within Canada between February 2019 and December 2020. We also asked programs to provide examples of any ongoing or past quality improvement initiatives related to transplant care. We contacted leadership from provincial agencies, nephrology division directors, transplant program administrative directors, and transplant nephrologists across Canada. We stopped the environmental scan once we received multiple responses from the majority of provinces and received no further feedback from other provinces.
We combined quality indicators into a single measure where similarities existed (eg, the proportion of individuals treated with maintenance dialysis referred for transplant was combined with the proportion of individuals treated with maintenance dialysis referred for transplant stratified by type of dialysis modality). Next, we organized each indicator into categories of transplant care (eg, referral and evaluation, waitlist activity and outcomes, hospitalization for transplant surgery, posttransplant care, organ utilization, living donor). We then classified indicators using the IOM (safe, effective, efficient, timely, person-centered, equitable) and Donabedian (structure, process, outcome) frameworks of health care quality. We also included balancing measures to evaluate any unintended negative effects that occur with provision of care (ie, infectious complications of immunosuppression). 15

Indicator Evaluation
We rated the identified indicators using a modified version of the American College of Physicians/Agency for Healthcare Research and Quality performance measure review criteria, which included the following dimensions (Supplemental Table 1): 13,16,17 • • Importance: The metric will lead to a measurable and meaningful improvement, or there is a clear performance gap. • • Evidence base: The metric is based on high-quality and high-quantity evidence. • • Measure specifications: The metric can be clearly defined (ie, numerator and denominator) and reliably captured. • • Feasibility and applicability: The metric is under the influence of health care providers and/or the health care system, with data collection and improvement activities both feasible and acceptable.
We rated each of these dimensions on a 9-point scale, where 1-3 indicated "does not meet criteria," 4-6 "meets some criteria," and 7-9 "meets criteria." Based on these ratings, each indicator then received a final global rating based on its overall ability to distinguish good quality from poor quality. 16 For the global rating, we considered quality indicators as "necessary" if the median rating was 7, 8, or 9 and there was no disagreement by any member. We considered indicators as "unnecessary" if the median rating was 1, 2, or 3 and there was no disagreement by any member. We considered all other indicators as "supplemental."

Overview of the Modified Delphi Process
We used a modified Delphi approach to evaluate the strengths and weaknesses of different transplant indicators. This process has been described previously to help classify quality indicators. 15,[18][19][20][21][22][23][24] The Delphi panel consisted of a 16-member volunteer national nephrology quality indicator committee with representatives from 7 of 10 provinces. The majority of members possessed advanced training or expertise in quality improvement. From this committee, a 4-person kidney transplant subcommittee was formed. The quality indicators from the environmental scan were made available to the kidney transplant subcommittee. Each member of the kidney transplant subcommittee separately rated the quality indicators in advance of a teleconference. The subcommittee members then discussed the individual ratings at the teleconference and agreed upon a group rating for each of the American College of Physicians/Agency for Healthcare Research and Quality domains. 14 Next, we circulated the initial group ratings to each subcommittee member for further feedback and to confirm consensus. Last, we shared the final group ratings with the entire 16-member committee, with further discussion of any ratings that differed by ≥3 points. Formal research ethics board review was not required by Queen's University based on the Tri-Council Policy Statement for ethical human research, as the focus of the study involved quality indicators and not human participants.
The range of indicators collected by provinces was between 12 and 34. Thirteen indicators were only being measured by a single province and not measured elsewhere. We observed overlap with 14 indicators used by at least 5 of 7 provinces. Of these 14 common indicators, there were 4 process measures, 7 outcome measures, and 3 balancing measures. These common indicators fell into the IOM categories of safe (n = 2), effective (n = 5), efficient (n = 2), timely (n = 1), person-centered (n = 2), and equitable (n = 2) care. The panel rated 10 of these indicators as "necessary" to distinguish high-quality from poor-quality care.
Of the 3 provinces that described local quality improvement initiatives, 2 had initiatives related to increasing access to transplant and living donation (eg, educational and cultural outreach programs). Other initiatives focused on providing safe care including vaccination before transplant, PJP prophylaxis, and cytomegalovirus (CMV)/BK viremia surveillance after transplant. Only 1 initiative was person-centered and focused on improving the posttransplant experience.
Three common themes emerged during the rating process. First, indicators could be precisely defined and specified, but the definitions were variable and dependent on local practice. For example, for the indicator "time from transplant referral to waitlisting," some programs received most/all required information at the time of referral, whereas other programs must initiate further work-up before wait-listing. Second, the feasibility to collect data or the documentation burden for any given indicator depends largely on data infrastructure or the capabilities of a transplant program's electronic medical record (EMR). This theme was particularly problematic for indicators that relied on education or quality of life (eg, being informed about transplant as an option). Finally, we rated most indicators (26/46, 57%) as usable for quality improvement (ie, under the influence of health care providers and/or the health care system, with data collection and improvement activities both feasible and acceptable). Exceptions included indicators that change too slowly for rapid cycle improvement activities (eg, kidney transplant prevalence), may not be under the sole control of health care providers (eg, % of highly sensitized individuals who receive a transplant), or may not be modifiable (eg, wait times by blood type).

Discussion
This environmental scan found 46 kidney transplant quality indicators currently being used across transplant programs and kidney agencies in Canada. The indicators spanned all major periods of transplant care, but there were few living donor-specific indicators. Using the IOM framework of health care quality, the majority of indicators were mapped to safe, effective, efficient, and timely care, revealing gaps in measuring person-centered and equitable care. There was variation in the indicators being used across provinces, with only 14 indicators common among most provinces. These results also provide kidney transplant programs with a selection of 24 "necessary" indicators to distinguish high-quality from low-quality care and highlights the need to ensure all domains of health care quality and aspects of kidney transplant care are being monitored. Our study complements ongoing work that aims to achieve consensus on which indicators should be used in kidney transplantation across Canada. 25 Table 1   Waitlist characteristics (total no., blood type, age, sex, allocation points)

Table 2. (continued)
Quality indicators in the field of organ transplantation were evaluated in a systematic review by Brett et al and were collected via a Medline, Embase, and Cochrane Central Register of Controlled Trials search from inception until 2017. A total of 114 unique indicators were found, 65 of which were related to kidney transplant. 10 There are similarities of note between this systematic review and our study. First, both studies categorized indicators by the IOM and Donabedian frameworks and demonstrate the breadth and quantity of measurement occurring in this field. Second, the majority of indicators were in the quality domains of safety and effectiveness, whereas a minority measured equity or person-centeredness. 10  Our study also identified several important issues with kidney transplant quality measurement currently occurring in Canada that warrant further attention. Notably, not all areas of transplantation are receiving equal attention, most notably gaps in living kidney donation, which is particularly important because rates of living donation have been stagnant in Canada for some time. 26 Similarly, although health care systems around the world are being redesigned to achieve the quadruple aim (improve population health, experience of care, reduce per capita costs and increase joy in work), 27,28 our study found no patient-reported outcome or experience measures (eg, functional impairment or quality of life) or cost/resource utilization indicators being routinely measured. We also identified many "necessary" process (n = 10) measures, with 70% used by more than 1 province. This is important to point out as process measures are more easily measured and can demonstrate change more rapidly than outcome measures, making them an important component of rapid-cycle microsystem quality improvement activities. 29 Recently, 2 Canadian studies attempted to address some of the described shortcomings. The first was a Canadian consensus workshop of key stakeholders in transplantation (physicians, patient representatives, allied health) that reviewed potential quality indicators according to predefined criteria (eg, relevant, actionable, measurable) and provided a recommendation of essential, optional, or exclude. 25 The second study was a Delphi panel to achieve consensus on quality indicators used to measure the efficiency of the living kidney donor evaluation. 30 An important observation from the indicators selected in these 2 studies is that the equity domain of quality continues to lack representation, as confirmed in our environmental scan. In the first workshop described above, indicators classified as equitable care included donor candidates deemed suitable to donate and number of living donor kidney transplants performed. In the living donor Delphi panel, there were no measures of equity. Equitable care metrics are important to develop because of known existing health care disparities in minority populations. 31,32 For example, women, individuals of lower socioeconomic status, and those of African or Indigenous backgrounds have consistently been shown to have a reduced likelihood of transplant referral, wait-listing, and subsequent kidney transplant, emphasizing the need to include patients and vulnerable populations as key stakeholders in an effort to fill the identified gaps. [33][34][35] Another important observation was the sheer number of indicators that resulted from these 2studies (54 and 26, respectively), in addition to the 46 indicators we found. Therefore, prioritization of indicators will need to occur as transplant programs may not have the means to undertake this amount of measurement and corresponding quality improvement activities. Our goal of this study is to complement the ongoing efforts by Knoll et al and Garg et al. in prioritizing indicators. Ideally, future indicators could be developed and piloted at a local level prior to them being implemented at a national level. A key consideration would be to promote regular reevaluation of whether these indicators remain valid and suitable for ongoing use and retire those indicators that no longer represent the interests of the various stakeholders. Ideally this reevaluation could be done at a national level to maintain alignment between independent programs. Our 16-member pan-Canadian quality collaborative is currently working with senior leadership across the country (Canadian Senior Renal Leaders Community of Practice group) to prioritize quality indicators in the various domains of kidney care. We hope this will help focus future local quality improvement initiatives on a few key areas and encourage collaboration between centers embarking on similar initiatives. 36 Our study assists the transplant community with these prioritization activities by identifying 24 "necessary" indicators (10 measured by 5 of 7 provinces) that programs are already devoting resources to as a starting point for standardization across Canada. Strengths of this work include the structured approach to indicator categorization and evaluation through applying the IOM and Donabedian frameworks along with the criteria used by American College of Physicians/Agency for Healthcare Research and Quality. In addition, we involved nephrologists with advanced training and real-life expertise in kidney transplantation and/or quality improvement to ensure relevance to frontline improvement efforts. Our panel included individuals who represent most regions of Canada which helped ensure that quality indicators would be relevant across different health care settings.
Limitations deserve mention. First, we did not receive indicators from all kidney transplant centers across Canada, and therefore our list of indicators is not exhaustive. Second, our study focused on identifying and rating current indicators without clarifying the operational definitions or how each indicator is being used for quality improvement. Third, we did not automatically include indicators from the Canadian Organ Replacement Register (a nationwide database reporting longterm trends in organ donation and transplantation) unless explicitly stated by programs that they were used for quality improvement activities. Fourth, our evaluation of indicators was not anonymized, which can result in the group arriving at conclusions primarily because others are doing so (the bandwagon effect). 18 Fifth, the panel was composed mainly of physicians and thus did not include the valuable perspectives of other stakeholders, including administrators, allied health professionals, and, importantly, patients. Finally, there is subjectivity when using the IOM classification to categorize indicators. For example, the time from transplant referral to receipt of a transplant could be classified as efficient or timely care. The determination of IOM domain should ideally be based on how that indicator is used to drive quality improvement activities.
In summary, we identified 46 kidney transplant quality indicators currently being measured across Canada. There was a paucity of living donor indicators, and we found that the majority of indicators focused on safe, effective, and efficient care, with significant gaps in measuring person-centered and equitable care. Ten of the necessary indicators were common among most provinces. We hope this overview serves as a useful guide and starting point that supports ongoing efforts to develop, standardize, and curate kidney transplant indicators across Canada.