Health System Quality and Performance Indicators for Evaluating Home Care Programming: A Scoping Review

A systematic scoping review of health system Performance and Quality Indicators (PQI) related literature for evaluating publicly funded home care programming was conducted. Through the scoping review we identify PQIs, map these based on quadrants of the Institute for Healthcare Improvement’s (IHI) Quadruple Aim, classify these PQIs under general “areas of focus” categories, and present recommendations for use. We selected all articles that identified, referenced, proposed, and/or described at least 1 PQI to measure home care service programming. Minimum inclusion criteria were a description of 1 or more PQIs designed to evaluate and/or measure home care programming, outcomes of home care clients and/or caregivers, or overall system impacts of home care service provision. The review identified 3,475 citations. A review of the abstracts led to the retrieval of 510 full-text articles for assessment of which 105 articles were selected for detailed review and extraction of identified PQIs. Of the 105 identified articles a total of 829 unique PQIs were gathered. Based on the IHI Quadruple Aim, 661 unique measures were identified as Clinical Outcome, 35 as Healthcare Provider Satisfaction, 99 Patient Experience, and 34 in the Financial/Sustainability quadrants. Many patient outcome specific PQIs exist, followed by patient experience measures. Very few PQIs exist in the financial or healthcare provider experience quadrants. The lack of indicators across all the IHI Quadruple Aim quadrants is an opportunity for further development supporting home care quality oversight.


Introduction
As of 2018, Canada spent $71.7B annually on hospital-based care making up 28.3% of all healthcare expenditure. 1 Hospital care is often the safety net for insufficient community-based services, resulting in inefficient use of this expensive resource. 2Healthcare systems across Canada are investing in community-based programs such as home care with the goals of increased quality of care, improved experiences of clients, and reduced use of expensive, often unnecessary, emergency department, and hospital-based care.With a goal to lessen the costs of healthcare, $11B in federal investments were announced in 2017 for home and community care programs. 36][7][8][9] Healthcare system capacity pressures, a growing and aging population, and increasing costs for complex hospital care are complicating how health systems can provide all the services.
In Canada, a national home care measurement framework does not exist, and there is limited reporting on the current state of home care investments and related outcomes.Yet, home care service efficiency and effectiveness is vitally important to ensure overall health system sustainability. 10In addition, there is limited reporting and understanding of the impacts of targeted home care investments, home care client service use, or data specific to client reported outcomes/ experiences, even though this component of the continuum of care may be critical to ensuring the overall health system is functioning sustainably.
To improve home care service delivery, and to provide useful information for policy makers, funders, and healthcare executives, a common measurement and evaluation framework is necessary that focuses on home care service programming.Measurement frameworks conceptually are system-wide balanced scorecards populated with Performance and Quality Indicators (PQIs) providing information on system outcomes that span the quadrants specific to healthcare system performance. 11For example, PQIs focused on portraying how the health system is improving population health (patient/system outcomes), patient experience, healthcare provider satisfaction and value for money/healthcare sustainability as outlined by the Institute for Healthcare Improvement (IHI) through the Quadruple Aim framework. 12PQI measures utilize date collected from various inputs within the health system that translate to information to informhealthcare leaders, policy makers, users of the system and frontline providers on the current ssystem outcomes.Various organizations and health systems are beginning to utilize the IHI Quadruple Aim framework as a way to assess whole system performance in a balanced way, most recently the Canadian Institutes for Health Research (CIHR) has adopted the IHI Quadruple Aim performance model within their 2021 to 2026 strategic plan. 13he performance quadrants can be applied to a wide range of healthcare challenges, such as reducing health care provider burnout (satisfaction, joy of work), combating risking healthcare expenditures (financial/value for money), optimizing services to improve client outcomes (population health/outcomes), and improving on the overall experiences people have receiving services (provider experience). 14Various health systems, and health researchers have implemented and/or conceptualized performance indicators as tools for quality improvement and posted/published these at national, regional, or institutional levels. 15The purpose of this scoping review was to review recent literature on home care program performance and quality to identify the body of PQIs, how these PQIs fit into the IHI Quadruple Aim framework, and common areas of focus.This included categorizing and mapping articles and PQIs based on the general areas of focus, population of focus, and the quadrants of the IHI Quadruple Aim.Secondarily, the scoping review findings are examined specific to understanding a need for future effort in developing additional PQIs and/or a home care program evaluation framework.

Methods
We identified relevant articles using a similar method as conducted by Stelfox completing a similar scoping review process focused on trauma care QPIs. 16We completed a detailed search and review of home care quality measurement literature, which included articles published between January 1, 2010 up to July 1, 2020 within the following databases: OVID MEDLINE, OVID Embase, EBSCOHost CINAHL, and the Wiley Cochrane Library, as well as Google for gray literature.The Boolean Operators "AND" and "OR" were used with the 2 main search concepts: home care and quality.For example, the home care concept was searched by MeSH headings Home Care Services and Palliative Care, and keywords home, home care, palliative, adult day programs, and primary care.Across some jurisdictions not all programs are clearly listed as "home care" but may have other nomenclature, such as those with services integrated within home care programs such as palliative (community based), adult day programs (as extensions of home care programs), and those where home care in integrated in other programs, such as where primary care bundles home care within.The quality concept was searched by MeSH headings such as Benchmarking, Quality Control, and keywords such as quality measures, outcomes evaluation, quality assurance, quality indicator, system outcomes, and others that are associated within the quality and performance literature.These terms helped to identify content that may contain home care program specific system/quality measures within the peerreviewed and gray literature (health authority reports, government reporting, and quality councils) which increased the sensitivity of the search strategy.
The search was restricted to articles written in English with a specific research question, methods, and results reported.Adapted from the Canadian Institute for Health Information (CIHI) Performance Measurement Framework for the Canadian Health System (2013), for this review PQIs are those measures designed to evaluate the outcomes of people, and/or of the system that focused directly on home care programing.This search included websites of relevant organizations with a home care focus (national, operational, and strategic) as well as searching via Google using the same search combination.Author R.Z. performed the searches.Using the same search criteria and technique as the first search, we completed a second search with updated date ranges of July 1, 2020 to June 15, 2021.

Article Selection
Articles that identified, referenced, or proposed 1 or more PQIs for evaluating home care service programming were selected.Minimum inclusion criteria were (1) the articles included at least 1 PQI; and (2) the PQI was designed, described, or proposed to evaluate the outcomes of the home care program services.As we conducted our 2 sets of reviews, we did not include articles that did not reference a PQI directly or not specific for home care service/program outcomes such as primary care programs/services not affiliated with home care programming/services, facility based LTC, or facility-based hospice care.We chose these liberal criteria to optimize the chances to identify as many relevant PQIs as possible as it was anticipated that few PQIs exist specific to the finance, and healthcare provider satisfaction quadrants of the IHI Quadruple Aim framework.

Article Review
Through 3 phases, we identified eligible articles.In the first phase, 2 researchers (M.J. and A.G.) independently reviewed the titles and abstracts of each of the articles identified through the first search conducted and identified the articles that met the selection criteria.Articles selected by either reviewer were included for the next round of review.The second phase focused on review of the articles identified by the second librarian search.Two independent researchers (M.J. and L.S.) conducted this title and abstract review; all articles found to be relevant by at least 1 researcher were retained and included for additional review.In phase 3, full text of the remaining articles (from phase 1 and phase 2) was individually reviewed and those with at least 1 PQI specific to home care programing/services were included.
Researcher (M.J.) studied each of the identified articles that satisfied the minimum inclusion criteria and abstracted data and entered information using a standardized form.Articles with relevant PQIs were classified using the prespecified model including demographic area and results for IHI Quadruple Aim quadrant classification, characteristics of article type, and country of origin.Owing to the heterogeneity of the results, articles, and PQIs were summarized as counts and proportions using Microsoft Excel 2016.

Results
The first literature search identified 3,061 citations between dates January 1, 2010 up to July 1, 2020 (phase 1).The second literature search identified 414 citations between dates July 1, 2020 and July 15, 2021 (phase 2).Together 3,475 unique citations were identified.A review of the titles and abstracts of the 3,475 articles led to the identification and retrieval of 510 full-text articles for assessment.The 510 articles were reviewed of which a final sample of 105 articles were selected (phase 3) for extraction of identified PQIs as these were the articles that had least 1 relevant PQI that met the definition (Figure 1).

Description of the Articles
Over half of the articles (59%) were published between years 2016 to 2021 as outlined by Figure 2. The majority (85%) of the articles were from academic research (Table 1).Twentyfive different countries are represented within the 105 articles with the USA (37%) and Canada (18%) making up 58 of the 105 articles (Figure 3).The demographics of focus within the articles were inclusive of all population groups, making up 67% of all articles.Those articles that did not describe population characteristics were grouped under "all" under the assumption there was no specific cohort intended.Those articles (24%) that focused on the palliative demographic as a primary population were categorized in this manner.When sorting and categorizing each of the 105 articles on which Quadruple Aim quadrants were targeted, 99 articles contained Patient Outcome PQIs, 40 contained Patient Experience PQIs, 19 contained Financial PQIs, and 9 contained Provider Satisfaction PQIs; some articles contained multiple quadrants (Table 2).Not all articles had PQIs in each of the quadrants.Most articles only focused on Patient Outcomes (51.4%).Only 2 articles had PQIs listed making up all 4 quadrants, 1 article was purely focused on the finance quadrant, and only 4 articles solely referenced patient experience PQIs (Tables 2 and 3).829 unique PQIs.The 829 unique PQIs were categorized within 1 of 60 areas of focus.These areas of focus were developed as a component of the categorization exercise undertaken by the expert review in phase 3 for the purpose of grouping like indicators together as sub-categories within the IHI Quadruple Aim.For example, indicators that focused on Activities of Daily Living patient outcomes were grouped under the "ADL" as an area of focus (43); indicators focused on Emergency Department usage were grouped together under "Acute Care Usage-Emergency Department" (22), those focused on patient cognitive performance were grouped under "cognitive performance" (29), and those related to wait time for home care services were grouped under "Timely Service Availability" (14) (Table 4).Most indicators were categories within the "General Patient Outcome" area of focus.These indicators specified a variety of general patient outcomes mostly based on home nursing assessments, such as hearing, vision, mood, sleep, weight, diet, delirium, ability to communicate, and other clinical assessments (Table 4).IHI Quadruple Aim categorical sorting of the 829 unique PQIs resulted in 661 unique measures identified as Clinical Outcome, 35 as Healthcare Provider Satisfaction, 99 as Patient Experience, and 34 in the Financial/Sustainability quadrants (Figure 4).The 829 PQIs were also classified by the IHI Quadruple Aim quadrants and grouped within general focus areas (Table 4 and Figure 4).Notably, 50 of the 60 areas of focus had indicators classified within only 1 of the IHI Quadruple Aim quadrants.The remaining 10 areas of focus had PQIs classified in only 2 of the IHI Quadruple Aim quadrants.No area of focus had PQIs in 3 or all 4 quadrants (Table 4).

Description of the Quality and Performance Indicators
Main highlights of the findings based on IHI Quadruple Aim and focus area sorting are the discovery of the 661 unique PQIs within the clinical outcome quadrant focus on areas ranging from acute care system utilization including direct patient health outcomes making up 85 individual PQIs to 226 patient specific health outcome PQIs, and 63 specific to palliative outcomes and location of death/care at end of life.Additionally, within the financial quadrant (making up 34 PQIs) the primary areas of focus are specific to cost per client, either day, or ED/acute care use, or all services at the population level.Furthermore, some PQIs break down these costs by province, health authority or community.The PQIs within the patient experience quadrant primarily focused on questions specific to patient satisfaction with their care and their experiences as home care clients, their quality-of-life scores, and the experiences of their caregivers (family and/or loved one).Lastly, the provider satisfaction quadrant (35 PQIs) focused on 4 primary areas, direct healthcare provider satisfaction score (e.g., from survey tools), education availability, work standards, and staff/workload to client ratios (Table 4, Figure 4).

Discussion
In our review of the literature on PQIs that focus on home care performance and/or outcomes, we identified 105 articles and 829 unique PQIs with 250 duplicates.We identified an extensive body of literature that references ways to evaluate home care programs using at least 1 PQI.Additionally, we identified the lack of the application of program wide PQIs that use the IHI Quadruple Aim as a framework for overall evaluation of programs and health systems.Articles available are mostly focused on clinical outcome related measures, but we did identify articles across all 4 IHI Quadruple Aim quadrants.Studies involving patient experience measures appeared the next most often.Few articles were identified that focused solely on financial or provider satisfaction PQIs for evaluating home care service and programs, suggesting a need for additional empirical research focused in these quadrants.
Importantly, in various jurisdictions across the globe, general assessments of patients are required to be conducted at a set frequency utilizing a standard tool.For example, in countries such as Canada, Australia, and UK, these assessments are conducted primarily utilizing the interRAI Resident Assessment Instrument-Home Care (RAI-HC)©.In this example, the RAI-HC collects a large amount of information and covers the majority of the categories listed within the general patient outcomes area of focus.The imbalance in PQIs across the 4 IHI quadruple AIM quadrants may be because when clinical information is gathered, this information due to availability can more easily become a PQI primarily based on that availability (data driven PQI).It is important to understand that the PQIs identified through this research did not assess for assessments of validity, reliability, or other means of validating/confirming the PQIs are scientifically appropriate for utilization within the balanced scorecard approach.Even with largely adopted standard tools, information gathered does not necessarily equate to availability of appropriate PQIs as illustrated by Wagner who examined the use of interRAI assessment-generated data as PQIs and identified there is insufficient reporting of validation processes and reliability of these PQIs being utilized to improve home care programming or patient outcomes. 18hey recommend that, since these PQIs are already gathered and used in many jurisdictions based on availability, investigations on appropriateness, reliability, and validity be undertaken.A future component of this research is to scientifically select indicators to comprise a balanced IHI quadruple AIM framework scorecard, and to assess the validity and reliability of the indicators and framework.
The single most important result of this review is that we did not find a common application of clearly defined universally accepted PQIs for evaluating home care service delivery and outcomes utilizing the IHI Quadruple Aim.There is a need to have transparent, explicit, systematic, data-driven performance measurement feedback systems but this cannot occur if there are no generally agreed upon measures. 16ommon measures are needed to support comparability within and across systems, jurisdictions, and countries.It is difficult for quality to be improved and for data-driven improvement through investments if common measures are not in place.Adding complexity, PQI applicability, and appropriateness may shift depending on client type and duration of services, such as activities of daily living scores may shift based on aging (worsen) or based on recovery from illness or injury (improvement).Examining the applicability based on populations served (long-term clients with multiple chronic diseases or short-term patients recovering from illness or injury) may result in some PQIs more being appropriate than others in measuring outcomes.
There is a lack of a system-focused home care evaluation framework that has balanced measures across all quadrants of the IHI Quadruple Aim.Evaluation frameworks provide an understanding of the impacts of investment decisions for healthcare policy makers and leaders.It is important for all health systems to understand the impacts of resource delivery within home care programs to direct these programs toward overall improved outcomes and health system sustainability.Developing quality measures is challenging.As per CIHI, developing home care measures is a multi-year process and, once measures are created there is a further   Note.Each identified article is listed once based on the IHI Quadruple Aim Quadrants the PQIs within the article were categorized under.Some articles contained PQIs categorized under 1 quadrant while others contained PQIs categorized under multiple quadrants.
process to evaluate their effectiveness. 19We have identified that the home care performance systems are not mature in their development; though there are numerous measures, but these are not organized in a way that allow for a health system to measure impacts using a balanced health system performance and measurement framework such as the IHI Quadruple Aim.One significant challenge for health systems is to collect the information properly and effectively apply measures.A mature data analytic architecture including data scientists who have robust understandings of system structures (data and clinical) is required; this is especially the case in more challenging for under-resourced systems.
In this research we identified 829 unique indicators and 250 duplicates.A need for consolidation and adoption of a common home care service performance framework is needed to allow for comparisons across systems and programs, and for policy makers to begin to understand how various changes to home care programming (investment or disinvestment) have impacted outcomes in a way that is understandable across jurisdictions.Consideration should be given to the differences between systems in terms of structure and function but comparability is required for health systems of all types (traditional, forming, and learning heath systems) to understand their performance in comparison to their own goals and those of other jurisdictions.With 661 individual clinical outcome indicators and only 35 individual financial or 34 healthcare satisfaction indicators that there may be an imbalance in system-level measures being considered by researchers and policy makers.This is highlighted with the identified articles with most articles (99) including clinical outcome PQIs, compared to 19 resources containing at least financial PQIs and only 9 containing healthcare satisfaction PQIs.To align with the IHI quadruple aim framework, a specific set of balanced PQIs is needed for a maturation and evolution in home care system program quality and performance management from single indicators to an integrated and balanced scorecard approach that includes selected key system indicators from across all quadrants.Lastly, the application of PQIs depends on the healthcare system and local context/need as every country, province, state, and territory has a different system; precautions should be considered in application of PQIs in the context of different healthcare systems, organization, or population sociodemographic characteristics.PQIs should be selected balancinge the needs of all impacted groups, such as front line providers and clients being served, and with those of policy makers and healthcare system administrators.Selection processes should include involvement and guidance from all those involved.
Limitations exist to this scoping review.The search most likely is not exhaustive even though multiple databases were targeted using comprehensive search strategies with the assistance of academic librarian and academic information technology research resources.PQIs in the gray literature, those collected within operational programs, and published government reports are likely to have not all been identified.Overall, our search gathered the broad categories of important PQIs resulting in our estimation that most have been identified.Secondly, it is difficult to extract accurate PQI descriptions from all publications.Exact PQIs methods can be found to be difficult to obtain as sources did not disclose materials or methods used, and/or descriptions are general, unclear, and/or difficult to interpret.Third, sorting and coding by various types and categories, even with the assistance of predefined data abstraction tools and classification schemes, is partly subjective, especially with PQIs that cross multiple quadrants within the IHI Quadruple Aim framework. 16Lastly, the PQIs gathered were not assessed for appropriateness, feasibility, or impact by the reviewers in this study.This is an important next step as there is a need to build upon the limited number of system quality and performance home care indicators developed under scientific methods and published at the national and provincial levels.Currently for validated measures in Canada, we identified that CIHI publishes a limited number of PQIs specific to home care at the national level; and that within the province of Ontario, various PQIs such as access times, experience with care, and clinical outcome indicators are reported publicly; with neither organization utilizing a balanced scorecard structure such as the IHI quadruple AIM framework. 17,19,20

Conclusion and Next Steps
This systematic review has shown that many PQIs for evaluating home care service and program performance have been proposed, identified, and or applied in various means but that the distribution is skewed across different components of the     and, within current health system evaluation and performance management frameworks such as the IHI Quadruple Aim.Evaluating the potential adaption and implementation of evidence-based PQIs is essential to measuring and improving home care system programming including indicators reflective of acceptability and potential usefulness of measures by population groups as an important future step.

Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Area of
Of the 105 articles, 1,079 individual PQIs were identified and categorized based on the IHI Quadruple Aim quadrants.Of the total PQIs identified, 886 are Patient Outcome, 113 Patient Experience, 40 Financial, and 40 Provider Satisfaction indicators.Furthermore, each of the 1,079 indicators were individually assessed for duplication.Indicators that were exact in description or intent were labelled as duplicate indicators.Analysis and review using this method identified 250 duplicate indicators.We consolidated these to create a list of

Figure 2 .
Figure 2. Article publication date distribution graph with proportional sum curve.Note.The proportional sum curve portrays that the majority of the identified articles/reports are from more recent dates (2016-2022) within the search criteria.

Figure 3 .
Figure 3. World map-article publication by country with totals portrayed by dot size.Note.A total of 105 articles with at least 1 identified PQI were published from 22 different countries, with the United States of America and Canada comprising of the majority.
Performance Monitoring AcƟviƟes Home Care Office Clinical and OperaƟonal/Standards Performance

Table 2 .
Number of IHI Quadruple Aim Quadrants by the Number of Articles.

Table 3 .
17ticle Classifications by IHI Quadruple Aim Quadrant, and Population of Focus.Note.IHI quadrants are adapted from the Quadruple Aim framework, the populations of focus is the population group the article focused on specifically.17IHIQuadruple Aim, populations, areas of focus, and overall healthcare systems.These results should promote future research in 3 areas.First, systematic reviews are warranted to explore whether additional evidence-based PQIs can be advanced from the existing literature including areas focused on community-based care (primary care, Emergency Medical

Table 4 .
Number of Unique PQIs per Area of Focus and IHI Quadruple Aim Quadrant.Services (paramedic services), and public health).Second, further investigational research is needed to understand if there are knowledge gaps for the finance, and provider satisfaction areas of the Quadruple Aim with a focus on home care programs.Third, further research is needed on how home care program PQIs are currently used in health systems to measure and improve outcomes.Including research to better understand how to effectively incorporate newly developed indicators into performance-improvement initiatives;