Evaluation of a mental health service reform program, the Pathways to Community Living Initiative, for people with severe mental illness and complex needs

Objective The Pathways to Community Living Initiative (PCLI) aims to reform mental health care for people with severe and persistent mental illness (SPMI) and complex needs. This study reports independent evaluation findings on transitions from hospital and practice change in mental health services. Methods Data for this mixed-methods evaluation were obtained from administrative collections and semi-structured interviews with PCLI program managers, teams and executive leads; aged care managers; and leaders in inpatient, community and older people’s mental health services. Results Between July 2015 and December 2020, 674 participants (67% of those eligible for the PCLI) were transitioned from hospital to community. Of those transitioned, 21 required subsequent long-stay admissions. The PCLI introduced resources, clearly defined processes, and state-wide networks to guide changes in practice which are becoming embedded in the operations and governance of mental health services across New South Wales. Conclusions Severe and persistent mental illness and complex needs can be managed in community settings with highly individualised planning and care, supported by specialised clinical teams in partnership with mental health, aged care and disability services. Evaluation findings highlight the importance of continued investment in rehabilitation psychiatry.


P
eople with severe and persistent mental illness (SPMI) and complex needs are at risk of long hospitalisation. 1,2These patients are a 'low volume, high need, high-cost group' 3 with little evidence-based guidance for treatment. 4Many have a primary diagnosis of psychosis.Their severe, treatment-resistant symptoms, combined with physical illness, disabilities, cognitive impairment, and/or substance misuse, affect all aspects of daily functioning.
Profound and lasting change in the treatment of people with SPMI cannot be achieved in a piecemeal fashion by services but needs to be driven and coordinated at the highest levels. 5As a major component of the New South Wales (NSW) mental health reform strategy, 5 the Pathways to Community Living Initiative (PCLI) aims to address challenges in psychiatric rehabilitation and overcome barriers for these patients.Design and implementation processes (Figure 1) were informed by consultation 6 and evidence. 7,8 independent evaluation of the PCLI 9 was conducted between 2017 and 2021.This article focuses on two evaluation questions: first, whether the PCLI succeeded in transitioning participants out of hospital, and second, whether the program influenced practice in mental health services to decrease the likelihood of future long hospital stays across the population of people with SPMI and complex needs.

Methods
The PCLI is funded and driven by the NSW Ministry of Health (the Ministry) in cooperation with Local Health Districts (LHDs).The program began in mid-2015 with recruitment of program managers in six LHDs where large, non-acute facilities were situated and later  expanded to all LHDs.Initially, the target population was people who had been inpatients for 365 days or longer; this was later expanded to those at risk of long stays.There are two PCLI cohorts, defined by functioning: adults with issues of ageing such as dementia or frailty (Stage One) and adults without issues of ageing (Stage Two).Five LHDs have Stage One and Stage Two PCLI teams, one LHD has a Stage One team only and at the remaining LHDs, Stage Two is the focus of PCLI activities.

Evaluation design
The mixed-methods evaluation was designed around an established framework 10 and had three components, which all received ethical approval.Two are reported here: patient outcomes and provider/system change.The third, patient experiences, is reported elsewhere in this issue. 11

Participants
The patient outcomes evaluation used administrative data for the entire PCLI cohort (Table 1), that is, all patients eligible for the program.The provider/system change evaluation included 167 semi-structured interviews, conducted over 5 years, with PCLI program managers, clinical teams and executive leads; aged care managers; and leaders in inpatient, community and older people's mental health services.The staff of mental health and aged care services who were interviewed for this study were carefully chosen (purposive sampling) because of their long exposure to, and deep knowledge of, the program and the patients who took part.

Materials
Patient outcomes were measured with routine tools 12 : Kessler Psychological Distress Scale (K10, 13 patient rating of global distress); Health of the Nation Outcome Scales (HoNOS, HoNOS65+ 14 ; severity of mental illness symptoms); Life Skills Profile (LSP-16, 15 quality of life and general functioning) and Resource Utilisation Groups -Activities of Daily Living (RUG-ADL, 16 functional independence in four domains: bed mobility, toileting, transfer and eating).Interview schedules were designed around the evaluation questions and focused on implementation (e.g.barriers and enablers to program delivery) and observed outcomes (e.g.changes in practice in mental health services, outcomes for patients who were transitioned to community living).

Procedure
De-identified administrative data were provided to the evaluation team from the Ministry.Data included patient demographics, inpatient stay characteristics, use of health services following transition and scores on the outcome measures before and after discharge.
Interviews took place with individuals or groups at 11 implementation sites and nine aged care facilities.Some individuals were interviewed on multiple occasions, providing a longitudinal perspective.Participants were provided with written information and gave written or verbal consent.

Analysis
The discharge date from the index staythe admission that ended in transition to communitywas identified for each participant and used as the reference point for subsequent health service use and outcome measures.For participants who had not yet transitioned, the index stay was the ongoing admission.Only valid, complete assessments were retained.Paired t-tests and Wilcoxon signed rank tests were used to examine differences between baseline and follow-up scores on the outcome measures for those who transitioned.
Interview transcripts were managed in NVivo 12 Plus.Three evaluators, each with extensive experience of qualitative analysis and lengthy exposure to the PCLI, worked collaboratively to code, index and analyse data.They used modified framework analysis 17 to achieve a holistic, descriptive overview and iterative categorisation 18 to move systematically from codes and themes to interpretation.

Transitions to community
Two-thirds of the PCLI cohort (674, 67%) were transitioned to the community by the end of December 2020 (Table 2).After transition there was no change in distress (K10) and mental health did not deteriorate significantly (HoNOS 'symptom', 'behaviour' and 'social' sub-scales and total scores; Table 3).General functioning and quality of life (LSP-16) improved on average for Stage One, but there was significant decline in HoNOS 65+ 'impairment' and in RUG-ADL scores for all domains except toileting.Stage Two showed deterioration in cognition and physical health (HoNOS 'impairment' sub-scale) and general functioning and quality of life (LSP-16).Nevertheless, clinicians and aged care managers reported that they had observed benefits of transition for many participants, including improved quality of life, functional independence, and social participation, and better mental and physical health.About 20% of Stage One and 33% of Stage Two participants were readmitted to mental health units.Most readmissions resulted in stays of 4 weeks or less (Table 4).A small proportion (7-8%) presented to emergency departments (ED).For Stage One, most presentations to ED were not mental health related.Three-quarters of Stage Two presentations were accounted for by one percent (9/777) of individuals.Most participants had at least one contact with mental health community services.Contacts occurred, on average, every 23 days for Stage One and every 8 days for Stage Two.

Practice change in mental health services
The PCLI introduced clearly defined processes to guide practice change.These included assessment of goals, capacities and support needs, careful matching with suitable community-based providers and capacity buildingtailored to each participantwith providers and community mental health services.Significant investment, including funding for aged care partnerships and strengthened community mental health teams, provided a foundation for implementing reform.The PCLI senior clinicians assisted with transition planning, built capacity within mental health care teams, and supported organisations providing aged care and disability care services to participants.PCLI peer workers supported patient engagement and were strong advocates within multidisciplinary teams.State-wide networks enabled common training for Interviewees pointed out that as more patients are transitioned out of hospital, there will be a greater need for services from community mental health teams, because the ongoing clinical care and behavioural support that some people require is not available via aged care services or disability services.Consequently, stakeholders believe that sustainability of the PCLI will depend on addressing resourcing issues in publicly funded community mental health care, maintaining clarity around roles and responsibilities, and continuing to build capacity among staff of aged care and disability services so they can better support people with SPMI and complex needs.

Discussion
During the evaluation period, two-thirds of people eligible for the PCLI were transitioned to community living.Despite their ongoing, severe mental illness and complex needs, the vast majority did not require subsequent long-stay hospital admissions.Most participants received clinical follow-up in the community, and mental health crises requiring ED attendance were rare.On average, there was no increase in distress and no exacerbation of mental health symptoms.Many became more sociable and better able to care for themselves and had improved mental and physical health and quality of life, according to the mental health clinicians and aged care managers who observed them.The outcomes data indicated that those with issues of ageing had, on average, significant increases in life skills but decreases in independent functioning, which may be explained by normal ageing processes.Declines for Stage Two in general functioning and quality of life were more difficult to interpret but may be due to sampling bias, as explained below.
The qualitative evaluation of provider/system change shed light on how outcomes were achieved.The PCLI challenged mental health services to avoid long stays and to manage risks associated with discharge.It equipped them to unpack the capacities and needs of people with SPMI and assemble appropriate supports.Successful transitions relied on individualised planning and tailored, integrated care, in partnership with community-based services.

Limitations
The quantitative findings rely on the accuracy of data from mental health services.Sometimes a continuous period of hospitalisation is reported as multiple stays, so the true duration remains unknown.Program guidelines indicate that PCLI participants should be followed up, and the outcome tools administered, every 6 months for up to 2 years after transition, however, the extent of follow-up was variable.This may have created a biased sample for the post-transition assessments, that is, participants requiring more intensive support may have been more likely to have a follow-up assessment, whereas those functioning relatively well in the community may have had missing data at follow-up.

Conclusions
Notwithstanding these limitations, the PCLI demonstrated that SPMI and complex needs can be managed in community settings when people receive tailored supports and services.Evaluation findings highlight the importance of ongoing investment in rehabilitation psychiatry in Australia.

Table 1 .
Profile of PCLI cohort.aThePCLI cohort was defined as people for whom entries existed in the PCLI database, that is, people who had been identified by LHD staff as having experienced, or at risk of experiencing, long hospital stays.The cut-off date for inclusion was 31 December 2020.b Diagnosis coding and DRG classification only occurs after the inpatient episode has concluded.Therefore, there was no diagnostic or DRG information available for ongoing or recently concluded stays.c Length of stay is reported as it is recorded in the Health Information Exchange.This does not include any previous stays.Length of stay for consumers who remained in hospital was calculated to 31 December 2020. a

Table 3 .
Health outcomes following transition.Note.Ã indicates significant at p < .05 and ÃÃ significant at p < .001.These p-values refer to the paired t-tests or Wilcoxon Signed Rank tests (non-parametric equivalent) used to examine differences between the baseline and the follow-up assessment.Sub-scale scores were standardised to represent percentage of maximum possible score to allow comparisons.Lower scores indicate less severe problems.HoNOS 65+ was used with people aged 65 years and over.RUG-ADL has a different scoring system and is not shown here; it was used only with Stage One.

Table 4 .
Health service use following transition.