Mapping Cognitive Trajectories and Detecting Early Dementia Using the Mini-Mental State Examination Cognitive Charts: Application to the French Three-City Cohort
Abstract
The Cognitive Quotient (QuoCo) classification algorithm monitoring decline on age- and education-adjusted Mini-Mental State Examination (MMSE)-derived cognitive charts has proved superior to the conventionally-used cut-off for identifying incident dementia; however, it remains to be tested in different settings. Data were drawn from the Three-City Cohort to 1) assess the screening accuracy of the QuoCo, and 2) compare its performance to that of serial MMSE tests applying different cut-offs. For the QuoCo, sensitivity was 74.2 (95% CI: 71.4–76.8) and specificity 84.1 (83.6–84.7) and for the MMSE < 24, 64.1 (61.1–67.0) and 94.8 (94.4–95.1), respectively; whereas overall accuracy and sensitivity was highest for MMSE cut-offs <25 and <26. User-friendly charts for mapping cognitive trajectories over visits with an alert for potentially ‘abnormal’ decline can be of practical use and encourage regular monitoring in primary care where the <24 cut-off is still widely used despite its poor sensitivity.
INTRODUCTION
It is now widely established that dementia and its early diagnosis is a major public health concern [1]. Yet studies suggest that over half of cases seen in primary care go unnoticed [2, 3]. A variety of tools are available to distinguish normal cognitive ageing from abnormal decline and an early degenerative process [4]. The Mini-Mental State Examination (MMSE) remains one of the most widely used tests [5], despite its well-known psychometric limitations (poor sensitivity to mild dementia, poor test-retest reliability, ceiling effects, etc.). Furthermore, the accuracy of cut-off scores has been found to vary with age and education, with for instance, lower normative values and cut-off scores for dementia screening found for older adults and in those with lower education [6–9]. The most widely used conventional cut-off is a score < 24 [10] but other cut-off values such as <25 have also shown to have high overall accuracy [5].
Cognitive charts have been developed to monitor change in cognitive functioning over time, considering age and education-specific performance on screening tools [11, 12]. As with pediatric growth charts, clinicians can map the cognitive functioning of patients over follow-up visits and be alerted by a decline that could indicate a possible pathological process. The MMSE Cognitive Charts method was developed and validated drawing data from two north American population studies [11], and the Cognitive Quotient (QuoCo) classification algorithm showed higher sensitivity than the conventional <24 MMSE cut-off for identifying patients in the early stages of dementia [11]. To date, little is known about the relevance of this method in other population settings. In a cohort of French older adults, we aimed to investigate the relevance and accuracy of this method for detecting incident dementia compared to the use of cut-off scores applied to serial MMSE tests.
METHODS
Study population
Data were drawn from the Three-City Study (E3C), a prospective cohort of community-dwelling persons aged 65 and above recruited in three French cities (Bordeaux, Dijon, Montpellier) [13]. A face-to-face interview and a clinical examination were performed at baseline and at the 2, 4, 7, 10, 12, and 14-year follow-up visits. The study protocol received ethics approval and all participants gave written informed consent. From the original 9,294 participants with at least one follow-up (n = 8,398), we excluded those with baseline dementia (n = 151), missing data for baseline MMSE or education (n = 51), and missing MMSE data at all follow-ups (n = 40). The analysis was performed on 8,156 participants. Participants excluded from the analysis were more likely to be male, older, with a lower level of education and with more physical health problems.
Definition of dementia
A three-step procedure was used to diagnose dementia cases at baseline and follow-up visits. First, screening was performed by trained psychologists based on a battery of neuropsychological tests including the Trail Making Test (Parts A and B), the Benton Visual Retention Test, Isaac’s Set Test and the original French version of the MMSE. Suspected cases of dementia based on their extensive neuropsychological examination were seen by a neurologist, with the exception of the Montpellier center where every participant had a neurological examination at each follow-up visit. Finally, all available information on suspected cases of dementia were analyzed by a common independent committee of neurologists and psychiatrists according to DSM-IV criteria, to obtain a consensus and validate the diagnosis and etiology [13].
The Cognitive Charts-MMSE
A linear model was constructed by Bernier et al. in the original development sample to represent the normal cognitive decline associated with age and education: Quoco = 786 - 5.77 × Standardised Age (SA) [11]. The cognitive quotient (QuoCo) corresponds to MMSE divided by age, while standardized age corresponds to age adjusted for years of education, . From this equation, parallel lines were obtained to describe the various percentiles of normal cognitive decline associated with age and education. The distance between these lines was chosen to optimize the discrimination between healthy participants and those with dementia. If an individual’s cognitive decline from an initial measurement was greater than this width (crossing an interval on the chart), it was considered ‘abnormal’. As a precaution, an age and education-adjusted ‘at-risk’ cut-off zone was determined. Participants falling into this zone, regardless of their decline, were considered to have ‘abnormal’ cognition. These two criteria applied to the charts are hereafter referred to as the QuoCo algorithm or method. A visual representation of the cognitive charts and a practical guide to their use in clinical settings can be found at https://quoco.org/. By entering a patient’s educational level and, at different examinations, MMSE score and age, cognitive performance can be tracked over time, with the QuoCo indicating whether the decline is to be considered ‘abnormal’ or not for the patient’s age and education.
In the original study, QuoCo values in healthy controls at baseline ran from approximately 140 to 460 and the method had a sensitivity (SN) of 80 and a specificity (SP) of 87, a positive likelihood ratio (LR+) of 6.2 and negative likelihood ratio (LR−) of 0.23 (see Appendix 4 in [11]).
Statistical analysis
Using the above equations, QuoCo values were calculated per SA 1) at baseline for health controls only to verify the model fit and 2) at all time-points for all participants to discriminate between non dementia and incident dementia based on Bernier et al.’s linear model and classification algorithm. The threshold values for crossing an interval or falling into the ‘at-risk’ zone to discriminate between ‘normal’ and ‘abnormal’ decline were provided by the authors [11]. Trajectories were examined by comparing QuoCo values at each follow-up with the baseline value using repeated logistic regression analysis. The analysis was also performed for serial MMSE scores dichotomized with cut-offs ranging from <23 to <27. Analyses were performed overall and per study center; and stratified, for the <24 cut-off [10], by age (<75/≥75), education (0–11/>12 years), and baseline cognitive performance (MMSE: <28/≥28), using the same dichotomization thresholds as in the original study to check for stability in sub-groups. SN, SP, and positive (PPV) and negative (NPV) predictive values were given with 95% confidence intervals (CI), considering correlations between observations from a single participant [14]. Youden indices were calculated [15]. Area under the curve (AUC), LR+and LR- for the MMSE were illustrated following Biggerstaff’s method (see Fig. 1 for detail) [16]. We used SAS EG 7.15 (SAS Institute, Inc. Cary, NC, USA).
RESULTS
Of the sample (61.3% women), there were 1043 (12.8%) cases of incident dementia over the median 9.2 (IQR: 7.5) year follow-up. Incident dementia cases ranged from 10.2% for Dijon, to 10.6% for Montpellier and 21.3% for Bordeaux. Cases were older, less educated, with a lower baseline MMSE score (Table 1). Baseline QuoCo values were plotted against SA to verify how well the data fitted the charts. The following equation was derived: QuoCo = 729 - 5.2 × SA, indicating that in healthy controls, approximately 5.2 QuoCo points were lost per yearly increase in SA.
| Healthy | Incident | |
|---|---|---|
| controls | dementia | |
| (N = 7,113) | (N = 1,043) | |
| Sample description | Median (IQR) | Median (IQR) |
| Age at start of study (y) | 72.8 (7.8) | 76.4 (7.4) |
| Follow-up duration (y) | 9.3 (7.5) | 8.8 (5.3) |
| Education: ≥12 years (%)1 | 30.4 | 25.3 |
| Centre (%) | ||
| Bordeaux | 20.2 | 37.4 |
| Dijon | 55.3 | 42.9 |
| Montpellier | 24.5 | 19.7 |
| MMSE | ||
| Score <24 at baseline (%) | 4.0 | 10.4 |
| Score at baseline | 28 (2) | 27 (3) |
| QuoCo | ||
| Score at baseline2 | 377.9 (51.7) | 348 (52) |
| In ‘at risk’ zone at | 7.8 | 16.0 |
| baseline (%) |
1Education was classified into the following 7 levels, each corresponding to an average number of years in school: no schooling (0), primary level (5), secondary level (9), apprenticeship (10), high school attendance (12), high school qualification (13), university level (14). The percentage represents the participants with 12 years of education or more. 2MMSE/age * 100.
For identifying incident dementia, the AUC for MMSE was 0.93 (95% CI: 0.92–0.94). The QuoCo algorithm yielded a higher SN but a lower SP than the MMSE < 24 cut-off, but performed overall less well than the <25 and <26 cut-offs, the latter corresponding to the highest Youden value (Table 2 and Fig. 1A). This was also the case in the three study centers, with cut-offs corresponding to the highest Youden values of <26 for Bordeaux and <25 for Montpellier and Dijon. In the Montpellier center, the QuoCo performed as well for SN as the <25 cut-off. Stratification in sub-groups showed greater overall stability for the QuoCo compared to the MMSE < 24 cut-off (Fig. 1B). With similar Youden indices, the MMSE < 24 cut-off showed superior SP whereas the QuoCo showed better SN, whatever the sub-group, except for MMSE < 28.
| Sensitivity | Specificity | Youden | PPV | NPV | LR+ | LR− | |
|---|---|---|---|---|---|---|---|
| (95% CI) | (95% CI) | index | |||||
| QuoCo | 74.1 (71.3–76.7) | 84.0 (83.5–84.6) | 0.58 | 14.2 (13.3–15.2) | 98.9 (98.8–99.0) | 4.63 | 0.31 |
| MMSE score < 23 | 51.7 (48.6–54.8) | 97.3 (97.0–97.5) | 0.49 | 40.5 (37.7–43.3) | 98.2 (98.1–98.4) | 19.15 | 0.50 |
| MMSE score < 24 | 64.1 (61.1–67.0) | 94.7 (94.3–95.0) | 0.59 | 30.2 (28.2–32.3) | 98.7 (98.5–98.8) | 12.09 | 0.38 |
| MMSE score < 25 | 75.9 (73.2–78.5) | 90.6 (90.1–91.1) | 0.67 | 22.5 (21.1–2.09) | 99.1 (98.9–99.2) | 8.07 | 0.27 |
| MMSE score < 26 | 85.3 (83.0–87.4) | 84.4 (83.7–85.0) | 0.70 | 16.3 (15.4–17.3) | 99.4 (99.3–99.5) | 5.46 | 0.17 |
| MMSE score < 27 | 93.5 (91.8–94.8) | 74.1 (73.3–74.8) | 0.68 | 11.4 (10.8–12.1) | 99.7 (99.6–99.8) | 3.61 | 0.09 |
CI, Confidence interval, calculated taking into account correlations between observations using generalized estimation equations [14].
A sensitivity analysis was performed excluding the participants falling into the QuoCo ‘at-risk’ zone at baseline (N = 7,694). Results were similar (SN: 73.4 (70.3–76.3), SP: 85.2 (84.7–85.8)). In this same sub-sample, we further restricted the QuoCo algorithm to crossing the interval zone between two curves regardless of the ‘at-risk’ zone. Again, results remained unchanged (SN: 71.9 (68.8–74.8), SP: 85.8 (85.2–86.3)).
DISCUSSION
Our findings support the applicability of the MMSE cognitive charts to other samples, namely healthier non-institutionalized older adults with a lower incidence of dementia (13.4% versus 18% in Bernier et al.’s development sample). In the healthy controls, the postulated relation between the QuoCo and SA was verified. The lower intercept (729 versus 786 in the development study) and lower decline (5.2 versus 5.8) could be partly explained by our choice not to exclude subjects with baseline cognitive impairment but no dementia in order to be as close as possible to ‘real life’ situations in primary care. Overall, the QuoCo algorithm performed less well in our study than in the development study, where values of 80 and 87 were reported for SN and SP, respectively. Again, our choice not to exclude subjects with baseline cognitive impairment but no dementia may contribute to this lower performance.
Our findings from a general population cohort of elderly subjects confirm that the MMSE is a valid scale for identifying early dementia with a high unadjusted AUC, considerably more so than in other studies [17, 18]. The QuoCo algorithm shows a slightly lower level of accuracy than the MMSE < 24 cut-off as measured by the Youden index. However, compared to the MMSE < 24, the QuoCo increased incident dementia identification by 10% (SN), despite a 10.7% reduction in ruling out dementia (SP). High sensitivity in detriment of specificity is of particular relevance for cognitive screening instruments in primary care as false positives will be further assessed and their status reconsidered whereas false negatives will be missed for early care. Findings were robust to the sensitivity analyses. Moreover, they were consistent with the original study where the QuoCo yielded a 10% increase in SN for a 8.9% decrease in SP [11]. Higher MMSE cut-off scores investigated in our study presented higher accuracy with both higher sensitivity and specificity for the <25 and <26 cut-offs compared to the MMSE < 24 and the QuoCo algorithm. The poor performance of the <24 cut-off does not fit with Creavin et al.’s meta-analysis where only a slightly lower sensitivity (0.85 versus 0.87) but higher specificity (0.90 versus 0.82) was found for the <24 versus the <25 cut-off [5].
Recent normative data reported for the French general population show important variations with education, with the 50th percentile values for the oldest age group studied (65–70 years) ranging from 26 for no diploma to 29 for college degree [7]. As in the original study, the QuoCo showed greater stability than the MMSE < 24 cut-off across age, education, and baseline cognitive status sub-groups, thereby overcoming the poorer performance of the MMSE previously shown with this cut-off in older adults [9], with a ceiling effect in those with high cognitive functioning [19]. The higher SN of the QuoCo in the group under 75-years of age and in that with 12 or more years of education is in keeping with Bernier et al.’s original study and a similar cognitive quotient algorithm based on the Montreal Cognitive Assessment [11, 12]. However, in a sensitivity analysis, the <25 and <26 cut-offs showed to be as stable as the QuoCo across the dichotomized sub-groups. To note, as we are comparing QuoCo-assessed decline over time to serial dichotomized MMSE scores regardless of previous MMSE tests, our findings do not allow us to single out the effect of age and education correction on the discriminative performance of the MMSE.
There is controversy over a potential loss of performance of cognitive screening tests when correcting for age and education, depending on whether they are considered as causal risk factors for dementia or only ‘noise’ influencing the raw scores. In some cases, correction has been shown remove meaningful information and lower sensitivity but in others, in the absence of a strong causal relationship, it can be beneficial [20]. In fact, age but not education has been found to reduce the validity of test scores predicting progression from mild cognitive impairment to dementia [21], but this has not been confirmed elsewhere [22]. Creavin et al. pooled accuracy estimates adjusted for level of education from 7 cross-sectional general population and primary care studies and found 0.97 for sensitivity and 0.70 for specificity [5]. In our longitudinal study, despite important differences in baseline characteristics such as age, education, general health, and dementia incidence between the centers [13], SN and SP for the QuoCo did not differ between them. However, optimal MMSE cut-offs did, indicating that raw cut-offs on the MMSE will perform differently according to the setting as previously reported [5]. Moreover, in all centers higher MMSE cut-offs than the conventional <24 yielded higher SNs and SPs. We believe that from a clinical perspective, the QuoCo method with its user-friendly visual online charts and classification algorithm requiring age, education and MMSE score, exempts the clinician from having to search and choose a cut-off or from using the conventional widely used but poorly performing <24 threshold as must be done in busy primary care settings.
Limitations inherent to most population cohorts include attrition, with participants who will go on to develop dementia being more likely to be censored than others. This limitation is partly balanced by a high accuracy of diagnosis through active screening and validation procedures. As in many studies [11, 18], the raw MMSE score was used in this diagnostic process. However, it was one among many screening instruments (MMSE, Isaacs Set Test, Benton Visual Retention Test, Trail Making Test) including also measures of dependency, in a stepped approach involving examination by a neurologist and a review of potential cases by an independent expert committee [13]. Another limitation was the lack of precision regarding education, documented by level rather than number of years.
Conclusion
In terms of accuracy, the QuoCo method showed poorer performance than the MMSE with cut-offs <24 and above. However, it showed higher sensitivity than the conventionally used MMSE < 24 cut-off for identifying incident dementia. This is, to the best of our knowledge, the first application of the QuoCo method in a non-North American population, in a large cohort of healthier community-dwelling older adults with more frequent examinations over a longer follow-up. Its stability across the 3 study centers and sub-groups adds to the validity and applicability of the method across different settings, periods, and follow-up procedures. Cost-effective, easy-to-use screening tools are essential for monitoring change and implementing effective dementia strategies in primary care [23]. This method has the advantage of combining one of the most widely used screening tools with a visual interactive representation of trajectories modelled on pediatric growth charts familiar to most clinicians; from age, level of education and MMSE score it will indicate whether the patient’s cognitive decline requires or not further investigation. This can have a considerable public health impact by enabling early referral for further investigations and planning ahead for care.
AUTHOR CONTRIBUTIONS
Joanna Norton (Formal analysis; Methodology; Writing – original draft) Laure-Anne Gutierrez, MSc (Data curation; Formal analysis; Methodology; Writing – review & editing); Christian Gourdeau (Conceptualization; Methodology; Writing – review & editing); Helene Amieva (Funding acquisition; Investigation; Validation; Writing – review & editing); Patrick Bernier (Conceptualization; Project administration; Validation); Claudine Berr (Conceptualization; Funding acquisition; Investigation; Methodology; Supervision; Validation; Writing – review & editing).
ACKNOWLEDGMENTS
We would like to thank all the participants of the E3C study.
CONFLICT OF INTEREST
The authors have no conflict of interest to report.
FUNDING
The 3C Study is carried out under a partnership agreement between the Institut National de la Santé et de la Recherche Medicale (INSERM), Victor-Segalen Bordeaux-II University, and Sanofi-Aventis. The Fondation pour la Recherche Medicale supported the preparation and initiation of the 3C study.
The study was also supported by the Caisse Nationale Maladie des Travailleurs Salariés, Direction Générale de la santé, MGEN, the Institut de la Longevité, Agence Nationale de la Recherche ANR PNRA 2006 (06-01-01) and Longvie 2007 (LVIE-003-01), Agence Française de Sécurité Sanitaire des Produits de Santé, the Regional Governments of Aquitaine, Bourgogne, and Languedoc-Roussillon, the Fondation de France, the Ministry of Research-INSERM Programme Cohorts and collection of biological material, Fondation Plan Alzheimer (FCS 2009–2012), the Caisse Nationale de Solidarité pour l’Autonomie (CNSA), Novartis.
The funders and sponsors had no role in the design and conduct of the study, collection, management, analysis, and interpretation of the data; and preparation, review, and approval of the manuscript.
Data availability statement
The datasets used and analyzed during the current study are available from the corresponding author on reasonable request.
REFERENCES
1. Gauthier S, Webster C, Servaes S, Morais JA, Rosa-Neto P(2022) World Alzheimer Report 2022: Life after diagnosis: Navigating treatment, care and support. Alzheimer’s Disease International, London, England.
2. Connolly A, Gaehl E, Martin H, Morris J, Purandare N (2011) Underdiagnosis of dementia in primary care: Variations in the observed prevalence and comparisons to the expected prevalence. Aging Ment Health 15, 978–984.
3. Creavin ST, Noel-Storr AH, Langdon RJ, Richard E, Creavin AL, Cullum S, Purdy S, Ben-Shlomo Y (2022) Clinical judgement by primary care physicians for the diagnosis of all-cause dementia or cognitive impairment in symptomatic people.CD. Cochrane Database Syst Rev 6, 012558.
4. Abd Razak MA, Ahmad NA, Chan YY, Mohamad Kasim N, Yusof M, Abdul Ghani MKA, Omar M, Abd Aziz FA, Jamaluddin R (2019) Validity of screening tools for dementia and mild cognitive impairment among the elderly in primary health care: A systematic review. Public Health 169, 84–92.
5. Creavin ST, Wisniewski S, Noel-Storr AH, Trevelyan CM, Hampton T, Rayment D, Thom VM, Nash KJ, Elhamoui H, Milligan R, Patel AS, Tsivos DV, Wing T, Phillips E, Kellman SM, Shackleton HL, Singleton GF, Neale BE, Watton ME, Cullum S (2016) Mini-Mental State Examination (MMSE) for the detection of dementia in clinically unevaluated people aged 65 and over in community and primary care populations. Cochrane Database Syst Rev 2016, CD011145.
6. Matthews F, Marioni R, Brayne C, Medical Research Council Cognitive F, Ageing S (2012) Examining the influence of gender, education, social class and birth cohort on MMSE tracking over time: A population-based prospective cohort study. BMC Geriatr 12, 45.
7. Ouvrard C, Berr C, Meillon C, Ribet C, Goldberg M, Zins M, Amieva H (2019) Norms for standard neuropsychological tests from the French CONSTANCES cohort. Eur J Neurol 26, 786–793.
8. Tombaugh TN, McIntyre NJ (1992) The mini-mental state examination: A comprehensive review. J Am Geriatr Soc 40, 922–935.
9. Grut M, Fratiglioni L, Viitanen M, Winblad B (1993) Accuracy of the Mini-Mental Status Examination as a screening test for dementia in a Swedish elderly population. Acta Neurol Scand 87, 312–317.
10. Mitchell AJ (2009) A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res 43, 411–431.
11. Bernier PJ, Gourdeau C, Carmichael PH, Beauchemin JP, Verreault R, Bouchard RW, Kroger E, Laforce R Jr. (2017) Validation and diagnostic accuracy of predictive curves for age-associated longitudinal cognitive decline in older adults. CMAJ 189, E1472–E1480.
12. Bernier PJ, Gourdeau C, Carmichael PH, Beauchemin JP, Voyer P, Hudon C, Laforce R Jr. (2023) It’s all about cognitive trajectory: Accuracy of the cognitive charts-MoCA in normal aging, MCI, and dementia. J Am Geriatr Soc 71, 214–220.
13. The 3C Study Group (2003) Vascular factors and risk of dementia: Design of the Three-City Study and baseline characteristics of the study population. Neuroepidemiology 22, 316–325.
14. Ying GS, Maguire MG, Glynn RJ, Rosner B (2020) Calculating sensitivity, specificity, and predictive values for correlated eye data. Invest Ophthalmol Vis Sci 61, 29.
15. Youden WJ (1950) Index for rating diagnostic tests. Cancer 3, 32–35.
16. Biggerstaff BJ (2000) Comparing diagnostic tests: A simple graphic using likelihood ratios. Stat Med 19, 649–663.
17. Hou XH, Feng L, Zhang C, Cao XP, Tan L, Yu JT (2019) Models for predicting risk of dementia: A systematic review. J Neurol Neurosurg Psychiatry 90, 373–379.
18. Mura T, Coley N, Amieva H, Berr C, Gabelle A, Ousset PJ, Vellas B, Andrieu S, GuidAge/DSA study group (2022) Cognitive decline as an outcome and marker of progression toward dementia, in early preventive trials. Alzheimers Dement 18, 676–687.
19. Proust-Lima C, Amieva H, Dartigues JF, Jacqmin-Gadda H (2007) Sensitivity of four psychometric tests to measure cognitive changes in brain aging-population-based studies. Am J Epidemiol 165, 344–350.
20. Piccininni M, Rohmann JL, Wechsung M, Logroscino G, Kurth T (2023) Should cognitive screening tests be corrected for age and education? Insights from a causal perspective. Am J Epidemiol 192, 93–101.
21. Hessler J, Tucha O, Forstl H, Mosch E, Bickel H (2014) Age-correction of test scores reduces the validity of mild cognitive impairment in predicting progression to dementia. PLoS One 9, e106284.
22. Quaranta D, Gainotti G, Vita MG, Lacidogna G, Scaricamazza E, Piccininni C, Marra C (2016) Are raw scores on memory tests better than age- and education- adjusted scores for predicting progression from amnesic mild cognitive impairment to Alzheimer disease? Curr Alzheimer Res 13, 1414–1420.
23. Costa A (2017) Charting age-associated cognitive decline. CMAJ 189, E1470–E1471.
Cite article
Cite article
Cite article
OR
Download to reference manager
If you have citation software installed, you can download article citation data to the citation manager of your choice
Information, rights and permissions
Information
Published In
Article first published online: February 19, 2024
Issue published: March 19, 2024
Keywords
Data availability statement
Data is available for this article. View more information
PubMed: 38393910
Authors
Metrics and citations
Metrics
Journals metrics
This article was published in Journal of Alzheimer’s Disease.
View All Journal MetricsArticle usage*
Total views and downloads: 43
*Article usage tracking started in December 2016
Articles citing this one
Receive email alerts when this article is cited
Web of Science: 0
Crossref: 0
There are no citing articles to show.
Figures and tables
Figures & Media
Tables
View Options
View options
PDF/EPUB
View PDF/EPUBAccess options
If you have access to journal content via a personal subscription, university, library, employer or society, select from the options below:
I am signed in as:
View my profileSign out
I can access personal subscriptions, purchases, paired institutional access and free tools such as favourite journals, email alerts and saved searches.
loading institutional access options
Alternatively, view purchase options below:
Access journal content via a DeepDyve subscription or find out more about this option.
