jpad journal

AND option

OR option



V. Dauphinot1, A. Garnier-Crussard1, C. Moutet1, F. Delphin-Combe1, H.-M. Späth2, P. Krolak-Salmon1,3,4


1. Clinical and Research Memory Center of Lyon, Lyon Institute For Elderly, University hospital of Lyon, Lyon, France; 2. EA 4129 “Parcours Santé Systémique”, University Lyon 1, Lyon, France; 3. Clinical Research Centre CRC – VCF (Vieillissement – Cerveau – Fragilité), Hospital of Charpennes, University Hospital of Lyon, Lyon, France; 4. Neuroscience Research Centre of Lyon, Inserm 1048, CNRS 5292, Lyon, France

Corresponding Author: Dr Virginie Dauphinot, Hôpital des Charpennes, 27 rue Gabriel Péri, 69100 Villeurbanne, France, Tel: +33 (0) 472433114. Fax: +33 (0) 472432054, E-mail address:
J Prev Alz Dis 2021;
Published online April 10, 2021,



Background: Alzheimer’s disease and related diseases (ADRD) are a major cause of health-related cost increase.
Objectives: This study aimed to estimate the real medical direct costs of care of patients followed at a memory center, and to investigate potential associations between patients’ characteristics and costs.
Design: Cross-sectional analyses conducted on matched data between clinical data of a cohort of patients and the claims database of the French Primary Health Insurance Fund.
Setting: Memory center in France
Participants: Patients attending a memory center with subjective cognitive complaint
Measurements: Medical or nonmedical direct costs (transportation) reimbursed by the French health insurance during the one year after the first memory visit, and socio-demographic, clinical, cognitive, functional, and behavioral characteristics were analyzed.
Results: Among 2,746 patients (mean ± SD age 79.9 ± 8 years, 42.4% of patients with dementia), the total direct cost was on average € 9,885 per patient during the year after the first memory visit: € 7,897 for patients with subjective cognitive complaint, € 9,600 for patients with MCI, and € 11,505 for patients with dementia. A higher functional and cognitive impairment, greater behavioral disorders, and a higher caregiver burden were independently associated with a higher total direct cost. A one-point decrease in the Instrumental Activities of Daily Living score was associated with a € 1,211 cost increase. The cost was higher in patients with Parkinson’s disease, and Lewy body disease compared to patients with AD. Diabetes mellitus, anxiety disorders and number of drugs were also significantly associated with greater costs.
Conclusions: Higher real medical direct costs were independently associated with cognitive, functional, and behavioral impairment, diabetes mellitus, anxiety disorders, number of drugs, etiologies as well as caregiver burden in patients attending a memory center. The identification of factors associated to higher direct costs of care offers additional direct targets to evaluate how interventions conducted in patients with NCD impact direct costs of care.

Key words: Costs of care, dependence, cognitive status, economics, Alzheimer’s disease.



Alzheimer’s disease and related diseases (ADRD) are considered as the main cause of health-related cost increase in developed countries (1, 2). Dementia represented a total cost of € 105.2 billion in Europe in 2010 (3). In France, the prevalence of dementia has been estimated to 7.9% among people aged 65 and over, and it is expected to reach 9.6% by 2050 (4). To anticipate and optimize interventions targeting patients developing neurocognitive disorders (NCD)(5), it is of crucial importance to evaluate the cost of resources associated with the main characteristics of NCD. Cost-of-illness studies for ADRD appear essential to anticipate future resource needs, nevertheless they remain difficult to conduct as aging is related to various health conditions and comorbidities, and it is unclear whether comorbidities should be directly linked to ADRD (6, 7). In cost-of-illness studies, costs of care have been mainly estimated from self-reported resource utilization by patients and/or their informal caregivers (8-10). The use of real costs associated to patients’ care (estimated from claims data) offers an objective evaluation of costs related to patient care, independently of the possible recall bias that self-report might induce (11). Most previous studies have estimated the average cost per patient selected with a specific diagnosis (12-14), and costs of care in patients with ADRD were generally related to symptoms severity such as cognitive, functional and behavioral impairment (6, 12-23). These costs were presented with different amplitudes depending on the study population characteristics, the perspective of the study (payer, societal), the components of the costs (direct, indirect, informal), and the time of evaluation, which makes difficult any comparisons (12, 24, 25). Analyses of economic data of patients suffering from neurocognitive disorders with real-world data are scare in France while they are needed to evaluate the economic impact of interventions and for policy makers (26). The present study aimed at estimating the real medical direct costs of care of patients of a memory center at all stages of cognitive impairment during the one year after their first memory visit, and at investigating the potential associations between patient socio demographic and clinical characteristics and the average real medical and non-medical direct costs. Furthermore, these associations were assessed in the sub-group of patients with Alzheimer’s disease (AD).



Study design and setting

The present study was a cross-sectional analysis conducted on matched data between the MEMORA cohort including patient clinical data and the claims database of the French Primary Health Insurance Fund (PHIF). The protocol of the MEMORA cohort has been published previously (27). The match between the two databases was performed using the date of the 1st visit at a memory center. The claims database includes real medical and nonmedical direct costs of care for patients. Claims data were analyzed for one year after the first visit at a memory center. The present study was conducted at the University Clinical and Research Memory Centre of Lyon (University Hospital of Lyon, France), in collaboration with the regional PHIF of Rhône (Lyon, France). Around 90% of the French population is covered by the PHIF (28).

Study population

The study population included consecutive patients who underwent a medical examination in a memory center with a neurologist, geriatrician, or psychiatrist between 2014 and 2017. The inclusion criteria of patients were: to attend a memory visit, to have an evaluation of the functional autonomy level, not to live in a nursing home, and to be covered by the PHIF. Patients under legal protection were excluded from the study. The ethics Committee for the Protection of Persons Lyon Sud-Est IV was consulted on the 21st June 2013, and as the study was not classified as an interventional study, no written consent was required for participation. Written information regarding collection of individual data was provided to the patients and their informal caregivers and they were given the possibility to decline participation. Authorization for handling these data has been granted by the French Data Protection Authority (CNIL: Commission Nationale de l’Informatique et Libertés).

Real medical and nonmedical direct costs of cares

Source of cost data

This study was carried out from the perspective of the main payer of cares in France: the PHIF. The PHIF collects for each patient the real costs in Euros (€) of each care, act, and treatment that are reimbursed to patients. The costs included all the medical direct costs supported by the PHIF and one nonmedical direct cost (medical transportation). The others nonmedical direct costs such as home support and the indirect costs were not included since they are not covered by the PHIF.

Collected items of costs

The collected items of costs were grouped as medical direct costs ((1) outpatient cares, i.e. consultations and cares provided by general practitioners or specialists, surgical procedures in private practice, ophthalmological and hearing devices, dental cares, laboratory analyses, radiology examinations (radiology, scanners, MRI, PET, echography, bone densitometry), immunization, home dialysis, at-home hospitalizations, and health cures, (2) paramedical cares, i.e. nursing, physiotherapist, speech therapist, orthoptist, (3) pharmaceutical treatment in retail pharmacies, (4) public hospital stays, and (5) private hospital stays) and nonmedical direct costs (the medical transportations).

Valorization of the costs

The total cost per patient was estimated by adding all the costs of care, act, and treatment that occurred during the first year after the first memory center visit. The costs were presented as constant costs after adjustment using the value of Euro in 2017 as a reference, this value was available from the French national institute for statistical and economic studies (INSEE: Institut National de la Statistique et des Etudes Economiques) ( For each care, act, and treatment, the PHIF applies a specific reimbursement level, which is similar nationally.

Socio-demographic and clinical data at the memory center

Socio-demographic and clinical data were collected from the MEMORA cohort database, upon the first visit to the memory center (27). Socio-demographic data were: gender, age, marital status, and educational level.
Diagnosis etiologies and stages were determined by the specialist physician (neurologist, geriatrician, or psychiatrist) in charge of the patient (29-33). Patients with a cognitive complaint and normal neuropsychological performances were considered as having subjective cognitive complaint. A time to death variable was considered and calculated as the number of months between the first visit to the memory center and either the occurrence of the death or the last time the patient was known to be alive (corresponding to the end of the study period).
The following comorbidities information was collected: hypertension, hypercholesterolemia, diabetes mellitus, anxiety disorders and depressive disorders using medical report, as well as the number of drugs. The functional autonomy level was assessed during the interview with the primary caregiver with the Instrumental Activities of Daily Living (IADL) scale, including 8 activities (34). The IADL score ranges from 0 (dependent) to 8 (independent). Overall cognitive performances were assessed using the Mini Mental State Examination (MMSE), which ranges from 0 to 30 (35). The behavioral and psychological symptoms of dementia were assessed using the Neuropsychiatric Inventory (NPI) (36). A higher overall NPI score (maximum 144) is indicative of more severe behavioral disorders. The caregiver burden was assessed using the mini-Zarit scale, ranging from 0 to 7 (37). MMSE, IADL, NPI, and mini-Zarit scores were considered as continuous scores and as tertiles. MMSE scores were also considered using categories i.e. <10 (severe), 10-20 (moderate), >20 (mild).

Statistical analysis

The study population characteristics were described using the mean value ± standard deviation (SD) or the percentage, as appropriate. The costs per patient were expressed in euros and decomposed according to the origin of the costs, using means and their 95% confidence intervals (CI), medians, the 25th and 75th percentiles, and the minimum and maximum values. As the nonmedical direct costs of medical transport did not represent a large amount of the total cost, they were added, after description, to the medical direct costs to obtain a global cost per patient for further analyses.
As the distribution of the global cost was skewed, the relationship between each patient’s characteristics and the global cost was studied using generalized linear model (GLM) with log link and gamma distribution (38, 39). All significant variables associated with the total cost were then modeled together in multivariate GLM. Due to missing values for NPI and mini-Zarit scores, two different multivariate models were performed: the model 1 did not take into account the NPI and mini-Zarit scores, the model 2 did. The same multivariate models were performed within the sub-group of patients with AD. Results were summarized and presented as unadjusted and adjusted mean total cost, 95% CI, and p value. Additionally, the adjusted cost per 1 unit of IADL decrease was estimated and represented graphically.
A sensitivity analysis was conducted to examine the effect of potential “outliers” on the results. The “outliers” were identified as individual total cost greater than 3SD from the mean. The characteristics of the patients with outlier costs were compared to the characteristics of other patients, using Student t test or Pearson’s chi-square test. A new model was then performed after excluding the costs from the patients with outliers to verify whether the associations remained similar. P values below 0.05 were considered as statistically significant. Analyses were performed using STATA version 13 for Windows (StataCorp. 2013. Stata Statistical Software: Release 13. College Station, TX: StataCorp LP), and SPSS (Statistical Package for the Social Sciences) version 19.0 for Windows (SPSS Inc., Chicago, Illinois, USA).



Study population characteristics

The study population included 2,746 patients (Table 1). Overall, 62.8% of patients were female; the mean age was 79.9 ± 8 years. The mean MMSE score was 19.7 ± 7, the mean IADL score was 4.2 ± 2, and the mean NPI score was 19.8 ± 17. In terms of etiology, 27.6% of patients had probable AD with or without cerebrovascular component, 6.4% had vascular encephalopathy (with no AD), 1.5% had Lewy body disease, and 0.4% had fronto temporal dementia. For 53.5% of the patients, the diagnosis etiology could not be established at the 1st visit at the memory center.

Table 1. Characteristics of the study population

* Time to death was the number of months from the first visit to the memory center until either the occurrence of the death or the last time the patient was known to be alive; IADL: Instrumental Activities of Daily Living; MMSE: Mini-Mental State Examination; NPI: Neuropsychiatric Inventory


Real direct costs of care

The total direct cost of cares was on average € 9,885 [9,175; 10,594] per patient during the year after the first memory center visit (including € 9,647 [8,946; 10,350] for medical direct costs and € 238 [213; 263] for the medical transport considered as nonmedical cost (Table 2). Most of the direct medical cost was attributable to the hospital care in public hospital (€ 6,158 [5,492; 6,824]), representing 62% of the total direct cost, followed by the cost of paramedical care (€ 1,933 [1,816; 2,050]), to the pharmaceutical treatment in retail pharmacies (€ 725 [671; 779]), to the ambulatory medicine (€ 595 [553; 637]), and finally to the care in private hospital (€ 238 [177; 299]).

Table 2. Direct costs of cares according to the origin of the cost (in Euros)

* It includes consultations and cares provided by general practitioners or specialists, surgical procedures in private practice, ophthalmological and hearing devices, dental cares, biological analyses, radiology examinations (radiology, scanners, MRI, PET, echography, bone densitometry), vaccinations, home dialysis, at-home hospitalizations, and SPA treatments; † It includes nursing, physiotherapy, speech therapy, orthoptist.


Unadjusted associations between patient characteristics and the mean total cost of cares

The mean total cost was significantly higher with increasing age (p<0.0001), whereas no difference was found between genders (p=0.53; Table 3). The mean total cost was significantly higher when the educational level was lower (p<0.0001) e.g. € 10,782 [9,850; 11,714] for patients with primary educational level vs. € 6,938 [5,909; 7,967] for patients with tertiary educational level. The mean total cost was also significantly associated with the marital status (p<0.0001) and it was higher for patients with diabetes mellitus (€ 12,042 [10,580; 13,505], p<0.0001) or anxiety disorders (€ 12,557 [11,388; 13,725], p<0.0001) compared to those without (€ 9,491 [8,963; 10,018] and € 8,844 [8,314; 9,375] respectively). The mean total cost was higher when the number of drugs increased (p<0.0001). The mean total cost varied depending on the diagnosis etiology (p<0.0001): € 10,444 [9,444; 11,444] for patients with AD, while it was the highest for patients with Parkinson’s disease (€ 21,155 [11,443; 30,866]) and Lewy body disease (€ 20,433 [11,913; 28,951]). The mean total cost increased with the diagnosis severity (p<0.0001): the mean total cost was € 7,897 [7,142; 8,651] for patients with subjective cognitive complaint, € 9,600 for patients with MCI [8,714; 10,487], and € 11,505 [10,614; 12,396] for patients with dementia. The mean total cost was higher when the cognitive performance was lower (based on the MMSE score, p<0.0001); a one-point decrease of the MMSE score was associated with an increase cost of € 288 [170; 408]. The mean total cost was higher when the functional autonomy level was lower (based on the IADL score, p<0.0001), and a one-point decrease of the IADL score was associated with an increase cost of € 1,359 [1,069; 1,648]. The mean total was higher when the behavioral disturbances were higher (based on the NPI score, p=0.0001).

Table 3. Unadjusted relationships between patient’s characteristics and costs of care (in Euros) (n=2,746)

* Generalized linear model with log-normal link and gamma distribution; † Time to death was the number of months from the first visit to the memory center until either the occurrence of the death or the last time the patient was known to be alive; IADL: Instrumental Activities of Daily Living; MMSE: Mini-Mental State Examination; NPI: Neuropsychiatric Inventory


Adjusted associations between patient characteristics and the mean total cost of care

When all the variables that were significantly associated with the mean total cost in the unadjusted models were modeled together, all the variables still contributed significantly to the model, and the adjusted mean total cost was 13,057 [10,957; 15,168] per patient (Table 4). More precisely, the IADL score was negatively correlated to the mean cost, and a one-point decrease in the IADL score corresponded to an increase of the cost of € 1,211 [890; 1,532], after adjustment for age, educational level, diabetes mellitus, anxiety disorders, number of drugs, marital status, diagnosis etiology, time to death variable and MMSE score (Figure 1).

Figure 1. Adjusted means* of medical costs according to the IADL score

*Adjusted for age, educational level, diabetes mellitus, anxiety disorders, number of drugs, marital status, diagnosis etiology, MMSE score and time to death; IADL: Instrumental Activities of Daily Living

Table 4. Adjusted relationships between patient’s characteristics and costs of care (in Euros)

* Model 1: Generalized linear model (GLM) with log-normal link and gamma distribution including age, educational level, marital status, diabetes mellitus, anxiety disorders, number of drugs, etiology, time to death, IADL score and MMSE score tertiles; † Model 2: GLM with log-normal link and gamma distribution including the variables of the model 1, and NPI score and mini-Zarit score tertiles; IADL: Instrumental Activities of Daily Living; MMSE: Mini-Mental State Examination; NPI: Neuropsychiatric Inventory


In the sub-group of patients with AD (n=758), the adjusted mean total cost was 11,421 [9,982; 12,859]; a higher age (p=0.002), the presence of hypercholesterolemia (p=0.045), the presence of anxiety disorders (p<0.0001), a higher number of drugs (p<0.0001), and a lower IADL score (p<0.0001) were independently associated with higher costs of care (Supplement Table 1). The multivariate model found higher costs for patients with a MMSE score between 13 and 18 (€ 11,918 [9,687; 14,149]) compared to patients with lower and higher MMSE score: € 7,913 [6,425; 9,401] (MMSE≤13), and € 8,400 [7,165; 9,636] (MMSE>18). When including the NPI score and the mini-Zarit score tertiles in the model (Model 2), higher behavioral disorders (p=0.003) and higher caregiver burden (p<0.0001) were significantly associated with higher cost of cares. For patients with AD, the IADL score was negatively correlated to the mean cost, and a one-point decrease in the IADL score corresponded to a cost increase of € 1,096 [372; 1,820], after adjustment for age, number of drugs, time to death variable, anxiety, hypercholesterolemia, and MMSE score (Supplement Figure 1).

Sensitivity analysis

Among the 2,746 patients included in the present study, 88 were identified as having outlier costs (Supplementary Table 2). These patients were characterized by a slightly older age (81.9 ± 7.5 years vs. 79.9 ± 7.9 years, p=0.02), a higher number of drugs (13.4 ± 5.8 vs. 11 ± 5.5, p<0.0001), a worse functional impairment (IADL≤3 in 69.4% vs. 40.8% patients, p<0.0001), a lower MMSE score (MMSE≤17 in 52.3 vs. 33.1% patients, p<0.0001) compared to the group of patients without outlier costs. Among these patients, the proportion of dementia was higher (56.8% vs. 41.9% patients, p=0.02). Caregiver burden was higher in the group with outliers compared to the group without (Mini-Zarit>4: 43.1% vs. 29.4%). The multivariate model without outliers found similar associations between characteristics and total costs (Supplementary Table 3) as obtained with the complete set of patient data, excepted for the educational level and the diabetes mellitus for which the statistical significance was not reached anymore.



The present study provides an estimation of real medical and non-medical (transportation) direct costs of cares occurring during one year after the first memory center visit, for a large sample of outpatients at all stages of cognitive impairment, from the perspective of the main health insurance: annual medical direct cost of € 9,885 per patient varying from € 7,897 in patients cognitively normal but with subjective cognitive complaint, to € 9,600 in patients with MCI and € 11,505 in patients with dementia. The main part of direct costs of cares in our study was related to cares provided in public hospitals. Also, higher direct costs were independently associated with functional, cognitive and behavioral impairments, diabetes mellitus, anxiety disorders, higher number of drugs as well as with higher caregiver burden. The costs also varied across NCD etiologies, in particular they were higher in patients with Parkinson’s disease, and Lewy body disease compared to patients with AD. The associations between higher direct costs and functional, cognitive and behavioral impairments, anxiety disorders, number of drugs as well as with higher caregiver burden remained significant in the sub-group of patients with AD and in the sensitivity analyses restricted to individuals for whom the cost was not considered as outlier.
These results are consistent with the study of Leibson et al. conducted in a US population-based sample, showing that 70% of the direct costs of care was related to public hospitals cares, and which found an annual medical direct cost at $ 11,678 for patients with prevalent dementia, $ 9,431 for patients with newly discovered dementia, $ 6,784 for patients with MCI, and $ 6,042 for patients considered as cognitively normal (11). Besides, one can note that these results are surprisingly close in terms of level of costs, given than healthcare systems differ between countries.
The present study also confirmed and extended findings of others studies conducted in different contexts and from different economic perspectives, showing that the functional abilities was a main cost driver (2, 9, 20, 25, 40-42). In Zhu et al., a decrease of one-point in functional capacities measured with the Blessed Dementia Rating Scale (score out of 22) was associated with an increase of $ 1,406 in medical direct costs among community-dwelling patients with probable AD (15), whereas a decrease of one-point in functional capacities measured with the IADL scale (score out of 8) was associated with an increase of € 1,096 for one-point decrease of IADL in the sub-group of AD patients in the present study.
In the present study, direct medical costs were higher in patients with Lewy body disease or Parkinson’s disease compared to others NCD etiologies such as AD. While sample sizes in these sub-groups were limited, this observation is sustained by previous studies showing that Lewy body dementia was the costliest compared to others dementia’s etiologies (43), explained in part by cost of cares related to falls, urinary incontinence or infection, depression, anxiety, dehydration, and delirium. Another study also showed that Parkinson’s disease was associated with higher direct health care cost per patient compared to dementia without providing possible explanations (3).
Additional evidences of the association between costs and the MMSE were provided herein, in accordance with others studies (12, 25). However, in the study of Lindholm et al., the MMSE was not associated with the costs after adjustment for functional abilities (40). In the latter study, the characteristics of the population (community-dwelling population-based with a mean MMSE score of 26.6 ± 6) differed from the characteristics of the population studied here (patients of a memory center with a mean MMSE score of 19.7 ± 7), and the sample size was smaller, which may partially explain this discrepancy. Interestingly, in adjusted models, higher direct costs were found in patients with a MMSE score in the second tertile in the whole population study as well as in the AD sub-group (i.e. 13-18), whereas costs were lower in patients with lower or higher MMSE. In particular, patients with a MMSE score at 13-18 were found to have higher costs related to hospital stays for surgery and geriatric cares, higher costs for ambulatory cares in linked with visits to physicians, hearing device, radiology, and laboratory evaluations, and higher costs linked with medical transportation compared to patients with lower or higher MMSE score (detailed results not shown). A possible explanation for this finding is that patients at a more advanced stage of NCD may undergo less elective surgery due to higher risks of complications and higher mortality rates following surgical procedures in patients with dementia (44), and they might have lower health care consumption since the diagnosis has been previously made and less exploratory examinations are needed.
Similarly, higher direct costs were significantly associated with behavioral disorders in accordance with some studies (20, 21), but not all (23, 45). The use of reimbursement data from claims database in the present study instead of self-report use of care reinforces the objectivity of the analysis and strengthens the conclusion that behavioral disorders are associated with a significant increase of cost independently of other patient characteristics.
Also, higher costs were associated with higher caregiver burden, as observed in a previous study conducted from a societal point of view (23). Since the association remained significant after adjustment for others characteristics, we hypothesize that higher costs could directly contribute to the higher burden carried by the informal caregiver, independently of the patients’ impairments. This hypothesis is supported by a previous study showing an association between the financial stress and the higher caregiver burden (46). Even though the costs were estimated from the perspective of the national health insurance in the present study, the insurance may not cover the entire cost borne by patients and their caregivers. Nevertheless, further evidences are needed to confirm this hypothesis.
An original result of the present study was that among the comorbidities considered in the present study, diabetes mellitus and anxiety disorders were independently related to higher costs, while hypertension and hypercholesterolemia were not in the whole sample. In patients with AD, diabetes mellitus was not associated with costs, whereas a slight association was observed with hypercholesterolemia. The results are controversial in the literature concerning the link between comorbidities and costs in patients with NCD, e.g. Jutkowitz et al did not find significant link between comorbidities and costs (42), whereas Hill et al did (47).

Strengths and limits of the study

The present study included a large sample of outpatients attending a memory center with matched patient clinical data and costs. Real costs of care were estimated from claims database from the PHIF that is the main insurance in France and covers 90% of the French population (28), this limited the introduction of selection biases in the population other than the ones considered for the study. The study included patients at all stages of cognitive impairment, which allowed to conduct study between stage groups and to have a global overview of the medical direct costs (plus the medical transport) of patients visiting a memory center. According to a previous study based on the French National Alzheimer Database (48), 118,776 patients with AD attended a memory center in 2010, based on the results of the present study, the total direct cost covered by the PHIF for patients with AD one year after the first visit would be estimated to €370,337,356.
Limitations should nevertheless be considered when interpreting the results herein. This study had a cross-sectional design which does not allow causal relationships between the associations to be determined. The study did not include the societal perspective: this cost analysis did not include the indirect costs (e.g. indirect consequences of the disease such as lost work productivity or earning and informal costs), and the others nonmedical direct costs (e.g expenditures linked with the disease but not associated with medical services such as home services, nonmedical transport), except medical transportation, which led to an underestimation of the costs related to NCD, nevertheless it was specified that this study is not a cost-of-illness study. Previous studies have shown that a major part of the cost related to patients with cognitive impairment was supported by informal caregivers (informal care costs), especially when the patients were living at home, and costs estimation required specific surveys, often based on self-reported caregiving time allowed to patient and an extrapolation of the caregiver’s loss of earnings (21). In addition, the interpretation of these results should take into account the fact that a part of the direct medical costs can also be covered by private insurance that the patients can contract, the present study is from the point of view of the main French public health insurance. This study should also be interpreted in regards of the setting since all patients with cognitive impairment are not managed in memory centers and a part are followed by community practitioners, the care and then the costs may differ. Finally, mean costs should be interpreted with caution as the distribution of costs was skewed, nevertheless the statistical models took into account this data distribution.



This large study showed that functional and cognitive impairment, behavioral disorders, caregiver burden, diabetes mellitus, anxiety disorders, and the number of drugs were independently associated with higher direct cost of care for patients attending a memory center, from the payer perspective (French health insurance). The identification of these factors associated to higher direct costs of care offers additional direct targets to evaluate how interventions conducted in patients with NCD impact direct costs of care. Further researches are needed to broaden the economic perspective to the societal one and verify whether societal costs remain driven by the same factors.


Funding: The MEMORA study has received funding supports from MSD Avenir Fund, and Biogen Inc. These sponsors enabled the funding of nurses to carry out the research and questionnaires; they had no role in the design, conduct, collection, analysis and interpretation of the data, as well as in the preparation of the manuscript, its review and approval.

Acknowledgements: We thank Mrs. Pascale Gauthier-Robino and Mr. Laurent Colas from the Primary Health Insurance Fund of the Rhône (CPAM Rhône, France) for their collaboration during this research, Dr Michele Potasham for her advice, Mrs. Hélène Boyer (Direction de la Recherche Clinique et Innovation, Hospices Civils de Lyon, Lyon, France) for her help in the manuscript preparation and Mrs. Sarah Achi for her help in data management. We are grateful to the participants and the hospital staff.

Conflict of interest: The authors declare that they have no competing interests.

Study registration: Identifier: NCT02302482. Registered: 27th November 2014,

Ethical standards: Written information regarding collection of individual data was provided to the patients and their informal caregivers and they were given the possibility to decline participation. This research conducted in routine care was considered as non-interventional by the local ethics committee CPP Lyon Sud-Est IV (Comité de Protection des Personnes / committee for the protection of people). Authorization for handling these data has been granted by the French Data Protection Authority (CNIL: Commission Nationale de l’Informatique et Libertés).

Supplementary Material


1. Wimo A, Jonsson L, Bond J, Prince M, Winblad B. The worldwide economic impact of dementia 2010. Alzheimers Dement. 2013;9(1):1-11.
2. Taylor DHJ, Schenkman M, Zhou J, Sloan FA. The relative effect of Alzheimer’s disease and related dementias, disability, and comorbidities on cost of care for elderly persons. J Gerontol B Psychol Sci Soc Sci. 2003;56(5):S285-96.
3. Olesen J, Gustavsson A, Svensson M, Wittchen HU, Jönsson B. The economic cost of brain disorders in Europe. Eur J Neurol. 2012;19:155-62.
4. Mura T, Dartigues JF, Berr C. How many dementia cases in France and Europe? Alternative projections and scenarios 2010-2050. Eur J Neurol. 2010;17(2):252-9.
5. McRae I, Zheng L, Bourke S, Cherbuin N, Anstey KJ. Cost-Effectiveness of Dementia Prevention Interventions. J Prev Alzheimers Dis. 2021;8(2):210-7.
6. Jönsson L, Eriksdotter Jönhagen M, Kilander L, et al. Determinants of costs of care for patients with Alzheimer’s disease. Int J Geriatr Psychiatry. 2006;21(5):449-59.
7. Costa N, Derumeaux H, Rapp T, et al. Methodological considerations in cost of illness studies on Alzheimer disease. Health Econ Rev. 2012;2(1).
8. Wimo A, Gustavsson A, Jönsson L, Winblad B, Hsu MA, Gannon B. Application of Resource Utilization in Dementia (RUD) instrument in a global setting. Alzheimer Dement. 2013;9(4):429-35.
9. Rapp T, Andrieu S, Molinier L, et al. Exploring the relationship between Alzheimer’s disease severity and longitudinal costs. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2012;15(3):412-9.
10. Tolppanen AM, Taipale H, Purmonen T, Koponen M, Soininen H, Hartikainen S. Hospital admissions, outpatient visits and healthcare costs of community-dwellers with Alzheimer’s disease. Alzheimer Dement. 2015;11(8):955-63.
11. Leibson C, Long KH, Ransom JE, et al. Direct medical costs and source of cost differences across the spectrum of cognitive decline: A population-based study. Alzheimer Dement. 2015;11:917-32.
12. Schaller S, Mauskopf J, Kriza C, Wahlster P, Kolominsky-Rabas PL. The main cost drivers in dementia: a systematic review. Int J Geriatr Psychiatry. 2015;30:111-29.
13. Sicras A, Rejas J, Arco S, et al. Prevalence, resource utilization and costs of vascular dementia compared to Alzheimer’s dementia in a population setting. Dement Geriatr Cogn Disord. 2005;19(5-6):305-15.
14. Murman DL, Von Eye A, Sherwood PR, Liand J, Colenda CC. Evaluated need, costs of care, and payer perspective in degenerative dementia patients cared for in the United States. Alzheimer Dis Assoc Disord. 2007;21:39-48.
15. Zhu CW, Leibman C, McLaughlin T, Scarmeas N, Albert M, Brandt J. The Effects of Patient Function and Dependence on Costs of Care in Alzheimer’s Disease. J Am Geriatr Soc. 2008;56:1497-503.
16. Wimo A, Reed C, Dodel R, et al. The GERAS Study: A Prospective Observational Study of Costs and Resource Use in Community Dwellers with Alzheimer’s Disease in Three European Countries – Study Design and Baseline Findings. J Alzheimer Dis. 2013;36(2):385-99.
17. Darbà J, Kaskens L, Lacey L. Relationship between global severity of patients with Alzheimer’s disease and costs of care in Spain; results from the co-dependence study in Spain. Eur J Health Econ. 2015;16(8):895-905.
18. Rigaud AS, Fagnani F, Bayle C, Latour F, Traykov L, Forette F. Patients with Alzheimer’s disease living at home in France: costs and consequences of the disease. J Geriatr Psychiatry Neurol. 2003;16(3):140-5.
19. Mauskopf J, Racketa J, Sherrill E. Alzheimer’s disease: The strength of association of costs with different measures of disease severity. J Nutr Health Aging. 2010;14(8):655-63.
20. Gustavsson A, Brinck P, Bergvall N, et al. Predictors of costs of care in Alzheimer’s disease: a multinational sample of 1222 patients. Alzheimer Dement. 2011;7(3):318-27.
21. Dodel R, Belger M, Reed C, et al. Determinants of societal costs in Alzheimer’s disease: GERAS study baseline results. Alzheimer Dement. 2015;11(8):933-45.
22. Mesterton J, Wimo A, By A, Langworth S, Winblad B, Jönsson L. Cross-sectional observational study on the societa costs in Alzheimer’s disease. Curr Alzheimer Res. 2010;7(4):358-67.
23. Handels RL, Wolf CA, Aalten P, Verhey FR, Severens JL. Determinants of care costs of patients with dementia or cognitive impairment. Alzheimer Dis Assoc Disord. 2013;27(1):30-6.
24. Jönsson L, Wimo A. The cost of dementia in Europe: a review of the evidence, and methodological considerations. Pharmacoeconomics. 2009;27(5):391-403.
25. Gustavsson A, Jonsson L, Rapp T, et al. Differences in resource use and costs of dementia care between European countries: baseline data from the ICTUS study. J Nutr Health Aging. 2010;14(8):648-54.
26. Garrison LJ, Neumann PJ, Erickson P, Marshall D, Mullins CD. Using real-world data for coverage and payment decisions: the ISPOR Real-World Data Task Force report. Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research. 2007;10(5):326-35.
27. Dauphinot V, Moutet C, Rouch I, et al. A multicenter cohort study to investigate the factors associated with functional autonomy change in patients with cognitive complaint or neurocognitive disorders: the MEMORA study protocol. BMC Geriatr. 2019;19(1):191.
28. Tuppin P, de Roquefeuil L, Weill A, Ricordeau P, Merlière Y. French national health insurance information system and the permanent beneficiaries sample. Rev Epidemiol Sante Publique. 2010;58(4):286-90.
29. Mckeith I, Dickson D, Lowe J, et al. Diagnosis and management of dementia with Lewy bodies: third report of the DLB Consortium. Neurology. 2005;65(12):1863-72.
30. Roman GC, Tatemichi TK, Erkinjuntti T, et al. Vascular dementia: diagnostic criteria for research studies. Report of the NINDS-AIREN International Workshop. Neurology. 1993;43(2):250-60.
31. Rascovsky K, Hodges JR, Knopman D, et al. Sensitivity of revised diagnostic criteria for the behavioural variant of frontotemporal dementia. Brain. 2011;134(Pt 9):2456-77.
32. McKhann G, Knopman D, Chertkow H, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association worgroups on diagnosis guidelines for Alzheimer’s disease. Alzheimer Dement. 2011;7(3):263-9.
33. American Psychiatric Association. Diagnostic and statistical manual of mental disorders (5th ed.). Washington, DC: Author. 2013.
34. Lawton M, Brody E. Assessment of older people: self-maintaining and instrumental activities of daily living. Gerontologist. 1969;9(3):179-86.
35. Folstein M, Folstein S. Mini-mental state: A practical method for grading the cognitive stade of patients for the clinician. J Psychiatr Res. 1975;12(3):189-98.
36. Mckeith I, Cummings J. Behavioural changes and psychological symptoms in dementia disorders. Lancet Neurol. 2005;4(11):735-42.
37. Zarit SH, Todd PA, Zarit JM. Subjective Burden of Husbands and Wives as Caregivers: A Longitudinal Study. The Gerontologist. 1986 1986-06-01;26:260-6.
38. Diehr P, Yanez D, Ash A, Hornbrook M, Lin D. Methods for analyzing health care utilization and costs. Annu Rev Public Health. 1999;20:125-44.
39. Mihaylova B, Briggs A, O’Hagan A, Thompson SG. Review of statistical methods for analysing healthcare resources and costs Health Econ. 2011;20(8):897-16.
40. Lindholm C, Gustavsson A, Jönsson L, Wimo A. Costs explainted by function rather than diagnosis – results from the SNAC Nordanstig elderly cohort in Sweden. Int J Geriatr Psychiatry. 2013;28:454-62.
41. Akerborg O, Lang A, Wimo A, et al. Cost of dementia and its correlation with dependence. J Aging Health. 2016.
42. Jutkowitz E, Kane R, Dowd B, Gaugler J, RF. M, Kuntz K. Effects of Cognition, Function, and Behavioral and Psychological Symptoms on Medicare Expenditures and Health Care Utilization for Persons With Dementia. J Gerontol A Biol Sci Med Sci. 2017;72(6):818-24.
43. Chen Y, Wilson L, Kornak J, et al. The costs of dementia subtypes to California Medicare fee-for-service, 2015 Alzheimer Dement. 2019;15(7):899-906.
44. Kassahun WT. The effects of pre-existing dementia on surgical outcomes in emergent and nonemergent general surgical procedures: assessing differences in surgical risk with dementia BMC Geriatr. 2018;18(1):153.
45. Reese JP, Hessman P, Seeberg G, et al. Cost and care of patients with Alzheimer’s disease: clinical predictors in German health care settings. J Alzheimer Dis. 2011;27(4):723-36.
46. Adelman RD, Tmanova LL, Delgado D, Dion S, Lachs MS. Caregiver burden: a clinical review. JAMA. 2014;311(10):1052-60.
47. Hill JW, Futterman R, Duttaqupta S, Mastey V, Lloyd JR, Fillit H. Alzheimer’s disease and related dementias increase costs of comorbidities in managed Medicare Neurology. 2002;58(1):62-70.
48. Le Duff F, Develay AE, Quetel J, et al. The 2008-2012 French Alzheimer Plan: Description of the National Alzheimer Information System. J Alzheimer Dis. 2012;29(4):891-902.


J. Cartailler1,2,*, C. Loyer3,*, E. Vanderlynden3, R. Nizard4, C. Rabuel3, L. Coblentz Baumann3,5, C. Hourregue6, J. Dumurgier6, C. Paquet6

1. Department of Anesthesiology and Intensive Care, Lariboisière – Saint Louis Hospitals, Paris, France; 2. Paris Diderot University, Paris, France, Inserm, UMRS-942, France; 3. Département de médecine générale, Université de Paris, France; 4. Département de chirurgie orthopédique et traumatologique, APHP, Hôpital Lariboisière-Fernand Widal, Université de Paris, France; 5. Patient-Centered Outcomes Research Unit, UMR 1123, Université Paris-Diderot & Inserm, France; 6. Cognitive Neurology Center, Saint-Louis Lariboisière-Fernand Widal Hospital, APHP, Université de Paris INSERU1144, France; * These authors contributed equally

Corresponding Author: C. Paquet, Cognitive Neurology Center, Saint-Louis Lariboisière-Fernand Widal Hospital, APHP, Université de Paris INSERU1144, France,



Background: Surgery and anesthesia can result in temporary or permanent deterioration of the cognitive functions, for which causes remain unclear.
Objectives: In this pilot study, we analyzed the determinants of cognitive decline following a non-emergency elective prosthesis implantation surgery for hip or knee.
Design: Prospective single-center study investigating psychomotor response time and changes in MoCA scores between the day before (D-1) and 2 days after (D+2) following surgery at the Lariboisière Hospital (Paris, France).
Participants: 60 patients (71.9±7.1-year-old, 72% women) were included.
Measurements: Collected data consisted in sociodemographic data, treatments, comorbidities and the type of anesthesia (local, general or both). Furthermore, we evaluated pain and well-being before as well as after the surgery using point scales.
Results: Post-operative (D+2) MoCA scores were significantly lower than pre-operative ones (D-1) with a median difference of 2 pts [IQR]=4pts, (p<0.001), we found no significant difference between locoregional and general anesthesia. Pre-operative benzodiazepine or anticholinergic treatments were also associated to a drop in MoCA scores (p=0.006). Finally, the use of ketamine during anesthesia (p=0.043) and the well-being (p=0.006) evaluated before intervention, were both linked to a reduced cognitive impact. Conclusion: In this pilot study, we observed a post-operative short-term cognitive decline following a lower limb surgery. We also identified pre and perioperative independent factors linked to cognitive decline following surgery. In a next stage, a larger cohort should be used to confirm the impact of these factors on cognitive decline.

Key word: Cognitive decline, lower limb surgery, benzodiazepine, ketamine, well-being.

Abbreviations: AC: Anticholinergic; BDZ: Benzodiazepine; MoCA: Montreal Cognitive Assessment; POCD: Post-Operative Cognitive Decline.



Post-Operative Cognitive Decline (POCD) is a major cause of mortality and morbidity costing over $150 billion dollars yearly in health care expenses in the United States (1). POCD includes post-operative delirium, NeuroCognitive Disorder (NCD) and delayed NeuroCognitive Recovery (2). NCD definition from the DSM-V consists in a significant cognitive decline from a previous level of performance, diagnosed at least 30 days after the surgery and assessed by standardized neuropsychological testing. Delayed neurocognitive recovery is tested using the same criteria as for NCD except that the diagnosis window must be less than 30 days after intervention (2). In the elderly (>60 years old (yo)) POCD can result in a loss of autonomy (3, 4). However, clear negative consequences of anesthesia on POCD are still debatable (5, 6). Some papers speculate that POCD in the elderly is attributable to the per-operative use of benzodiazepines (BDZ) and anticholinergic (AC) drugs (7, 8), while others suggest that POCD result from a pre-existing cerebral fragility (4, 6). Taking into account population ageing which drives a growing demand for surgery, the identification of factors increasing the risk of POCD or more severe cognitive dysfunctions is needed.
Diagnosing NCD and delayed neurocognitive recovery requires the collection of preoperative cognitive status along with additional stages into the routine care. Indeed, a minimal setup depends upon establishing a baseline of cognitive functions prior the surgery, then on the administration of at least a second neuropsychometric test in the post-operative period (9). Tools for a quick assessment of POCD are for example the Confusion Assessment Method (CAM), Mini Mental State (MMS) or the Montreal Cognitive Assessment (MoCA) (3, 10).
In particular, the MoCA test evaluates several cognitive functions which impairment is a common mechanism shared by most of POCD, including memory, concentration and verbal abstraction among others. Identifying POCD risk factors (drugs, comorbidity, surgical or anesthesia setups) might help targeting patients that should be followed-up, leading to a personalized care. This prospective pilot study aims to investigate the cognitive decline following a non-emergency elective hip or knee replacement, and to identify independent risk factors associated with it. Additionally, we search for factors linked with persistent cognitive decline evaluated six weeks after the surgery.



Study population

Between February and September 2017, patients from the orthopedic department of Lariboisière Hospital, programmed for a non-emergency hip or knee replacement and who provided consent, were included in this monocentric, prospective study based on the daily clinical practice. We did not include non-French-speaking patients and those refusing to participate to the study. Based on the routine clinical practice, all patients underwent a standardized clinical examination, including medical history and physical examination, laboratory tests were performed in all subjects including chemistry panel and complete blood count. Patients underwent either a general anesthesia (GA), a locoregional anesthesia (LRA) or LRA+ propofol (LRA+P).

Cognitive assessment

To evaluate the cognitive diminution, we administered the MoCA one day before (D-1), then two days (D+2)and six weeks (W+6) after the surgery. The same practitioner administered MoCA tests. Additionally, the same patient never performed twice the same version of the test.


To identify determinants of post-operative decline, we collected during the pre-, per-, and postoperative periods the following covariates:
Preoperative period: Age; Tobacco; Alcohol; Diabetes; Cholesterol; Hypertension (HT); Thyroids; Feeling of defective memory; Previous Neoplasia; Previous GA; Previous LRA; Cognitive Complaints; Anti-hypertensive treatment; Anti-diabetic treatment (per os); AC drugs; Pre-op. Antalgic; Pre-op. NSAID; BDZ; Antidepressants; MoCA (D-1) score; MoCA (D-1) evaluation duration (min); Self-evaluation of pain (D-1) based on a point scale ranging from 0 (no pain) to 10 (extreme pain); Well-being score evaluated using a custom point scale ranging from 0 (no happiness) to 10 (extreme happiness); Instrument Activity Daily Living (iADL) based on the Lawton scale.
Peroperative period (surgery and anesthesia): Type of anesthesia as described above; Ketamine; Sufentanyl; Corticoids; Droperidol, Ephedrine; Atropine; Tranexamic acid; Clonidine chlorhydrate; Surgery duration; Occurrence of complications (see Appendix).
Post-operative period: Antalgic; NSAID; Nefopam chlorhydrate; Pregabalin; AC; BDZ; Peri-operative complications; MoCA (D+2, W+6) score; MoCA (D+2, W+6) evaluation duration (min); Well-being scale; Self-evaluation of pain (D+2, W+6); Well-being (D+2) score.

Statistical analysis

Numerical variables were expressed by the median and interquartile range or mean and standard deviation, categorical variables were expressed as the count and percentage. For all statistical test, we chose a significant level alpha=0.05. Patient characteristic data, duration of surgery, and MoCA scores were compared using chi2, Student’s t-test or Wilcoxon rank-sum test as appropriate. When t-test was used, the distribution normality was assessed with the two-tailed Shapiro-Wilk and Lilliefors tests. Sensitivity of variable of interest was computed from confusion matrices.
Comparison of MoCA (D-1) scores distribution and medians between GA, LRA, and LRA+P categories were analyzed with Kolmogorov-Smirnov and Mann-Whitney tests, both two-tailed.
For binary analysis, we associated for each patient the dummy variable 1 for the loss of at least one MoCA point between (D-1) and (D+2), 0 otherwise. To identify factors associated to a post-operative decline, we first proceeded with a univariate analysis (Chi2 for dummy variables and Logistic Regression (LR) for non-binary variable). We collected associations conditioned to have at least 5 occurrences in the entire population. Time duration of MoCA administration were compared using a two tailed paired t-test, after log-transform.
To investigate the impact of AC and BDZ treatment, we constructed two dummies variables for ‘AC’ AND ‘BDZ’ and for ‘AC’ OR ‘BDZ’ treatments. We searched for an association between these variables and cognitive decline (ΔMoCA≥1) between D-1 and D+2.
We used a multivariate class weight corrected logistic regression model to produce a risk model for loss of at least one MoCA point between D-1 and D+2. Variables with a p-value ≤0.05 in the univariate analysis were introduced in the multivariate model. In the case of incomplete data, patients were excluded from the multivariate analysis. We performed statistical analysis using R-studio software.



Sixty patients (71.9±7.1 yr., 72% women) above 60 year-old were prospectively included in this study (Fig. 1, Tab. 1). Among them, 27 (resp. 33) underwent a non-emergency elective prosthesis hip (resp. knee) implantation surgery. The distributions of age, time of surgery, MoCA, pain and well-being scores were not different between patient from the hip and the knee groups (Kolmogorov-Smirnov test, unsignificant difference, data not shown). Therefore, for the statistical analysis we decided to consider patients that had a hip and knee surgery as a single study group.

Table 1. Comparison of population characteristic data between the patient with ΔMoCA≥ 1pt and patients with no point loss. Binary data are number and percentage, while non binary data are shown in median and interquartile range (IQR). NSAIDs ; Nonsteroidal anti-inflammatory drugs, † includes the following complications: failed LRA – modification of the surgery – Hemorrhage – incisors defect (see Supplementary)

Figure 1. Flowchart


Patients underwent GA, (n=27) or LRA (n=14) or LRA+ P (n = 13). There was no significant difference, neither in mean nor in variance, on the ΔMoCA between D-1 and D+2 between the 3 groups of anesthesia (Fig. 2). We investigated if MoCA scores obtained one day before (D-1), two days after (D+2) and six weeks (W+6) following the surgery were significantly different. MoCA score drop was 2[4] points (median[IQR], p<0.001) between (D-1) and (D+2) (Fig. 3A) including 8, 14 and 19 patients who lost 1, 2 or 3 or more points respectively. We found no significant differences between D-1 and W+6 (MoCA median[IQR]: 24[3.5] versus 24 [4], p-value= 0.831), although among the 31 patients evaluated at 6 weeks, 32% did not recovered their baseline level score. We observed a significant decrease in the evaluation duration of the MOCA between (D-1) and (D+2) (p <0.001) and between (D+2) and (W+6) (p<0.001). The surgery duration did not affect the cognitive functions (median[IQR] = 70[40] min, ranging: 30 to 200min).

Figure 2. Distribution of ΔMoCA between D-1 and D+2 for the three types of anesthesia (GA, LRA, LRA+Propofol) showing no significant difference in (Wilcoxon test)


In the univariate analysis, we found a significant positive relationship between ΔMoCA≥1 and BDZ treatment (p-value = 0.047, sensitivity = 93.8%), AC drugs (p-value = 0.023, sensitivity = 100%), ‘BDZ or AC’ (p-value = 0.006, sensitivity = 93.8%), perioperative complications (p-value = 0.035, sensitivity = 100%) and a negative significant result with the use of ketamine agent (p-value = 0.043, sensitivity = 62.5%). Results are summarized in Tab. 1. Furthermore, among non-binary variables, only well-being (p-value = 0.006, sensitivity = 94.6%) was significantly associated with cognitive decline. Using a multivariate analysis, we found that ‘well-being’, ‘Ketamine’, ‘AC’ or ‘BDZ’, were independent risk factors associated with cognitive decline between D-1 and D+2 (see Tab. 2A).
Similarly, we found that, persistent cognitive decline was associated with BDZ (p-value = <0.001) and perioperative complications (p-value = 0.013). Additionally, we also found that post-operative confusion was also associated with a W+6 cognitive decline (p-value = 0.002). Results are shown in Tab. 2B.

Table 2. Factors associated with cognitive decline at D+2 and w+6



In this study, MoCA scores were significantly impacted by the surgery, with a loss of 2.25 points in average. We described that POCD were independently associated with pre-operative well-being, per-operative use of ketamine, peri-operative complications and pre-existing treatment based on either BDZ or AC drugs. In particular, well-being and ketamine had a protective effect, while complications and BDZ/AC treatments were linked to cognitive decay. Here, the type of anesthesia was not linked to cognitive decline while persistent cognitive decline was linked to pre-existing BDZ medication and per-operative complications. Finally, MoCA administration duration was neither affected by the type of anesthesia nor linked to POCD, however, the duration at W+6 was significantly shorter than D-1 and D+2 reflecting the training effect (Fig. 3B). These results suggest insight to improve perioperative cognitive protection and in particular to identify upstream of the intervention patients that could be affected by surgery or anesthesia.
Ketamine is a non-barbiturate anesthetic inhibiting acetylcholine that could lead to an increased number of delirium (11). Per-operative use of ketamine is still debated, with on one side studies defending its neuroprotective and anti-inflammatory actions (12), and on the other side studies arguing for no beneficial effects (13). Interestingly ketamine was shown to speed up recovery to consciousness after GA, yet possible impact on POCD was unclear (14). In the present study, per-operative ketamine administration was associated to fewer POCD, this support previous findings suggesting that administration of ketamine during a surgery reduces occurrences of POCD.
Chaiwat et al 2019 (15) reported more POCD among patients sedated with propofol, which was also suspected to decrease mean arterial pressure with a risk of brain hypoxia leading to cognitive decline (16). However, ketamine could preserve the hemodynamic stability during propofol-based anesthesia. In our study, we compared three groups with and without propofol. We did not find any impact of the kind of anesthesia on the cognitive decline suggesting that propofol did not impact cognition in this cohort. Furthermore, patients who received ketamine also received propofol supporting the hypothesis of a benefic effect of the ketamine during propofol infusion. However, even if several studies are in favor of ketamine protective effect, further prospective randomized controlled studies would be needed to confirm this hypothesis.
BDZ and AC drugs have long been suspected to be involved in drug-induced cognitive decline (17-19). On one hand, BDZ act as positive allosteric modulators of GABA-A (γ-aminobutyric acid) receptors, which results in an overall increased cortical inhibition, the latter possibly responsible for mental deterioration (20). On the other hand, delirium pathway arises from a dopamine excess and acetylcholine decrease, that could be due or accentuate directly by AC drugs (21). Furthermore, consequences of BDZ or AC treatment on cognition have been extensively described for speed-up age-related cognitive decay and Alzheimer’s disease, not for POCD (22, 23). In the present study, using a homogenous prospective non-emergent population, we observed detrimental effects of BDZ and AC drugs on cognitive functions. This finding supports previous results and hypotheses, however larger prospective and multicentric studies will be needed to confirm these results, allowing recommendations regarding the use of BDZ and AC drugs in non-emergent surgery. Clarifying a possible mechanisms underpinning BDZ/AC deleterious effects on sedated patients will benefit the prevention of temporary delayed NeuroCognitive Recovery as well as permanent POCD (NCD). We already know that the GABA-A agonist role of propofol is potentiated by BDZ, that could in turn promote POCD (24). However, among our 54 patients, 15 had no propofol, yet for this subpopulation the cognitive decline was statistically undistinguishable from the 39 other patients. A regular intake of BDZ was also associated to a reduced cognitive reserve (22), the latter linked to more POCD (25). Nevertheless, we found no correlation between MoCA (D-1) scores or sociocultural levels, and intake of BDZ/AC drugs.
Per-operative complications were associated with ΔMoCA both two days and six weeks after the surgery. These findings are consistent with previous works (26), and even more with recent works about complications following total hip arthroplasties (27). This suggests that an extended period under anesthesia does not affect post-operative cognition while a physiological response to a surgical complication does.Indeed, surgery-related inflammatory response has been associated to POCD (28). While we did not collect inflammatory-related biomarkers, we can note that among patient with peri-operative complications, only two had NSAIDs and three had ketamine.
Finally, depressed mood, in particular among the elderly, was associated with cognitive impairment (29). Several studies outlined the link between POCD and elderly mood deterioration, assessed using Geriatric Depression Scale both for patients receiving a cardiac as well as a non-cardiac surgery (26, 30). In our study, we found no link between cognitive decline and patient treated for depression, which agrees with these studies. Interestingly, we found that well being measured before and after the surgery were not linked to pain scores, in fact pre-operative well-being was here an independent factor associated with POCD, which was not the case for pain. These results might be explained by the fact that pain was carefully managed after the surgery (av. score ≈4), which does mitigate POCD (31). In the end, it emerges from this study that well-being measured before the surgery is a better indicator of cognitive decline than post-operative pain, which suggests that the patient’s state before surgery affects post-operative cognitive trajectory.

Limitations and perspectives

This study has some limitations. The patients included were very homogeneous but the inclusion criteria, the prospective methodology during a limited period led to a small number of patients. Larger multicentric studies would be needed to confirm or not these results and to propose recommendation regarding non-emergency arthroplasty.
In some group the number of patients was too low to provide useful data and/or multivariate analysis. However, the prospective methodology and our statistical approach allowed us to underline interesting insights about POCD. This article is the first result of a pilot study suggesting that cognitive decline occurring after a surgery could be anticipated based on pre-operative factors, yet only partially. We are performing a larger study in order to identify patients who are at risk of both delayed neurocognitive recovery and permanent cognitive decline.



To conclude, we reported a significant cognitive decline two days following a non-emergency lower limb surgery. BDZ or AC drugs were associated with a poor cognitive trajectory, both in short-term, but also in long-term in the case of BDZ drugs. At the same time, use of per-operative ketamine appeared to have a cognitive protective effect. Pre-operative well-being and perioperative complications were the two non-drug factors associated with positive and negative outcome respectively. Findings in the present study suggest that patient’s medical prescriptions and the surgery/anesthesia smooth progress can both independently affect POCD. Nevertheless, further research, based on a larger cohort, should investigate separately factors discussed above, in particular drugs for which neurophysiological mechanisms remain unclear and the role in cognitive decline/protection is still a divisive topic.


– Drop of MoCA after surgery is not associated with the type of anesthesia
– Pre-operative treatment and well-being are associated to MoCA points loss after surgery
– Per-operative complications and ketamine administration affect post-operative MoCA


Statement of Informed Consent: We obtained an authorization by the French data privacy administrative body ‘Commission Nationale de l’Informatique et des Libertés’ (CNIL), under the reference number z2c2680420w. In agreement with the ethics committee for this non-interventional study an oral agreement was obtained from each patient during their first medical visit.

Funding: Assistance Publique – Hôpitaux de Paris (APHP), Hôpital Lariboisière-Fernand Widal, Université de Paris.
Conflict of Interest: Pr. C. PAQUET is member of the International and National Advisory Boards of Lilly, ROCHE, Biogen. She is consultant of Fujiribio, ALZOHIS, NEUROIMMUNE and GILEAD and is involved as investigator in several clinical trials for Roche, Esai, Lilly, Biogen, Astra-Zeneca, Lundbeck, Neuroimmune. Dr. J. DUMURGIER is investigator in several passive anti-amyloid immunotherapies and other clinical trials for Roche, Eisai, Lilly, Biogen, Astra-Zeneca, Lundbeck. Similarly, Dr. C. HOURREGUE is investigator for Roche, Eisai and Biogen. Pr. R. NIZARD, Dr. C. RABUEL, Dr. L. COBLENTZ BAUMANN, Dr. C. LOYER, Dr. J. CARTAILLER and Dr. E. VANDERLYNDEN declare that they have no conflict of interest.

Authors’ contributions: Claire Paquet designed the study and analyzed the data and prepared the manuscript. Remy Nizard, Chirstophe Rabuel and Camille Loyer included the patients. Jerome Cartailler, Julien Dumurgier and Claire Paquet have performed statistical analysis and results interpretation. All authors read and validated the manuscript.



1. E. Braunwald, A. S. Fauci, D. L. Kasper, S. L. Hauser, D. L. Longo and J. L. Jameson, Harrison’s principles of internal medicine, McGraw Hill, 2001.
2. E. Mahanna-Gabrielli, K. J. Schenning, L. I. Eriksson, J. N. Browndyke, C. B. Wright, L. Evered, D. A. Scott, N. Y. Wang, C. H. Brown IV, E. Oh and others, «State of the clinical science of perioperative brain health: report from the American Society of Anesthesiologists Brain Health Initiative Summit 2018,» British Journal of Anaesthesia, 2019.
3. B. A. Fritz, P. L. Kalarickal, H. R. Maybrier, M. R. Muench, D. Dearth, Y. Chen, K. E. Escallier, A. B. Abdallah, N. Lin and M. S. Avidan, «Intraoperative electroencephalogram suppression predicts postoperative delirium,» Anesthesia and analgesia, 2016; vol. 122, p. 234.
4. T. S. Wildes, A. M. Mickle, A. B. Abdallah, H. R. Maybrier, J. Oberhaus, T. P. Budelier, A. Kronzer, S. L. McKinnon, D. Park, B. A. Torres and others, «Effect of electroencephalography-guided anesthetic administration on postoperative delirium among older adults undergoing major surgery: the ENGAGES randomized clinical trial,» Jama, 2019; vol. 321, pp. 473-483.
5. L. S. Rasmussen, A. Steentoft, H. Rasmussen, P. A. Kristensen and J. T. Moller, «Benzodiazepines and postoperative cognitive dysfunction in the elderly. ISPOCD Group. International Study of Postoperative Cognitive Dysfunction,» British journal of anaesthesia, 1999; vol. 83, pp. 585-589.
6. U. Dokkedal, T. G. Hansen, L. S. Rasmussen, J. Mengel-From and K. Christensen, «Cognitive functioning after surgery in middle-aged and elderly Danish twins,» Anesthesiology: The Journal of the American Society of Anesthesiologists, 2016; vol. 124, pp. 312-321.
7. P. Pandharipande, A. Shintani, J. Peterson, B. T. Pun, G. R. Wilkinson, R. S. Dittus, G. R. Bernard and E. W. Ely, «Lorazepam is an independent risk factor for transitioning to delirium in intensive care unit patients,» Anesthesiology: The Journal of the American Society of Anesthesiologists, 2006; vol. 104, pp. 21-26.
8. C. Pratico, D. Quattrone, T. Lucanto, A. Amato, O. Penna, C. Roscitano and V. Fodale, «Drugs of anesthesia acting on central cholinergic system may cause post-operative cognitive dysfunction and delirium,» Medical hypotheses, 2005;vol. 65, pp. 972-982.
9. T. L. Tsai, L. P. Sands and J. M. Leung, «An update on postoperative cognitive dysfunction,» Advances in anesthesia, 2010; vol. 28, pp. 269-284.
10. Z. S. Nasreddine, N. A. Phillips, V. Bédirian, S. Charbonneau, V. Whitehead, I. Collin, J. L. Cummings and H. Chertkow, «The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment,» Journal of the American Geriatrics Society, 2005; vol. 53, pp. 695-699.
11. J. Sleigh, M. Harvey, L. Voss and B. Denny, «Ketamine–More mechanisms of action than just NMDA blockade,» Trends in anaesthesia and critical care, 2014; vol. 4, pp. 76-81.
12. O. Dale, A. A. Somogyi, Y. Li, T. Sullivan and Y. Shavit, «Does intraoperative ketamine attenuate inflammatory reactivity following surgery? A systematic review and meta-analysis,» Anesthesia & Analgesia, 2012; vol. 115, pp. 934-943.
13. M. S. Avidan, H. R. Maybrier, A. B. Abdallah, E. Jacobsohn, P. E. Vlisides, K. O. Pryor, R. A. Veselis, H. P. Grocott, D. A. Emmert, E. M. Rogers and others, «Intraoperative ketamine for prevention of postoperative delirium or pain after major surgery in older adults: an international, multicentre, double-blind, randomised clinical trial,» The Lancet, 2017; vol. 390, pp. 267-275.
14. V. S. Hambrecht-Wiedbusch, D. Li and G. A. Mashour, «Paradoxical EmergenceAdministration of Subanesthetic Ketamine during Isoflurane Anesthesia Induces Burst Suppression but Accelerates Recovery,» Anesthesiology: The Journal of the American Society of Anesthesiologists, 2017; vol. 126, pp. 482-494.
15. O. Chaiwat, M. Chanidnuan, W. Pancharoen, K. Vijitmala, P. Danpornprasert, P. Toadithep and C. Thanakiattiwibun, «Postoperative delirium in critically ill surgical patients: incidence, risk factors, and predictive scores,» BMC anesthesiology, 2019; vol. 19, p. 39.
16. K. Wild, D. Howieson, F. Webbe, A. Seelye and J. Kaye, «Status of computerized cognitive testing in aging: a systematic review,» Alzheimer’s & Dementia,2008; vol. 4, pp. 428-437.
17. J. M. Starr and L. J. Whalley, «Drug-induced dementia,» Drug Safety, 1994;vol. 11, pp. 310-317.
18. R. T. Bartus, R. L. 3. Dean, B. Beer and A. S. Lippa, «The cholinergic hypothesis of geriatric memory dysfunction,» Science, 1982; vol. 217, pp. 408-414.
19. H. Hampel, M. M. Mesulam, A. C. Cuello, A. S. Khachaturian, A. Vergallo, M. R. Farlow, e. al and APMI, «Revisiting the Cholinergic Hypothesis in Alzheimer’s Disease: Emerging Evidence from Translational and Clinical Research,» The journal of prevention of Alzheimer’s disease, 2019; vol. 6, no. 1, pp. 2-15.
20. C. E. Griffin III, A. M. Kaye, F. R. Bueno and A. D. Kaye, «Benzodiazepine pharmacology and central nervous system–mediated effects,» The Ochsner Journal, 2013;vol. 13, pp. 214-223.
21. J. R. Maldonado, «Neuropathogenesis of delirium: review of current etiologic theories and common pathways,» The American Journal of Geriatric Psychiatry, 2013;vol. 21, pp. 1190-1222.
22. S. B. Gage, Y. Moride, T. Ducruet, T. Kurth, H. Verdoux, M. Tournier, A. Pariente and B. Bégaud, «Benzodiazepine use and risk of Alzheimer’s disease: case-control study,» Bmj, 2014; vol. 349, p. g5205.
23. G. Grande, I. Tramacere, D. L. Vetrano, S. Pomati, C. Mariani and G. Filippini, «Use of benzodiazepines and cognitive performance in primary care patients with first cognitive complaints,» International psychogeriatrics, 2018; vol. 30, pp. 597-601.
24. B. A. Orser and D. R. Miller, «Propofol-benzodiazepine interactions: insights from a “bench to bedside” approach,» Canadian Journal of Anesthesia/Journal canadien d’anesthésie, 2001;vol. 48, pp. 431-434.
25. I. Feinkohl, G. Winterer and T. Pischon, «Hypertension and risk of post-operative cognitive dysfunction (POCD): A systematic review and meta-analysis,» Clinical practice and epidemiology in mental health: CP & EMH, 2017; vol. 13, p. 27.
26. N. H. Greene, D. K. Attix, B. C. Weldon, P. J. Smith, D. L. McDonagh and T. G. Monk, «Measures of executive function and depression identify patients at risk for postoperative delirium,» Anesthesiology: The Journal of the American Society of Anesthesiologists, 2009; vol. 110, pp. 788-795.
27. K. T. Aziz, M. J. Best, Z. Naseer, R. L. Skolasky, K. E. Ponnusamy, R. S. Sterling and H. S. Khanuja, «The Association of Delirium with Perioperative Complications in Primary Elective Total Hip Arthroplasty,» Clinics in orthopedic surgery, 2018; vol. 10, pp. 286-291.
28. A. Alam, Z. Hana, Z. Jin, K. C. Suen and D. Ma, «Surgery, neuroinflammation and cognitive impairment,» EBioMedicine, 2018.
29. J. A. Brommelhoff, M. Gatz, B. Johansson, J. J. McArdle, L. Fratiglioni and N. L. Pedersen, «Depression as a risk factor or prodromal feature for dementia? Findings in a population-based sample of Swedish twins.,» Psychology and aging, 2009; vol. 24, p. 373.
30. J. L. Rudolph and E. R. Marcantonio, «Postoperative delirium: acute change with long-term implications,» Anesthesia and analgesia, 2011; vol. 112, p. 1202.
31. S. K. Inouye, T. Robinson, C. Blaum, J. Busby-Whitehead, M. Boustani, A. Chalian, S. Deiner, D. Fick, L. Hutchison, J. Johanning and others, «Postoperative delirium in older adults: best practice statement from the American Geriatrics Society,» Journal of the American College of Surgeons, 2015; vol. 220, pp. 136-148.
32. J. A. Yesavage, T. L. Brink, T. L. Rose, O. Lum, V. Huang, M. Adey and V. O. Leirer, «Development and validation of a geriatric depression screening scale: a preliminary report,» Journal of psychiatric research, 1982;vol. 17, pp. 37-49.
33. P. K. Mistry, G. S. Gaunay and D. M. Hoenig, «Prediction of surgical complications in the elderly: Can we improve outcomes?,» Asian journal of urology, 2017; vol. 4, pp. 44-49.
34. J. A. Hudetz and P. S. Pagel, «Neuroprotection by ketamine: a review of the experimental and clinical evidence,» Journal of cardiothoracic and vascular anesthesia, 2010. vol. 24, pp. 131-142.



T. Berkness1, M.C. Carrillo2, R. Sperling3,4, R. Petersen5, P. Aisen1, C. Flournoy1, H. Snyder2, R. Raman1,*,
J.D. Grill6,7,8,*
1. Alzheimer’s Therapeutic Research Institute, University of Southern California, San Diego, CA, USA; 2. Alzheimer’s Association, Division of Medical and Scientific Relations, Chicago, IL, USA; 3. Department of Neurology, Brigham and Women’s Hospital, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; 4. Department of Radiology, Division of Nuclear Medicine and Molecular Imaging, Massachusetts General Hospital, Harvard Medical School, Boston, MA, USA; 5. Department of Neurology, Mayo Clinic College of Medicine, Rochester, MN, USA; 6. Institute of Memory Impairment and Neurological Disorders, University of California at Irvine, Irvine, CA, USA; 7. Department of Psychiatry & Human Behavior, University of California at Irvine, Irvine, CA, USA; 8 Department of Neurobiology & Behavior, University of California at Irvine, Irvine, CA, USA; * Joint senior authors

Corresponding Author: Tyler Berkness, Alzheimer’s Therapeutic Research Institute, University of Southern California, San Diego, CA, USA,

J Prev Alz Dis 2021;
Published online April 2, 2021,



Background: Alzheimer’s Disease and Related Dementias (ADRD) clinical trials require multidisciplinary expertise in medicine, biostatistics, trial design, biomarkers, ethics, and informatics.
Objectives: To provide focused interactive training in ADRD clinical trials to a diverse cadre of investigators.
Design: The Institute on Methods and Protocols for Advancement of Clinical Trials in ADRD (IMPACT-AD) is a novel multidisciplinary clinical trial training program funded by the National Institute on Aging and the Alzheimer’s Association with two educational tracks. The Professionals track includes individuals who fill a broad variety of roles including clinicians, study coordinators, psychometricians, and other study professionals who wish to further their knowledge and advance their careers in ADRD trials. The Fellowship track includes current and future principal investigators and focuses on the design, conduct and analysis of ADRD clinical trials.
Setting: The 2020 inaugural iteration of IMPACT-AD was held via Zoom.
Participants: Thirty-five trainees (15 Fellowship track; 20 Professionals track) were selected from 104 applications (34% acceptance rate). Most (n=25, 71%) identified as female. Fifteen (43%) were of a non-white race; six (18%) were of Hispanic ethnicity; eight (23%) indicated they were the first person in their family to attend college.
Measurements: Participants completed daily evaluations as well as pre- and post-course assessments of learning.
Results: Across topic areas, >90% of trainees evaluated their change in knowledge based on the lectures as “very much” or “somewhat increased.” The mean proportion correct responses in pre- and post-course assessments increased from 55% to 75% for the Professionals track and from 54% to 78% for the Fellowship track.
Conclusions: IMPACT-AD successfully launched a new training opportunity amid a global pandemic that preliminarily achieved the goals of attracting a diverse cohort and providing meaningful training. The course is funded through 2025.

Key words: IMPACT-AD, training, Alzheimer’s disease, ADRD, clinical Trials, diversity.



Key to the US National Plan to Address Alzheimer’s Disease and Related Dementias (ADRD) will be clinical trials of therapies that are capable of slowing or preventing the onset of symptoms (1). In addition to individuals living with dementia, ADRD trials enroll participants with mild cognitive impairment and preclinical Alzheimer’s disease stages, each requiring novel designs and methods (2, 3). There remains no FDA approved therapy for neuropsychiatric symptoms of ADRD (4) and these trials face unique challenges (5). ADRD trials incorporate a variety of clinical outcome measures, including cognitive, functional, and biomarker assessments (6-8). ADRD biomarkers can also be used as inclusion criteria and to support claims of disease modification (9). Across ADRD trial types, novel aspects of recruitment and retention (10), informed consent (11), and other ethical issues (12) such as the role of study partners, require sensitive attention. In short, ADRD trials are complex, multifaceted, and require unique training.
There is a dearth of qualified investigators with adequate training and expertise to conduct these complex studies (13). Such training is rarely provided through the traditional course of medical or biostatistical education. The complexity of ADRD trials requires a team science approach, often inclusive of medical doctors, neuropsychologists, biostatisticians, neuroimagers, and biomarker scientists, to name a few. The low availability of ADRD trialists, including clinical investigators, statisticians, and other experts represents a threat to the national ADRD research agenda. Not only must the pipeline of qualified trialists be increased, the makeup of this pool of investigators and research teams must be diversified (14).
A diverse team of investigators brings a multitude of ideas and perspectives to trial design and is essential to facilitate inclusive enrollment in ADRD trials (15-19). Diversifying study teams is a core component of the mission of the Alzheimer’s Clinical Trials Consortium (ACTC). The ACTC’s Inclusion, Diversity, and Education in Alzheimer’s disease Clinical Trials (IDEA-CT) Committee is charged with developing goals, formulating a strategic plan, and serving as a source of oversight to support the ACTC’s core values of inclusion, diversity and training in ADRD clinical trials.
To address these needs and goals, members of the ACTC IDEA-CT committee developed the Institute on Methods and Protocols for Advancement of Clinical Trials in ADRD (IMPACT-AD). IMPACT-AD is a novel multi-disciplinary clinical trial training program funded by and developed in partnership with the National Institute on Aging (NIA) and the Alzheimer’s Association. IMPACT-AD is funded through 2025 with the goal of developing a network of well-trained and diverse investigators that will shape the future of the field.
In this manuscript, we describe the development of the IMPACT-AD course and the results of the inaugural iteration, which was forced to move to a virtual format due to the COVID-19 pandemic.



Program Structure

We designed IMPACT-AD to include two tracks of training. A “Professionals Track” focused on training ADRD clinical trials team members who sought to further their knowledge and advance their careers in ADRD trials including clinicians, study coordinators, psychometricians, and other study professionals. A “Fellowship Track” focused on training current and future principal investigators and emphasized the design, conduct, management and analysis of ADRD clinical trials.
Four committees supported the planning and conduct of IMPACT-AD. A Curriculum Committee ensured fulfillment of learning objectives. Two application review committees evaluated applicants on merit while promoting diversity in IMPACT-AD. A Program Evaluation Committee assisted in determining the short and long-term effectiveness of the course. Thirty-seven experienced clinical trial investigators, primarily composed of ACTC site PIs and unit leaders, served as course faculty (Table 1). Sixteen “core faculty” provided mentorship in protocol development to the Fellowship track trainees.

Table 1. Course Faculty (*Core Faculty)


Outreach and Application Process

We employed a breadth of strategies to ensure our goal of a robust and diverse course applicant pool. A Request for Applications (RFA) announced the course and outlined the application requirements, including: 1) personal statement; 2) letter of support from a mentor or supervisor; and 3) NIH biosketch. For the Fellowship track, a draft protocol using the ACTC Protocol Synopsis template was also required. The RFA was disseminated widely. The Alzheimer’s Association’s International Society to Advance Alzheimer’s Research and Treatment (ISTAART) shared the RFA with their mailing list (n=2100) and active research awardees (n=540), including their diversity fellowship recipients. The NIA distributed the RFA to 2019 grantees (n=2300) and to alumni of the Butler-Williams Scholars Program. We sent the RFA to the ACTC steering committee members and investigative teams for numerous studies coordinated by the USC Alzheimer’s Therapeutic Research Institute (n=530) and to the National Alzheimer’s Coordinating Center’s mailing list (n=780). Applications were submitted through the Alzheimer’s Association’s centralized ProposalCentral web-based grant management service.

Selection Criteria

Each application was reviewed and scored by no fewer than five reviewers including the course co-directors. Selection criteria included: 1) demonstration of passion and commitment for ADRD clinical trials and likelihood of future involvement in ADRD research; 2) level of support from a supervising faculty member; 3) publication record; and for the Fellowship track 4) the quality of the draft protocol. Two remote study sections were convened to discuss applications and select the class of 2020.

Course Curriculum

The course curriculum included didactic lectures and active learning workshops over four days. Professionals track trainees participated for two days; Fellowship track trainees participated for the duration of the course. Didactic lectures addressed fundamental concepts in clinical trials as well as unique aspects within ADRD (Table 2). Three active learning workshops addressed scientific communication, trial publications, and securing funding. For the Fellowship track, additional protocol workgroups focused on trial design and protocol development skills. Workgroups were comprised of three Fellowship track trainees and at least three course core faculty members, including two clinical and one biostatistical faculty. Protocol workgroups focused on five specific topics: 1) trial designs; 2) selecting a sample and developing inclusion criteria; 3) selecting a primary (and other) outcome measures; 4) statistical analysis plans; 5) safety monitoring and other conduct considerations.

Table 2. Didactic Lectures and Workshop Content

* Workshops

Course Evaluations

We collected evaluations on all sessions and lectures within each session. Trainees assessed several aspects of the course including the value of each covered topic, prior knowledge of the topic and the effect on the participant’s knowledge of the lecture. Trainees scored sessions using Likert response scales tailored to each question (e.g., “Very strong”, “Strong”, “Moderate” and “Weak” as options for “What was your prior knowledge of this topic?”).
We used pre- and post-course evaluations of knowledge to determine the overall educational value of the course. Separate post-test evaluations were performed at the conclusion of Days 2 (end of the Professionals track) and 4 (end of the Fellowship track). We compared the group scores pre- and post-course completion.



Characteristics of Applicants and Selected Trainees

We received 104 eligible applications including 48 for the Fellowship track and 56 for the Professionals track. Sixteen individuals applied to both tracks. Most applicants were female and nearly half identified as being from a non-white race and/or Hispanic/Latino ethnicity (Table 3). Twenty-three applicants (22%) indicated that they were the first in their family to attend college. Forty-six (44%) were from Institutions outside of the ACTC network.

Table 3. IMPACT-AD Applicant and Trainee Demographics


Thirty-five trainees (15 in the Fellowship track and 20 in the Professionals track) were selected to participate in the course, resulting in a 34% acceptance rate. Among selected trainees, the majority were female. Seven (20%) identified as African American or Black, four (11%) as Asian, twenty (57%) as White/Caucasian, three (8.5%) as multi-racial, one (3%) as Other race and six (18%) identified as being of Hispanic ethnicity. Eight trainees (23%) identified as being the first person in their family to attend college. Eleven (31%) held Professional degrees (e.g. MD, DDS, MBBS), fifteen (43%) held Doctorate degrees (e.g. PhD, PsyD), six (17%) held Master’s degrees, and three (9%) had a Bachelor’s degree. Thirteen (37%) were from institutions outside of the ACTC network. For the Fellowship track, eleven (73%) trainees proposed trials of nonpharmacological interventions, and four (27%) proposed drug trials.

Course Evaluations and Assessment of Learning

Each day of the course achieved at least an 80% response rate for program evaluations. Table 4 overviews the course evaluations for each of the sessions. On average, lecture topics were rated as “essential” by 76% and “valuable” by 22% of trainees. None of the topics received any assessment of “not necessary.”
Across lecture topics, 22%, 26%, 42%, and 10% of trainees rated their prior knowledge of topics as “very strong,” “strong,” “moderate,” and “weak”, respectively. The areas deemed as the greatest need by trainees (most responses of weak prior knowledge) included those in statistical design and analysis, with 22% of trainees identifying their prior knowledge as weak.
Across topic areas, 52%, 39%, 7%, and 3% of trainees self-reported their change in knowledge based on the lectures as “very much increased,” “somewhat increased,” “slightly increased,” and “no change”, respectively. Based on pre- and post-course assessments, each track demonstrated a positive effect of the course on trial knowledge (Figure 1). The mean proportion correct responses for the Professionals track increased from 55% to 75%. The Fellowship track improved from 54% to 78% correct responses.

Table 4. Evaluation Summaries

Mean scores are presented for each session, which included 2-6 lectures of varying lengths.

Figure 1. Pre- and Post-Course Quizzes of Course Learning

Mean performance on pre- and post-course assessments of knowledge are presented for days 1 vs. 2 (panel A), which included both the Professionals and Fellowship tracks (n=33 pre and n=35 post), and for days 1 vs. 4 (panel B), which included only the Fellowship track (n=14 pre and n=15 post).



IMPACT-AD was envisioned as an annual in-person course held at the ACTC Coordinating Center/University of Southern California’s Alzheimer’s Therapeutic Research Institute in San Diego, CA. The COVID-19 pandemic caused by the novel coronavirus SARS-CoV-2 forced implementation of a virtual format for the inaugural iteration of IMPACT-AD. As a result, we significantly adjusted the course’s structure and format in an effort to accommodate trainees’ time zones and ensure achievement of the course objectives. The course days had to be shortened and morning and evening activities were cancelled. The planned educational content remained largely intact. The results presented here indicate that these efforts were successful.
The course achieved its primary educational goals. Trainees received instruction in key topics related to ADRD interventional research and showed increased knowledge as a result of their training. Long-term evaluations will assess whether trainees continue their roles in ADRD trials, whether they achieve career advances supported by their participation in the course, and whether Fellowship track trainees successfully conduct their proposed trials.
Increasing investigator diversity is an important goal for the ACTC and specifically the IDEA-CT committee and more broadly for the field of ADRD research (20, 21). The inaugural IMPACT-AD course achieved the goal of including a diverse cohort of trainees. Trainees were diverse in sex, race and ethnicity, as well as professional backgrounds and current positions. Notably, eight trainees were the first in their families to attend college. While these diverse trainees were generally already working in ADRD trials, the course aims to give them added tools to be successful, advance in their careers, and inspire them to continue their work in the field.
A main goal of the IMPACT-AD course is to establish a network of peers that can remain connected, learn from each other, and support each other’s careers. Establishing this sense of camaraderie was made more challenging by the necessitated virtual conduct of the course. In partnership with the trainees, however, we created an IMPACT-AD Alumni Platform through the professional networking site LinkedIn. Thirty-one of 35 trainees (89%) have enlisted in this group. The Alumni Platform plans to interact virtually to discuss recent publications, plan and hold seminars, and discuss available funding and collaboration opportunities. The group is led by an IMPACT-AD Alumni Committee, composed of four trainees (two from each track). We also plan to hold an in-person event with the Class of 2020 at the earliest safe opportunity and will pursue other opportunities to connect alumni from subsequent iterations of the course.
IMPACT-AD has received funding to hold an annual course for the next four years. Based on the first year’s conduct, several changes are planned. Applicants will be required to select only one track. We anticipate holding informational webinars to answer potential applicant questions and offer guidance on the qualities that distinguished successful applications. Course content will be reorganized, emphasizing fundamental information on trial design (randomization, blinding, etc.) earlier in the agenda. We also anticipate developing some recorded lectures or webinars that will be offered to participants prior to the course to address the areas acknowledged by trainees as greatest needs (i.e., basic design and statistical analysis). We aim to improve evaluation completion rates.



The first year of the IMPACT-AD course was successful, despite unforeseen challenges resulting from the COVID-19 global pandemic. A diverse cohort of trainees was recruited and trained, and available data suggest that the training was effective. With strong partnerships with the NIA, the Alzheimer’s Association, and ACTC, the IMPACT-AD course is poised to continue its mission to train and diversify the next generation of ADRD trial investigators.


Funding: This work was supported by NIA U13AG067696, NIA U24AG057437, and Alzheimer’s Association SG-20-693744. JDG is supported by NIA AG066519 and NCATS UL1 TR001414.

Acknowledgments: We thank Dr. Laurie Ryan and Dr. Kristina McLinden from the National Institute on Aging for their key contributions in leadership and scientific guidance on developing and holding the inaugural iteration of IMPACT-AD. We thank the USC ATRI Events and IT teams and Ms. Chelsea Cox and Ms. Kirsten Klein from UCI MIND for their invaluable support.

Ethical standards: This paper does not describe human subjects research and therefore there was no IRB approval needed.

Open Access: This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.



1. National Plan to Address Alzheimer’s. In: Services HaH, ed.
2. Sperling RA, Aisen PS, Beckett LA, et al. Toward defining the preclinical stages of Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011;7(3):280-292.
3. Petersen RC, Thomas RG, Aisen PS, et al. Randomized controlled trials in mild cognitive impairment: Sources of variability. Neurology. 2017;88(18):1751-1758.
4. Geda YE, Schneider LS, Gitlin LN, et al. Neuropsychiatric symptoms in Alzheimer’s disease: past progress and anticipation of the future. Alzheimers Dement. 2013;9(5):602-608.
5. Cummings JL, Tractenberg RE, Gamst A, Teri L, Masterman D, Thal LJ. Regression to the mean: implications for clinical trials of psychotropic agents in dementia. Current Alzheimer research. 2004;1(4):323-328.
6. Fleisher AS, Raman R, Siemers ER, et al. Phase 2 safety trial targeting amyloid beta production with a gamma-secretase inhibitor in Alzheimer disease. Arch Neurol. 2008;65(8):1031-1038.
7. Rinne JO, Brooks DJ, Rossor MN, et al. 11C-PiB PET assessment of change in fibrillar amyloid-beta load in patients with Alzheimer’s disease treated with bapineuzumab: a phase 2, double-blind, placebo-controlled, ascending-dose study. Lancet neurology. 2010;9(4):363-372.
8. van Dyck CH, Nygaard HB, Chen K, et al. Effect of AZD0530 on Cerebral Metabolic Decline in Alzheimer Disease: A Randomized Clinical Trial. JAMA neurology. 2019.
9. Cummings JL, Doody R, Clark C. Disease-modifying therapies for Alzheimer disease: challenges to early intervention. Neurology. 2007;69(16):1622-1634.
10. Grill JD, Karlawish J. Addressing the challenges to successful recruitment and retention in Alzheimer’s disease clinical trials. Alzheimers Res Ther. 2010;2(6):34.
11. Karlawish JH. Research involving cognitively impaired adults. The New England journal of medicine. 2003;348(14):1389-1392.
12. Milne R, Karlawish J. Expanding engagement with the ethical implications of changing definitions of Alzheimer’s disease. Lancet Psychiatry. 2017;4(4):e6-e7.
13. Sung NS, Crowley WF, Jr., Genel M, et al. Central challenges facing the national clinical research enterprise. JAMA. 2003;289(10):1278-1287.
14. Getz K, Faden L. Racial disparities among clinical research investigators. Am J Ther. 2008;15(1):3-11.
15. Arean PA, Gallagher-Thompson D. Issues and recommendations for the recruitment and retention of older ethnic minority adults into clinical research. Journal of consulting and clinical psychology. 1996;64(5):875-880.
16. Rabinowitz YG, Gallagher-Thompson D. Recruitment and retention of ethnic minority elders into clinical research. Alzheimer Dis Assoc Disord. 2010;24 Suppl:S35-41.
17. Watson JL, Ryan L, Silverberg N, Cahan V, Bernard MA. Obstacles and opportunities in Alzheimer’s clinical trial recruitment. Health affairs (Project Hope). 2014;33(4):574-579.
18. Olin JT, Dagerman KS, Fox LS, Bowers B, Schneider LS. Increasing ethnic minority participation in Alzheimer disease research. Alzheimer Dis Assoc Disord. 2002;16 Suppl 2:S82-85.
19. Dilworth-Anderson P, Thaker S, Burke JM. Recruitment strategies for studying dementia in later life among diverse cultural groups. Alzheimer Dis Assoc Disord. 2005;19(4):256-260.
20. Elliot CL. Together We Make the Difference: National Strategy for Recruitment and Participation in Alzheimer’s and Related Dementias Clinical Research. Ethnicity & disease, 2020, Vol.30 (Suppl 2), p. 705 -708
21. Schneider J, Jeon S, Gladman JT, Corriveau RA. ADRD Summit 2019 Report to the National Advisory Neurological Dosirders and Stroke Counsil. Alzheimer’s Disease-Related Dementias. 2019, March 14-15, p. 7-19.


E.N. Madero1, J. Anderson1, N.T. Bott1,2, A. Hall1, D. Newton1, N. Fuseya1, J.E. Harrison1,3,4,5, J.R. Myers1, J.M. Glenn1,6

1. Neurotrack Technologies, Redwood City, CA, USA; 2. Clinical Excellence Research Center, Stanford University School of Medicine, Palo Alto, CA, USA; 3. Metis Cognition Ltd, Wiltshire, UK; 4. Institute of Psychiatry, Psychology & Neuroscience, King’s College London, London, UK; 5. Alzheimer Center Amsterdam, Department of Neurology, Amsterdam, the Netherlands; 6. University of Arkansas, Department of Health, Human Performance, and Recreation, Fayetteville, AR, USA.

Corresponding Author: Jennifer Rae Myers, 399 Bradford Street Ste. 101, Redwood City, CA 94063, USA, Email:, Phone: 1 (301) 531-4179

J Prev Alz Dis 2021;
Published online March 17, 2021,



The current demand for cognitive assessment cannot be met with traditional in-person methods, warranting the need for remote unsupervised options. However, lack of visibility into testing conditions and effort levels limit the utility of existing remote options. This retrospective study analyzed the frequency of and factors associated with environmental distractions during a brief digital assessment taken at home by 1,442 adults aged 23-84. Automated scoring algorithms flagged low data capture. Frequency of environmental distractions were manually counted on a per-frame and per-trial basis. A total of 7.4% of test administrations included distractions. Distractions were more frequent in men (41:350) than women (65:1,092) and the average age of distracted participants (51.7) was lower than undistracted participants (57.8). These results underscore the challenges associated with unsupervised cognitive assessment. Data collection methods that enable review of testing conditions are needed to confirm quality, usability, and actionability.

Key words: Cognitive testing, remote assessment, unsupervised, quality assurance, eye tracking.



Demand for cognitive testing continues to outpace the supply of providers available for in-person evaluation, and this disparity is expected to increase as the population ages (1). Furthermore, the onset of the COVID-19 pandemic has disproportionately affected the availability of neurocognitive testing due to the few reliable and valid remote digital testing options (2). High quality digital cognitive assessments that can be administered remotely and asynchronously are urgently needed to meet the growing demand and backlog of patients requiring neuropsychological assessment. While the availability of computerized cognitive assessments has increased rapidly over the past decade (3, 4), the clinical validity of these assessments in a remote setting remains a significant issue for both researchers and clinicians. Moreover, the use of computerized cognitive assessments in such unsupervised settings raises an equally important issue regarding environmental validity (5).
Few studies have compared the outcomes of digital cognitive assessments taken in supervised and unsupervised environments, showing similar overall results between administration settings (6-8). However, the modality of data collection with most computerized cognitive assessments precludes the ability to assess the physical environment, level of effort, or to verify the identity of the participant during assessment administration. Lack of insight into these factors has prevented widespread adoption of remote unsupervised cognitive assessment in clinical and research settings. The difficulties surrounding reliability and validity of remote cognitive assessments have been magnified during the COVID-19 pandemic. With in-person testing unavailable, clinicians and clinical researchers have sought out digital testing options; however, a paucity of data demonstrating reliability and validity across clinical populations and settings exists for remote administration of digital cognitive assessments. Without an ability to “have eyes on the patient” there is significant clinical risk that environmental distractions will result in test performances that do not reflect the participant or patient’s true abilities.
The recording of eye movements with device-embedded cameras to assess cognition is a burgeoning area of research. As web cameras have become standard hardware in most smartphones, tablets, and laptop computers, opportunities exist to develop eye movement-based tasks to efficiently and quickly assess cognitive function through these devices.9 Visual paired comparison task paradigms assess recognition memory through eye movements and have been shown to reliably detect memory dysfunction, representing a readily deployable paradigm to devices with web cameras for the rapid assessment of declarative memory dysfunction (10-12). The collection of video data for eye tracking purposes also provides an opportunity to assess environmental conditions and quantify the occurrence of distractions during test administration. This study aimed to investigate the frequency of and factors associated with environmental distractions during a brief unsupervised digital cognitive assessment in a real-world setting.



This was a retrospective study of 1,442 adults aged 23-84 who completed a 5-minute eye tracking-based visual paired comparison task in an unsupervised remote setting and was approved by the University of Arkansas Institutional Review Board. Participants completed the task in their homes utilizing a web camera on their laptop or desktop computer. Briefly, participants were shown a series of identical image pairs during a familiarization phase. Participants were then shown a series of non-identical image pairs during the test phase, each consisting of one novel and one familiar image and tasked with focusing their gaze on the novel image. The main outcome measure was novelty preference, or the proportion of time spent viewing the novel images compared to familiar images, which is lower in individuals with impaired memory function than in individuals with normal memory function. The task is described in more detail elsewhere (13).
Automated algorithms scored the exams and subsequently flagged low data capture across the 20 test trials. Distractions were operationalized by a third party source. The frequency of environmental distractions which resulted in participants looking away from the camera (e.g., interruptions, fatigue, lack of interest) were manually counted on a per-frame and per-trial basis. Overall frequency within the sample was counted to investigate the percentage of tests impacted by environmental distraction. A Fisher’s exact test was used to compare the frequency of distractions during the assessment by sex. A Welch’s t-test was used to compare the age of participants across assessment administrations with and without environmental distractions. A Welch’s t-test was used to compare the novelty preference scores for participants who were distracted during the assessment administration and participants who were not distracted.



Results are highlighted in Figure 1. A total of 1,442 participants (mean age = 57.4, SD 12.2) completed the visual paired comparison task. Seventy six percent of the participants (n = 1,092) were female. Of the 1,442 assessment administrations, 106 (7.4%) included environmental distraction resulting in participants looking away from the screen at least one time during test trials.
Assessment administrations with environmental distractions were more frequent in male participants (41:350) than female participants (65:1,092), with an odds ratio of 2.10 (p <.001). The mean age of participants with environmental distractions (M = 51.7, SD=13.8) was significantly lower than participants without the presence of environmental distractions (M = 57.8, SD=11.9) (t = -4.44, p <.001). Lastly, novelty preference scores were lower for participants who were distracted (M = 55.6%, SD = 8.0) compared to those who were not distracted (M = 58.8%, SD = 9.1) (t = 3.7, p <.001).

Figure 1. Descriptive comparisions of participants with and without environmental distractions



Digital cognitive assessments that can be taken remotely and asynchronously represent a compelling solution to meet the growing demand for cognitive testing. Adoption of available testing options remains low due to uncertainty surrounding the quality, usability, and actionability of the data collected. In this study, we set out to measure the occurrence of environmental distractions, defined as periods of time spent looking away from the camera, during an unsupervised at-home administration of a brief cognitive assessment.
The role environmental distractions play in assessment variability is not limited to remote, asynchronous test administration. For example, Schatz and colleagues (2010) reported that high school athletes completing group baseline Impact testing that reported the presence of environmental distractions endorsed significantly more behavioral symptoms than those who did not report environmental distractions. The frequency of environmental distractions during the brief unsupervised cognitive assessment in this study (7.4%) were comparable to what Schatz and colleagues previously reported during cognitive testing batteries administered in group settings (9.7%) (14). While there is currently no established threshold for the amount of distraction that is acceptable to maintain test validity, these rates of distraction frequency likely introduce enough uncertainty to preclude clinicians and researchers from using remote cognitive testing data from unsupervised tests without a reliable and validated form of quality assurance. We also found relationships between the frequency of distractions and both the age and gender of the participants. These data suggest it may be possible to predict which participants are more likely to become distracted during a remote cognitive testing session based on standard demographic information. Additionally, participants who were distracted during the testing session scored significantly lower than participants who were not distracted. This highlights how time looking off screen may negatively impact scores and can also yield a potentially artificially low cognitive performance.
These results also underscore the challenges of high-quality data collection associated with unsupervised comprehensive cognitive assessment. This study used a brief 5-minute VPC task and included instructions at the beginning to ensure the testing environment was quiet and free from distractions. Most standard comprehensive digital cognitive assessments or assessment batteries require 30 to 45 minutes (e.g. the 30-minute National Institutes of Health Toolbox Cognitive Battery) (15). It is likely that longer durations of testing administration will result in increased likelihood of distractions during remote, asynchronous at-home administrations. The inability to determine which participants’ results may have been affected by distractions presents a challenge for researchers and clinicians attempting to use the data in clinical or research decisions.
Despite these challenges, unsupervised and asynchronous neuropsychological assessment remains a promising method for the efficient remote measurement of cognition, but only when data quality metrics can be collected and verified. The use of eye tracking-based cognitive assessments presents the unique opportunity to collect such data by having “eyes on the patient.” Our use of an automated algorithm to flag periods of low data capture and manual coding of environmental distractions when participants looked away from the screen for reasons including fatigue, interruptions, and lack of interest, provides a model for a scalable analysis of environmental conditions during remote cognitive assessment administrations.
The ubiquity of webcams in mobile devices, tablets, and computers presents an intriguing opportunity to further develop methods to enable the collection and rigorous analysis of remotely collected cognitive assessment data. In the future, the incorporation of methods that allow for identity verification will assure researchers and clinicians that the correct person is in fact the one completing the assessment. These developments can provide a level of data quality assurance not previously possible and lay the groundwork for the wider adoption of remote cognitive assessment options by clinicians and researchers to help meet the growing demand.


Funding: This work was supported by funding from Neurotrack Technologies, Inc.

Acknowledgments: The authors would like to thank the participants for their time.

Conflict of Interest: Dr. Harrison reports personal fees from Astra Zeneca, personal fees from Axon Neuroscience, personal fees from Axovant, personal fees from Biogen Idec, personal fees from Boehringer Ingelheim, personal fees from Signant, personal fees from CRF Health, personal fees from Eisai, personal fees from Eli Lilly, personal fees from GfHEU, personal fees from Heptares, personal fees from Kaasa Health, personal fees from MyCognition, personal fees from Neurocog, personal fees and other from Neurotrack, personal fees from Novartis, personal fees from Nutricia, personal fees from Probiodrug, personal fees from Regeneron, personal fees from Sanofi, personal fees from Servier, personal fees from Takeda, personal fees from vTv Therapeutics, personal fees from Lundbeck, personal fees from Compass Pathways, personal fees from G4X Discovery, personal fees from Cognition Therapeutics, personal fees from AlzeCure, personal fees from FSV7, personal fees from BlackThornRx, personal fees from Winterlight Labs, personal fees from Rodin Therapeutics, personal fees from Lysosome Therapeutics, personal fees from Syndesi Therapeutics, personal fees from Vivoryon Therapeutics, personal fees from Neurodyn Inc, personal fees from Aptinyx, personal fees from Athira Therapeutics, personal fees from EIP Pharma, personal fees from Cerecin, personal fees from Neurocentria, personal fees from Curasen, personal fees from Samumed, personal fees from Cognition Therapeutics, personal fees from ReMynd, personal fees from Ki-Elements, personal fees from The NHS, outside the submitted work.

Ethical standards: Ethical approval was obtained from the University of Arkansas Institutional Review Board.



1. Rao A, Manteau-Rao M, Aggarwal Nt. [P1-561]: Dementia Neurology Deserts: What Are They And Where Are They Located In The U.S.? Alzheimers Dement. 2017;13(7S_Part_10):P509-P509. doi:10.1016/j.jalz.2017.06.577
2. Hantke NC, Gould C. Examining Older Adult Cognitive Status in the Time of COVID-19. J Am Geriatr Soc. 2020;68(7):1387-1389. doi:10.1111/jgs.16514
3. Charalambous, A. P., Pye, A., Yeung, W. K., Leroi, I., Neil, M., Thodi, C., & Dawes, P. (2020). Tools for App-and Web-Based Self-Testing of Cognitive Impairment: Systematic Search and Evaluation. Journal of Medical Internet Research, 22(1), e14551. doi:10.2196/14551
4. Sabbagh, M. N., Boada, M., Borson, S., Doraiswamy, P. M., Dubois, B., Ingram, J., … & Vellas, B. (2020). Early detection of Mild Cognitive Impairment (MCI) in an at-home setting. The Journal of Prevention of Alzheimer’s Disease, 1-8. doi:10.14283/jpad.2020.22
5. Bilder, R. M., Postal, K. S., Barisa, M., Aase, D. M., Cullum, C. M., Gillaspy, S. R., … & Morgan, J. M. (2020). InterOrganizational practice committee recommendations/guidance for teleneuropsychology (TeleNP) in response to the COVID-19 pandemic. The Clinical Neuropsychologist, 34(7-8), 1314-1334.
6. Maljkovic, V., Pugh, M. A. M., Yaari, R., Shen, J., & Juusola, J. (2019). At Home Cognitive Testing (CANTAB battery) in Healthy Controls and Cognitively Impaired Patients: A Feasibility Study. Age, 66(69.65), 0-001.doi:10.1016/j.jalz.2019.06.1057
7. Cromer JA, Harel BT, Yu K, et al. Comparison of Cognitive Performance on the Cogstate Brief Battery When Taken In-Clinic, In-Group, and Unsupervised. Clin Neuropsychol. 2015;29(4):542-558. doi:10.1080/13854046.2015.1054437
8. Backx R, Skirrow C, Dente P, Barnett JH, Cormack FK. Comparing Web-Based and Lab-Based Cognitive Assessment Using the Cambridge Neuropsychological Test Automated Battery: A Within-Subjects Counterbalanced Study. J Med Internet Res. 2020;22(8):e16792. doi:10.2196/16792
9. Bott NT, Madero EN, Glenn JM, et al. Device-Embedded Cameras for Eye Tracking-Based Cognitive Assessment: Implications for Teleneuropsychology. Telemed J E Health. 2020;26(4):477-481. doi:10.1089/tmj.2019.0039
10. Bott N, Madero EN, Glenn J, et al. Device-Embedded Cameras for Eye Tracking–Based Cognitive Assessment: Validation With Paper-Pencil and Computerized Cognitive Composites. Journal of Medical Internet Research. 2018;20(7):e11143. doi:10.2196/11143
11. Crutcher MD, Calhoun-Haney R, Manzanares CM, Lah JJ, Levey AI, Zola SM. Eye tracking during a visual paired comparison task as a predictor of early dementia. Am J Alzheimers Dis Other Demen. 2009;24(3):258-266. doi:10.1177/1533317509332093
12. Zola SM, Manzanares CM, Clopton P, Lah JJ, Levey AI. A behavioral task predicts conversion to mild cognitive impairment and Alzheimer’s disease. Am J Alzheimers Dis Other Demen. 2013;28(2):179-184. doi:10.1177/1533317512470484
13. Bott NT, Lange A, Rentz D, Buffalo E, Clopton P, Zola S. Web Camera Based Eye Tracking to Assess Visual Memory on a Visual Paired Comparison Task. Front Neurosci. 2017; 11:370. doi:10.3389/fnins.2017.00370
14. Schatz P, Neidzwski K, Moser RS, Karpf R. Relationship between subjective test feedback provided by high-school athletes during computer-based assessment of baseline cognitive functioning and self-reported symptoms. Arch Clin Neuropsychol. 2010;25(4):285-292. doi:10.1093/arclin/acq022
15. Weintraub S, Dikmen SS, Heaton RK, et al. The cognition battery of the NIH toolbox for assessment of neurological and behavioral function: validation in an adult sample. J Int Neuropsychol Soc. 2014;20(6):567-578. doi:10.1017/S1355617714000320



R.E. Amariglio1,2, S.A.M. Sikkes6, G.A. Marshall1,2, R.F. Buckley2,7,8, J.R. Gatchel3,4, K.A. Johnson1,5, D.M. Rentz1,2, M.C. Donohue9, R. Raman9, C.-K. Sun9, R. Yaari10, K.C. Holdridge10, J.R. Sims10, J.D. Grill11, P.S. Aisen9, R.A. Sperling1,2 and the A4 Study Team


1. Center for Alzheimer Research and Treatment, Department of Neurology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, USA; 2. Department of Neurology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA; 3. Department of Psychiatry, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA; 4. Division of Geriatric Psychiatry, McLean Hospital, Belmont Massachusetts, USA; 5. Department of Radiology, Massachusetts General Hospital, Harvard Medical School, Boston, Massachusetts, USA; 6. Alzheimer Center Amsterdam, Department of Neurology, Amsterdam University Medical Centers, Amsterdam, Netherlands; 7. Florey Institute, University of Melbourne, Parkville, Victoria, Australia; 8. Melbourne School of Psychological Sciences, University of Melbourne, Parkville, Victoria, Australia; 9. Alzheimer’s Therapeutic Research Institute, Keck School of Medicine of the University of Southern California, San Diego, CA, USA; 10. Eli Lilly and Company, Indianapolis, IN, USA; 11. University of California Irvine, Irvine, USA

Corresponding Author: R.E. Amariglio, Center for Alzheimer Research and Treatment, Department of Neurology, Brigham and Women’s Hospital, Harvard Medical School, Boston, Massachusetts, 02115, USA,

J Prev Alz Dis 2021;
Published online March 5, 2021,



Background: Greater subjective cognitive changes on the Cognitive Function Index (CFI) was previously found to be associated with elevated amyloid (Aß) status in participants screening for the A4 Study, reported by study partners and the participants themselves. While the total score on the CFI related to amyloid for both sources respectively, potential differences in the specific types of cognitive changes reported by either participants or their study partners was not investigated.
Objectives: To determine the specific types of subjective cognitive changes endorsed by participants and their study partners that are associated with amyloid status in individuals screening for an AD prevention trial.
Design, Setting, Participants: Four thousand four hundred and eighty-six cognitively unimpaired (CDR=0; MMSE 25-30) participants (ages 65-85) screening for the A4 Study completed florbetapir (Aß) Positron Emission Tomography (PET) imaging. Participants were classified as elevated amyloid (Aß+; n=1323) or non-elevated amyloid (Aß-; n=3163).
Measurements: Prior to amyloid PET imaging, subjective report of changes in cognitive functioning were measured using the CFI (15 item questionnaire; Yes/Maybe/No response options) and administered separately to both participants and their study partners (i.e., a family member or friend in regular contact with the participant). The impact of demographic factors on CFI report was investigated. For each item of the CFI, the relationship between Aß and CFI response was investigated using an ordinal mixed effects model for participant and study partner report.
Results: Independent of Aß status, participants were more likely to report ‘Yes’ or ‘Maybe’ compared to the study partners for nearly all CFI items. Older age (r= 0.06, p<0.001) and lower education (r=-0.08, p<0.001) of the participant were associated with higher CFI. Highest coincident odds ratios related to Aß+ for both respondents included items assessing whether ‘a substantial decline in memory’ had occurred in the last year (ORsp= 1.35 [95% CI 1.11, 1.63]; ORp= 1.55 [95% CI 1.34, 1.79]) and whether the participant had ‘seen a doctor about memory’ (ORsp= 1.56 [95% CI 1.25, 1.95]; ORp =1.71 [95% CI 1.37, 2.12]). For two items, associations were significant for only study partner report; whether the participant ‘Repeats questions’ (ORsp = 1.30 [95% CI 1.07, 1.57]) and has ‘trouble following the news’ (ORsp= 1.46[95% CI 1.12, 1.91]). One question was significant only for participant report; ‘trouble driving’ (ORp= 1.25 [95% CI 1.04, 1.49]).
Conclusions: Elevated Aβ is associated with greater reporting of subjective cognitive changes as measured by the CFI in this cognitively unimpaired population. While participants were more likely than study partners to endorse change on most CFI items, unique CFI items were associated with elevated Aß for participants and their study partners, supporting the value of both sources of information in clinical trials.

Key words: Subjective cognitive cecline, amyloid, clinical trial, Alzheimer’s disease.



After a series of disappointing results at the symptomatic stages of Alzheimer’s disease (AD), therapeutic trials are increasingly moving towards prevention at the preclinical stage (1). Individuals enrolled in a secondary prevention trial are characterized as clinically normal, but are considered at increased risk of AD due to elevated biomarkers, such as amyloid (Aß) Positron Emission Tomography (PET) imaging. By shifting the focus earlier in the disease, however, demonstrating clinically meaningful treatment effects is challenging, since most individuals at the preclinical stage are not expected to demonstrate overt cognitive and functional impairment during the course of a trial (2). As such, efforts to identify new methods to capture subtle changes in cognitive functioning prior to the onset of objective clinical impairment are needed to quantify treatment effects with greater resolution.
Subjective report of everyday high-level cognitive functioning from both the individual and a close family member or friend, may offer a window into early cognitive changes along the preclinical stage. Indeed, prior studies have shown that both greater cognitive complaints from the participant, as well as from a study partner, are associated with higher likelihood of subsequent cognitive decline and clinical progression (3-6).
In addition to serving as outcome measures, subjective cognitive assessments may also facilitate the process of identifying individuals who meet criteria for preclinical AD. As individuals age, cognitive complaints become increasingly common and are not necessarily specific to a pathological process (7). Elucidating particular patterns of complaints from the participant and the study partner that relate to AD biomarkers may enhance the utility of subjective report that is sometimes dismissed for being non-specific (8, 9). Further, better characterization of the subtle changes that are observed at the preclinical stage may ultimately help to identify specific targets for therapeutic intervention.
In the current study, we examined data from individuals screening for the Anti-Amyloid Asymptomatic Alzheimer’s (A4) Study testing solanezumab, an anti-amyloid antibody, in a secondary AD prevention trial. In particular, we sought to build upon previous findings in the A4 screen data that found both participant and study partner report related to Aß on the total score of the Cognitive Function Index (CFI) (10), a subjective questionnaire that asks a participant and study partner about change in the participant’s cognitive functioning over the last year (3, 11). Here, we investigated the potential impact of demographic factors on participant vs. study partner report on the total score of the CFI, as well as which specific individual items on the CFI related to amyloid burden on PET. In this way, we aimed to elucidate the pattern of cognitive complaints at the preclinical stage of AD from both the perspective of the participant and study partner.



Data presented here come from participants who were screened for the A4 Study. In brief, the A4 Study is a preclinical stage treatment trial that is being conducted at 67 clinical trial sites in the U.S., Canada, Japan, and Australia, among participants with elevated Aß as determined by florbetapir PET. Participants first underwent an initial clinic screening visit and if eligibility criteria were met, subsequently underwent Aß PET imaging at a second screening visit. Participants who completed screening for the A4 study, were ages 65-85 years and were considered cognitively unimpaired, based on a global CDR (12) score of 0, Mini-Mental State Exam (MMSE) (13) score of 25-30, and Logical Memory II subscale delayed paragraph recall (LM-IIa) of the Wechsler Memory Scale-Revised (WMS-R) (14)score of 6-18. Moreover, participants did not have unstable or exclusionary medical or psychiatric problems. Participants had adequate literacy in English, Spanish, or Japanese, and had adequate vision and hearing to complete the required cognitive tests. Participants were required to have a study partner who was willing to provide collateral information about the participant’s everyday cognitive functioning; study partners were required to have at least weekly contact with participants in person, by phone, or by email. Key exclusion criteria for participants were diagnosis of cognitive impairment or dementia, use of AD medications, unstable anxiety or depression, or other unstable medical conditions, although participants with treated hypertension, diabetes, and other common medical ailments were permitted. Four thousand four hundred and eighty-six participants meeting these criteria then underwent florbetapir PET imaging.

Cognitive Function Index

The CFI was originally developed as a 14-item, self-administered mail-in screening instrument for AD diagnostic evaluation in prevention trials (10). The CFI has a participant version in which participants report on their own cognitive functioning, as well as a study partner version in which study partners report on the participants’ cognitive functioning. The CFI was previously found to have adequate validity and reliability (6, 10). All questions on the CFI ask about cognitive changes over the last year with response options that include Yes (2), No (0), and Maybe (1). Questions range from cognitive items (e.g., “Compared to one year ago, do you feel that your memory has declined substantially?”) to functional items (e.g., “Compared to one year ago, do you have more difficulty managing money?”). The A4 version of the CFI added an additional question, “In the past year, have you seen a doctor about memory concerns?” with response options: Yes (1) or No (0). On a few questions, (e.g., “Has your work performance (paid or volunteer) declined significantly, compared to one year ago?”), Non-Applicable (N/A) was also a response option. A total CFI score can be derived by summing each item of the participant and study partner versions of the questionnaire respectively (the range is 0-29 with higher scores indicating greater complaints about cognitive functioning difficulties). The CFI was administered separately to both the participant and their study partner at the first screen visit prior to florbetapir PET imaging at the second screening visit.

Amyloid PET Imaging

Florbetapir PET was acquired 50-70 minutes after injection of 10 mCi of florbetapir F 18. Amyloid eligibility (elevated [Aß+] and eligible to continue in screening vs. not elevated [Aß-] and ineligible) was assessed using an algorithm combining both quantitative standardized uptake value ratio (SUVr) methodology and qualitative visual read performed at a central laboratory. Mean cortical standardized uptake value ratio [SUVr] using a whole cerebellar reference region of ≥1.15 was utilized to define elevated amyloid as the primary criterion, as quantitative assessment was thought to be more sensitive to the presence of early amyloidosis in the preclinical stage of AD. A SUVr between 1.10 and 1.15 was considered to be elevated amyloid only if the visual read was considered positive by a two independent-reader consensus determination (10).

Statistical Analyses

Demographic factors of the participant (e.g., sex, age, education) and of the study partner (e.g., partner sex, partner age, living status with participant) were summarized by Aß status with means, standard deviations, and two-sample t-test; or counts, percentages, and Fisher’s Exact test. Pearson’s correlation coefficients were calculated for continuous characteristics and CFI scores. Linear regression models were used to determine whether there was an interaction between Aß status and each participant and study partner characteristic on the CFI score for the participant and study partner report.
To compare level of endorsement on each item of the CFI between participant and study partner, a cumulative odds mixed effects model was employed with CFI response (No=0, Maybe=1,Yes= 2) as the outcome and CFI source (participant or study partner), age, sex, and education as the predictors; and dyad-specific random intercepts. Non-applicable data was treated as missing. Item level missing data was rare and was not analyzed with imputation. To determine the relationship between Aß status and level of endorsement on each item, separate cumulative odds models were fit for each CFI item. CFI response for participant and study partner were run separately as the dependent variable and Aß status as the predictor controlling for age, education, and sex. The false discovery rate for item level analysis was adjusted within source (participant vs. study partner) by the method of Benjamini and Hochberg (1995) (15). Statistical analyses were performed using the R software (


Of the 4486 participants, 1322 were categorized as Aß+ (29.5%) and 3163 were Aß-. Aß+ participants were slightly older than Aß- (see table 1). No differences between Aß groups were observed by sex, years of education, or marital or retirement status. Study partners were majority spouses (62%) and female (60%) and were age 65.8±11.2 years. There were no differences in relationship to study partner by Aß group.

Table 1. Demographic variables of all participants with comparison of Not elevated (Aß-) and elevated amyloid (Aß+) groups

1. Fisher’s Exact test

Higher score on the participant CFI was associated with older age (r= 0.06, p<0.001) and lower education (r=-0.08, p<0.001), but was not associated with sex (t=-2.68, p=0.79). Study partner CFI score was higher for female study partners (meanfemale= 2.94, meanmale= 2.54; t= 3.5, p=0.0005) and if a study partner lived with the participant (meanlive with= 2.92, meanlive separate= 2.49; t=-3.7, p=0.0003). Older age of the participant related to higher study partner CFI score (r= 0.07, p<0.001).
Next, we were interested in whether demographic factors modified the relationship between Aß and the CFI total score for participant report and study partner report respectively. When examining participant report, there was not a significant interaction between Aß and sex (ß= -0.14, p=0.30), Aß and age (ß= 0.015, p=0.27), nor Aß and education level of participant (ß= 0.106, p= 0.47) to predict participant CFI score. For study partners, there was not a significant interaction between Aß and sex of the study partner (ß= 0.05, p =0.85) nor Aß and living situation of study partner with participant (ß= 0.17, p=0.53). The interaction between Aß and age of study partner to predict study partner CFI score was at the significance nominal threshold (ß= 0.02, p= 0.05). Specifically, among Aß-, younger study partners tended to report higher CFI scores. However, among Aß+, age of study partner did not modify CFI report.
At the item level, the 3 most commonly endorsed (‘Yes’ or ‘Maybe’ response) items were the same for both participant and study partner report (see figure : ‘trouble with names and words’ ‘relying on written reminders’ and ‘misplacing things’ (See figure 1, supplemental table 1). The 3 least endorsed items were also the same for participant and study partner report: ‘managing money’ ‘difficulty with hobbies’ ‘difficulty with appliances’. For one item, ‘trouble with work performance,’ 18% of participants and 25% of study partners did not find this item applicable.
In general, greater endorsement was found for participants compared to their study partners (see figure 1). The only items that did not significantly differ in level of endorsement between participant and study partner included ‘Seen a doctor about memory,’ ‘Repeating questions,’ ‘Difficulty with appliance and electronic devices.’

Figure 1. Percentage of endorsement on CFI for participant and study partner report at the item level


As was previously reported (10), Aß+ participants reported higher total CFI (Cohen d = 0.31, p<0.001) and study partners reported Aß+ participants had higher total CFI (Cohen d = 0.23, p<0.001). Here, we examined the relationship between Aß status and level of endorsement for each item. For 7 of the 15 items, Aß+ was associated with a higher odds of endorsement on CFI for both participant and study partner report and included: ‘Seen a doctor about memory,’ ‘Substantial memory decline,’ ‘Misplacing things,’ ‘Help to remember appointments,’ ‘Disoriented when traveling,’ ‘Trouble with names and words,’ and ‘Relying on written reminders.’ For participant report, but not study partner, Aß+ was associated with greater endorsement on ‘trouble with driving.’ For study partner report but not participant, Aß+ was associated with ‘trouble following the news’ and ‘repeating questions’ (see figure 2). Additionally, Aß+ was associated with ‘decline in work performance’ but as noted above 25% of study partners answered ‘not applicable’ leading to less stable estimates on this item compared to other items on the CFI. All items significantly associated with Aß+ remained significant after an FDR adjustment.

Figure 2. CFI items and odds of endorsement related to Aß

Items presented highest odds of endorsement and Aß+ to lowest for participant and study partner report. * = significant association of item and Aß+



Extending previous findings from the A4 screen data that found for Aß+ participants, participant and study partner CFI scores were higher than for Aß- participants (10), here we found at an item-level, 7 of the 15 CFI items were related to Aß+ for participant and study partner report. Further, the odds of endorsement associated with elevated Aß was numerically higher for participant report compared to study partner report. Items that were related to both study partner and participant report reflected predominantly cognitive changes (e.g., ‘Substantial memory decline,’ ‘Misplacing things’). Functional items (e.g., ‘Hobbies more difficult,’ ‘Reduced social activities,’ ‘Difficulty managing money,’ and ‘Difficulty with appliances’) were much less likely to be endorsed by either a participant or a study partner and were not associated with Aß+.
Additionally, the item ‘seen a doctor about memory’ related to Aß+ for both participant and study partner report. Unlike the other CFI items, this question is based on a specific event rather than an overall subjective experience. The association of ‘seen a doctor about memory’ with Aß is consistent with previous work demonstrating individuals recruited from a memory clinic are more likely Aß+ compared to community-based individuals with subjective memory complaints (9, 16).
Several items were only related to study partner or participant report alone. Specifically, two items were related to Aß+ for study partner report (i.e., ‘Repeating questions’ and ‘trouble following the news’). Interestingly, the fact that study partners report ‘repeating questions,’ but not participants themselves is consistent with clinical observations, in that the participant may not realize they are repeating themselves and would be less likely to endorse this item.
These findings suggest that, even at the preclinical phase, subtle changes in cognitive functioning are recognized by and at the level of awareness of some participants and some study partners. Participant report, while potentially sensitive at the earliest stages of disease, has faced criticism as an outcome in clinical trials as changes in self-awareness (i.e., anosognosia) in patient reported outcomes can occur, particularly by the stage of dementia (17). Thus, there have been concerns as to whether participant report can reliably serve as an indicator of symptom progression for the duration of a trial as individuals move towards clinical impairment. Conversely, it has been unclear what changes if any, a study partner can observe at the preclinical stage when individuals are entirely independent in their daily activities. Importantly, in this study of individuals screening for a prevention trial, study partner report was consistent with participant report, despite the fact that individuals had a global CDR score of 0.
We also examined the impact of demographic features of participants and their study partners on the relationship between Aß status and CFI total score. For participants, while there was a higher level of endorsement for CFI that related to older age and lower education, these demographic variables did not significantly modify the relationship between Aß status and CFI score. Likewise, there was a higher level of endorsement for study partner-reported CFI if the study partner was female or lived with the participant, however these demographic variables did not significantly modify the relationship between Aß status and CFI score. Finally, while age of the study partner did not relate to study partner reported CFI score, there was a nominally significant modifying effect of study partner age on the relationship between Aß status and CFI score. Taken together, the associations between Aß+ and CFI seem to remain even after after adjusting for age, education, sex, and living situation of study partner with participant.
A few limitations to this study a worth noting. Participants in this study were highly educated with limited ethnic and racial diversity, typical for clinical trial populations. Thus, it remains unknown how these findings might generalize to the larger population, as there may be educational and cultural differences in the value of subjective reporting as it relates to risk for AD.
Our findings are in support of subjective report of cognitive functioning to characterize early manifestations of AD among those in an early-treatment trial. While participant report appears to be numerically higher as it relates to Aß status compared study partner report, study partners are also observing similar changes as they relate to Aß status. In the future, examining items longitudinally with tau PET in combination with amyloid PET will also help to better approximate the optimal utility of participant and study partner report as individuals decline or improve during the course of a trial.


Funding: The A4 Study is a secondary prevention trial in preclinical Alzheimer’s disease, aiming to slow cognitive decline associated with brain amyloid accumulation in clinically normal older individuals. The A4 Study is funded by a public-private-philanthropic partnership, including funding from the National Institutes of Health-National Institute on Aging (U19 AG010483, U24AG057437, R01 AG063689), Eli Lilly and Company, Alzheimer’s Association, Accelerating Medicines Partnership, GHR Foundation, an anonymous foundation and additional private donors, with in-kind support from Avid and Cogstate. The companion observational Longitudinal Evaluation of Amyloid Risk and Neurodegeneration (LEARN) Study is funded by the Alzheimer’s Association (LEARN-15-338729) and GHR Foundation. The A4 and LEARN Studies are led by Dr. Reisa Sperling at Brigham and Women’s Hospital, Harvard Medical School and Dr. Paul Aisen at the Alzheimer’s Therapeutic Research Institute (ATRI), University of Southern California. The A4 and LEARN Studies are coordinated by ATRI at the University of Southern California, and the data are made available through the Laboratory for Neuro Imaging at the University of Southern California. The participants screening for the A4 Study provided permission to share their de-identified data in order to advance the quest to find a successful treatment for Alzheimer’s disease.

Acknowledgements: We would like to acknowledge the dedication of all the participants, the site personnel, and all of the partnership team members who continue to make the A4 and LEARN Studies possible. The complete A4 Study Team list is available on:
Conflict of Interest: R. Amariglio has nothing to disclose. J. Grill has nothing to disclose. D. Rentz has nothing to disclose. G Marshall reports personal fees and institutional support from Eisai Inc., institutional support from Eli Lilly and Company, Janssen Alzheimer Immunotherapy, Novartis, and Genentech, and personal fees from Grifols Shared Services North America, Inc, Pfizer outside the submitted work. R. Buckley has nothing to disclose. R. Yaari reports personal fees from Eli Lilly and Company during the conduct of the study. R. Raman reports grants from NIA, grants from Eli Lilly during the conduct of the study, grants from Janssen and grants from Eisai, outside the submitted work. CK. Sun reports grants from NIA and grants from Eli Lilly and Company during the conduct of the study. J. Sims he is an employee and stock holder of Eli Lilly and Company outside the submitted work. M. Donohue reports grants from NIH, grants and personal fees from Eli Lilly and Company during the conduct of the study, personal fees from Roche, personal fees from Biogen, personal fees from Neurotrack, other from Janssen, personal fees from Vivid Genomics outside the submitted work. P. Aisen reports grants from Janssen, grants from NIA, grants from FNIH, grants from Alzheimer’s Association, grants from Eisai, personal fees from Merck, personal fees from Biogen, personal fees from Roche, personal fees from Lundbeck, personal fees from Proclara, personal fees from Immunobrain Checkpoint outside the submitted work. K. Holdrige reports she is an employee and minor stockholder of Eli Lilly and Company. S Sikkes reports grants from Zon-MW OffRoad, grants from EU-JPND, institutional support from Lundbeck, Boehringer, and Toyama outside the submitted work. Dr. Gatchel reports grants from Alzheimer’s Association, grants from NIH/NIA, personal fees from Huron , grants from Merck , outside the submitted work.

Ethical Standards: Study protocols were approved by the Partners Institutional Review Board, and all participants provided written informed consent before undergoing any study procedures.”

Open Access: This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.



1. Jack C, Bennett D, Blennow K, et al. 2018 NIA-AA Research Framework: Toward a biological definition of Alzheimer’s disese. Alzheimers Dement 2018; 14: 535-562.
2. Administration USFD: Alzheimer’s Disease: Developing Drugs for Treatment Guidance for Industry 2018.
3. Amariglio RE, Donohue MC, Marshall GA, et al. Tracking early decline in cognitive function in older individuals at risk for Alzheimer disease dementia: the Alzheimer’s Disease Cooperative Study Cognitive Function Instrument. JAMA Neurol 2015; 72:446-454.
4. Gifford KA, Liu D, Carmona H, et al. Inclusion of an informant yields strong associations between cognitive complaint and longitudinal cognitive outcomes in non-demented elders. J Alzheimers Dis 2015; 43:121-132.
5. Nuño, MM, Gillen D, Grill J. Study partner types and prediction of cognitie performance: implications to preclinical Alzheimer’s trials. Alzheimer’s Research & Therapy 2019; 11:92.
6. Li C, Neugroschl J, Luo X, et al. The utility of the Cognitive Function Instrument (CFI) to detect cognitive decline in non-demented older adults. J Alzheimers Dis 2019; 60: 427-437.
7. Zwan MD, Villemagne VL, Dore V, et al. Subjective Memory Complaints in APOEvarepsilon4 Carriers are Associated with High Amyloid-beta Burden. J Alzheimers Dis 2016; 49:1115-1122.
8. Buckley, R, Ellis, KA, Ames, D, et al. Phenomenological characterization of memory complaints in preclinical and prodromal Alzheimer’s disease. Neuropsychology 2015; 29:571-81.
9. La Joie R, Perrotin A, Egret S, et al. Qualitative and quantitative assessment of self-reported cognitive difficulties in nondemented elders: Association with medical help seeking, cognitive deficits, and beta-amyloid imaging. Alzheimers Dement (Amst) 2016; 5:23-34.
10. Sperling RA, Donohue MC, Raman R, et al. Association of Factors With Elevated Amyloid Burden in Clinically Normal Older Individuals. JAMA Neurol 2020; e pub.
11. Walsh SP, Raman R, Jones KB, et al. ADCS Prevention Instrument Project: the Mail-In Cognitive Function Screening Instrument (MCFSI). Alzheimer Dis Assoc Disord 2006; 20:S170-17811.
12. Morris JC. The Clinical Dementia Rating (CDR): current version and scoring rules. Neurology 1993; 43:2412-2414.
13. Folstein MF, Folstein SE, McHugh PR. «Mini-mental state». A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975; 12:189-198.
14. Wechsler D. WMS-R Wechsler Memory Scale Revised Manual, New York, The Psychological Corporation, Harcourt Brace Jovanovich, Inc, 1987
15. Benjamini, Y., & Hochberg, Y. Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal statistical society: series B (Methodological), 1995; 57: 289-300.
16. Snitz BE, Wang, T, Cloonan YK, et al. Risk of progression from subjective cognitive decline to mild cognitive impairment: The role of study setting. Alz & Dement 2018; 14: 734-742.
17. Frank L, Lenderking WR, Howard K, et al. Patient self-report for evaluating mild cognitive impairment and prodromal Alzheimer’s disease. Alzheimers Res Ther 2011; 3:35.



D.A. Loewenstein, R.E. Curiel Cid, M. Kitaigorodsky, E.A. Crocco, D.D. Zheng, K.L. Gorman


Center for Cognitive Neuroscience and Aging, Department of Psychiatry and Behavioral Sciences, University of Miami Miller School of Medicine, 1695 NW 9th Avenue, Miami, Florida,. U.S.A.

Corresponding Authors: David A. Loewenstein, PhD, ABPP-CN; Director, Center for Cognitive Neuroscience and Aging; Professor of Psychiatry and Behavioral Sciences; Professor of Neurology; University of Miami, 1695 NW 9th Ave, Suite 3202, Miami, FL 33136;; Phone: (305) 355-7016; Fax: (305) 255-9076

J Prev Alz Dis 2021;
Published online February 7, 2021,



Background: Difficulties in inhibition and self-monitoring are early features of incipient Alzheimer’s disease and may manifest as susceptibility to proactive semantic interference. However, due to limitations of traditional memory assessment paradigms, recovery from interference effects following repeated learning opportunities has not been explored.
Objective: This study employed a novel computerized list learning test consisting of repeated learning trials to assess recovery from proactive and retroactive semantic interference.
Design: The design was cross-sectional.
Setting: Participants were recruited from the community as part of a longitudinal study on normal and abnormal aging.
Participants: The sample consisted of 46 cognitively normal individuals and 30 participants with amnestic mild cognitive impairment.
Measurements: Participants were administered the Cognitive Stress Test and traditional neuropsychological measures. Step-wise logistic regression was applied to determine which Cognitive Stress Test measures best discriminated between diagnostic groups. This was followed by receiver operating characteristic analyses.
Results: Cued A3 recall, Cued B3 recall and Cued B2 intrusions were all independent predictors of diagnostic status. The overall predictive utility of the model yielded 75.9% sensitivity, 91.1% specificity, and an overall correct classification rate of 85.1%. When these variables were jointly entered into receiver operating characteristic analyses, the area under the curve was .923 (p<.001).
Conclusions: This novel paradigm’s use of repeated learning trials offers a unique opportunity to assess recovery from proactive and retroactive semantic interference. Participants with mild cognitive impairment exhibited a continued failure to recover from proactive interference that could not be explained by mere learning deficits.

Key words: Proactive semantic interference, retroactive semantic interference, prodromal Alzheimer’s disease, mild cognitive impairment, intrusions.



Hasher and Zacks (1988) first described age-related changes in inhibitory processes that diminished the ability to ignore distracting information (1). This was confirmed in subsequent studies (2-4). Difficulties in inhibitory processes and self-monitoring have also been seen as early features of incipient Alzheimer’s disease (AD; 5-8). Loewenstein and colleagues (2004) posited that learning deficits are related to deficiencies in the semantic network and found that proactive interference of competing to-be-remembered lists of semantically related targets were especially sensitive to the mild cognitive impairment (MCI) stages of AD (6). Curiel et al (2013) employed a novel paradigm (9), the Loewenstein and Acevedo Scales for Semantic Interference and Learning (LASSI-LTM) that required learning a list of 15 target items representing three semantic categories (fruits, musical instruments, and articles of clothing). Maximal learning was facilitated by category cues at both acquisition and recall. Proactive semantic interference (PSI) and the failure to recover from PSI (frPSI) were assessed by having the examinee attempt to learn 15 new targets on List B (representing the identical semantic categories used for List A targets) over two additional learning trials while using these identical category cues during both acquisition and retrieval.
Subsequent studies conducted in independent cohorts in the United States and other countries have found that performance deficits on the LASSI-L were superior to several traditionally used memory tests (e.g., list learning measures, delayed paragraph recall) in distinguishing between cognitively normal older adults and those with preclinical AD or early and late stage MCI. Various studies on the LASSI-L have related these early cognitive changes to biological markers of AD such as in-vivo amyloid imaging (10-12) and neurodegeneration measured by magnetic resonance imaging (MRI; 13-14), functional MRI (15), and fluorodeoxyglucose positron emission tomography (PET/CT; 16). In a majority of these studies, AD pathology was more associated with deficits in frPSI than impairments in initial PSI. Using Receiver Operator Characteristic Curve (ROC) analyses, Matias-Guiu and colleagues found that the LASSI-L was superior to the Free and Cued Selective Reminding Test (FCSRT), in detecting MCI patients with suspected AD (16) and in differentiating both early and late stage MCI individuals from cognitively normal older adults.
It has been proposed that both PSI and frPSI can be assessed in different manners (11). These include the number of correct responses on List B relative to List A or the number of semantic intrusions rendered on List B recall trials. In one recent study, MCI patients that were amyloid positive and had presumptive AD evidenced significantly more intrusion errors than MCI participants who had a clinical history consistent with AD but were amyloid negative, or MCI participants diagnosed with other neurological and neuropsychiatric conditions who were also amyloid negative (11).
The finding that frPSI is particularly sensitive to incipient AD raises an interesting theoretical as well as empirical question. Will deficits in frPSI continue in the presence of additional opportunities to learn two competing semantic word lists? That is, could extending additional opportunities to learn both List A and List B provide deeper insights into initial learning deficits in aMCI participants at risk for AD, as well as their ability to completely recover from PSI deficits over time? An additional question is whether the failure to recover from retroactive interference (frRSI) is an issue in persons with aMCI. These issues have not been addressed by the LASSI-L and other paradigms.
To test the abovementioned potential limitations of this novel assessment paradigm, we employed the Cognitive Stress Test (CST). The CST required learning of 18 targets words, all of which belonged to one of three semantic categories: occupations, household items and types of transportation. Identical category cues were provided during each of the three learning trials as well as during each of the three cued recall trials for each list. This provided a unique opportunity to directly assess the immediate and persistent effects of semantic interference over multiple trials. In addition, we assessed the ability to recover from retroactive semantic interference, which has not been previously examined in aMCI and AD research. We hypothesized that failure to recover from proactive semantic interference would continue to be problematic for individuals with aMCI despite multiple trials that would allow the recovery from these deficits.



Participants were part of an NIH-funded longitudinal study on normal and abnormal aging. All participants provided informed consent for this IRB-approved study. In this investigation, we carefully selected 46 individuals classified as cognitively normal (CN) and 30 participants with amnestic mild cognitive impairment (aMCI). Inclusion and exclusion criteria are as follows.

Cognitively normal group (n=46)

Participants were classified as CN if there were: a) no subjective cognitive complaints made by the participant and/or a collateral informant; b) no evidence of memory or other cognitive decline after an extensive interview with the participant and an informant; c) Global Clinical Dementia Rating (CDR) scale score of 0 (17); and d) all memory (e.g.: Hopkins Verbal Learning Test, Revised (HVLT-R; 18) or delayed paragraph recall from the National Alzheimer’s Coordinating Center Uniform Data Set (NACC UDS; 19) and non-memory measures (e.g., Category Fluency (20), Trails A and B (21), WAIS-IV Block Design subtest (22)) were less than 1.0 standard deviation below normal limits for age, education, and language group.

Amnestic MCI group (n=30)

Participants were classified as aMCI if: (a) they fulfilled Petersen’s criteria (23) for MCI, b) subjective cognitive complaints were reported by the participant and/or collateral informant; c) Global CDR scale score was 0.5; and d) delayed recall was impaired (i.e., 1.5 standard deviations or more below the mean, accounting for age, education, and language of testing) on either the HVLT-R or delayed paragraph recall from the NACC UDS.

Exclusion Criteria for all study groups

Exclusion criteria included significant sensory or motor deficits (e.g., visual or hearing impairment, paralysis) or literacy lower than the 6th grade level on the WRAT-4 (24) evidenced during the clinical evaluation by Drs. Loewenstein or Curiel and judged to preclude completion of the study measures; 2) DSM-5 diagnosis of major depressive disorder, bipolar disorder, current psychotic disorder, substance use disorder or any DSM-5 Axis 1 diagnosis after an extensive interview by the study clinicians using the SCID (25). Individuals with major depressive disorder were excluded from the study given that this condition often results in attention and/or concentration difficulties and psychomotor slowing that may adversely affect test performance on neuropsychological measures. Individuals with major neurocognitive disorder were not included in this sample.

Cognitive Stress Test (CST)

We employed a novel computerized measure called the Cognitive Stress Test (CST) that expands upon our previous work with the widely-studied Loewenstein-Acevedo Scale for Semantic Interference and Learning (LASSI-L), including the computerized version of the LASSI-L which has evidenced high test-retest reliability and discriminative validity (Curiel et al, in press). The CST employs the following: 1) semantic cuing at both acquisition and retrieval of 18 List A targets representing three semantic categories (occupations, household items, or types of transportation) over three initial learning trials, 2) three consecutive presentations of a second list of 18 new targets (List B) representing the same categories as the first list to examine PSI and frPSI, and 3) use of category cues to elicit recall of List A targets to assess retroactive semantic interference (RSI), with an additional learning trial to examine failure to recover from retroactive semantic interference (frRSI). The CST represents an exciting approach to preclinical AD assessment in that it builds upon our previous work and is a fully computer-administered web-based task, which facilitates remote deliverability, reduces the need for a skilled psychometrist, and allows for automatically recording of correct responses, intrusions and other errors.

Statistical Analyses

Statistical analyses were conducted using SPSS Version 26. First, age, gender, education, and language of testing and then global cognitive function were evaluated between diagnostic groups using one-way ANOVAs and Chi-square analyses with Yate’s Correction for Discontinuity. CST cued recall and intrusion scores were compared using ANOVA while adjusting for factors that were distributed differently between diagnostic groups. The alpha cutoff value was adjusted using the Bonferroni correction for multiple comparisons. Step-wise logistic regression models were employed to determine the best independent classification using CST variables. These were followed by a ROC analysis examining significant independent predictors with regards to area explained under the ROC curve.



As depicted in Table 1, there were no statistically significant differences between aMCI and CN groups with regards to mean age, education and language of testing. Participants in the aMCI group had lower mean Mini-Mental State Examination (MMSE) scores (26) and there were more males in the aMCI group than the CN group.
Table 2 indicates that individuals with aMCI had lower scores on all CST trials . After adjusting for baseline differences in MMSE scores and using sex as a covariate, aMCI participants scored lower on all three List A initial learning trials and all three List B trials susceptible to PSI and frPSI. After covariate adjustment, there were no aMCI and CN differences on recall trials susceptible to retroactive semantic interference (RSI) or the ability to recover from RSI (frRSI). Table 3 denotes intrusion errors across the different CST trials. With and without adjustment for covariates, the only measures that differentiated groups were semantic intrusions on List B1 (which measures PSI), List B2 (which measures frPSI) and List B3 (which measures persistent frPSI after repeated learning trials).

Table 1. Demographics by Diagnostic Group

Table 2. CST Cued Recall Scores by Diagnostic Group

*Values survived Bonferroni Correction at 0.05/8=0.00625

Table 3. CST Intrusion Errors by Diagnostic Group

*Values survived Bonferroni Correction at p<.05

We calculated PSI, the initial failure to recover from proactive semantic interference after 1 additional learning trial (frPSI1), and the persistence of proactive semantic interference after 2 additional learning trials (frPSI2). PSI was calculated using the ratio of Cued B1 Recall to Cued A1 Recall. FrPSI1 was calculated using the ratio of Cued B2 Recall to Cued A2 Recall. FrPSI2 was calculated using the ratio of Cued B3 Recall to Cued A3 Recall.
There were no aMCI versus CN differences in the Cued B1/ Cued A1 ratio (F(1.74)= 1,59; p=.211). However, aMCI participants demonstrated more frPSI1 (F(1.74)= 8,25; p=.005) and frPSI2 (F(1.74)=19,45; p<.001). As depicted in Table 4, on the Cued B3 recall trial, which followed two additional learning trials of List B items, CN participants were able to recover so that they could recall an average of 88.6% of the that they recalled during Cued A3 recall. In contrast, participants with aMCI were only able to recover an average of 67.4% of the items that they recalled during Cued A3 recall.
Step-wise logistic regression was employed to determine which of the initial learning and PSI measures best discriminated between aMCI and CN groups. As indicated in Table 5, Cued A3 recall, Cued B3 recall and Cued B2 intrusions were predictors of diagnostic status. The overall predictive utility of the model yielded 75.9% sensitivity and 91.1% specificity, and overall correct classification rate of 85.1%. When these variables were jointly entered into ROC analyses, the area under the ROC curve was .923 (p<.001).

Table 4. Proactive Interference and Failure to Recover from Proactive Interference Ratios

Table 5. Step-Wise Logistic Regression Using Measures of Initial Learning and Susceptibility to Proactive Interference to Distinguish Amnestic Mild Cognitive Impairment and Cognitively Normal Groups

*Model at step 3 yielded 75.9%. sensitivity and 91.1% specificity (overall classification 85.1%)


The current investigation used a novel computerized paradigm with semantically competing target word lists, the Cognitive Stress Test, to investigate whether the effects of proactive semantic interference (PSI) and the initial failure to recover from PSI (frPSI) would persist or diminish with additional learning trials. The obtained pattern of results indicated that, despite repeated administrations of the second list, participants with amnestic MCI had a persistent failure to recover from proactive semantic interference (frPSI). This cannot be explained by mere learning deficits alone since proactive semantic interference deficit ratios adjusted for initial learning on the corresponding trial of List A targets. The unique nature of proactive semantic interference deficits was also evidenced by increased intrusion errors, which were produced by aMCI participants on Cued B1, Cued B2 and Cued B3 trials but not on additional trials of List A susceptible to retroactive interference. In fact, no measure of retroactive interference reached statistical significance, which is consistent with the notion that PSI is uniquely related to early cognitive function in older adults with aMCI at risk for AD (17, 27). Previous studies have suggested that PSI effects may be more associated with MCI and early AD than RSI (27-28). In contrast, in 2012 Ricci and colleagues (29) found RSI but lack of PSI effects using the Rey Auditory Verbal learning Test (RAVLT). It should be noted, however, that the RAVLT list-learning task did not specifically elicit semantic interference, which is the focus of the current investigation.
Unlike previous studies, the current investigation incorporated multiple trials of two sets of 18 different targets, each belonging to one of three semantic categories. The current findings suggest that even after repeated learning trials, aMCI participants are not able to overcome the effects of semantic interference. Our finding of a combined area under the ROC curve exceeding .92 for Cued A3 Recall, Cued B3 recall and Cued B2 intrusions indicates that aMCI participants have deficits in initial learning as well as a failure to recover from proactive interference. The latter is evidenced by increasing deficits in recall of List B relative to List A targets over time (percentage of correct responses), as well as intrusion errors on measures susceptible to proactive interference and the failure to recover from proactive interference. This suggests that different measures of failure to recover from proactive semantic interference may have different biological underpinnings. Indeed, using the LASSI-L, which only affords one opportunity to recover from proactive semantic interference, Cued B2 recall was correlated with atrophy in AD prone regions (13-14). In contrast, Loewenstein et al., (2018) showed that it was not Cued B2 recall but Cued B2 semantic intrusions that could differentiate between MCI groups who were amyloid positive versus other MCI groups who were amyloid negative (11), suggesting the potential specificity of intrusion errors as a cognitive breakdown associated with AD brain pathology. Similarly, Sanchez and colleagues (2017) found that among clinically asymptomatic middle-age offspring of AD parents, Cued B2 intrusions were highly related to abnormal limbic connectivity issues on fMRI (15).
Torres et al. (2019) conducted a qualitative analysis on List B intrusion errors and found that the vast majority were incorrect recall of List B targets followed by semantic errors related to the List B target but not explicitly derived from List A (30). This indicates potential disruptions in cortical-limbic difficulty observed by others (13) and suggests that semantic intrusions represent potentially greater deficits in executive inhibitory processes that allow the individual to access source memory and inhibit previously learned responses.
Strengths of the current paradigm include computerized and uniform administration of three learning trials of 18 targets (representing three different categories) to assess maximum learning using cues at both the encoding and retrieval stages. When applied to three additional trials of 18 different targets (representing identical semantic categories), there was a unique opportunity to study proactive interference and failure to recover from proactive interference (as assessed by the ratio of correct recall on List B to correct recall of List A on the same trial) and semantic intrusions. Participants were comprehensively assessed by both clinical and neuropsychological assessment and compared to older adults of equivalent age with similar educational attainment. There did not appear to be any issues with ceiling or floor effects using 18 to-be-remembered targets, which may have been related to adequate category cues provided at acquisition and retrieval. Finally, the CST was not used in diagnostic formulation to avoid potential issues with circularity.
Potential limitations of the study involve relatively modest numbers of participants and lack of longitudinal follow-up. We intend to keep recruiting and following these participants and obtaining both structural MRI as well as amyloid and tau PET. Future work with fMRI may further elucidate the mechanisms underlying the inability of aMCI participants to break free from the effects of semantic interference when provided with additional learning opportunities. Normal controls appear to be able to increasingly recover from proactive semantic interference effects over time, but this does not hold true with individuals with aMCI. Further exploration into this phenomenon has significant theoretical and clinical implications.


Funding: R01AG061106-02 Loewenstein, David, PI; Florida Department of Health Ed and Ethel Moore Grant #8AZ23. The sponsors had no role in the design and conduct of the study; in the collection, analysis, and interpretation of data; in the preparation of the manuscript; or in the review or approval of the manuscript

Conflict of interest: This study was. supported by the National Institute on Aging (NIA). The CST measure was developed by and is intellectual property held by Drs. Loewenstein and Curiel at the University of Miami.

Ethical standards:This study was IRB approved and met all national and international standards for the protection of human subjects.



1. Hasher, L, Zacks, RT. Working memory, comprehension, and aging: A review and a new view. In Bower GH (ed) Psychol Learn Motive 1988;22:193–225.
2. Amieva H, Phillips LH, Della Sala S, et al. Inhibitory functioning in Alzheimer’s disease. Brain 2004;127:949-64.
3. Collette F, Amieva H, Adam S, et al. Comparison of inhibitory functioning in mild Alzheimer’s disease and frontotemporal dementia. Cortex 2007;43(7):866-874.
4. Clapp WC, Gazzaley A. Distinct mechanisms for the impact of distraction and interruption on working memory in aging. Neurobiol Aging 2012;33(1):134-148.
5. Belleville S, Bherer L, Lepage E, et al. Task switching capacities in persons with Alzheimer’s disease and mild cognitive impairment. Neuropsychologia 2008;46(8):2225-2233.
6. Loewenstein DA, Acevedo A, Luis C, et al. Semantic interference deficits and the detection of mild Alzheimer’s disease and mild cognitive impairment without dementia. J Int Neuropsychol Soc 2004;10(1):91-100.
7. Dewar M, Pesallaccia M, Cowan N, et al. Insights into spared memory capacity in amnestic MCI and Alzheimer’s disease via minimal interference. Brain Cogn 2012;78(3):189-199.
8. Aurtenetxe S, García-Pacios J, Del Río D, et al. Interference impacts working memory in mild cognitive impairment. Front Neurosci 2016;10:443.
9. Curiel RE, Crocco E, Acevedo A, et al. A new scale for the evaluation of proactive and retroactive interference in mild cognitive impairment and early Alzheimer’s disease. J Aging Sci 2013;1(1):1-5.
10. Loewenstein DA, Curiel RE, Greig MT, et al. A novel cognitive stress test for the detection of preclinical Alzheimer disease: discriminative properties and relation to amyloid load. Am J Geriatr Psychiatry 2016;24(10):804-813.
11. Loewenstein DA, Curiel RE, DeKosky, S, et al. Utilizing semantic intrusions to identify amyloid positivity in mild cognitive impairment. Neurology 2018;91(10):e976-e984
12. Curiel Cid RE, Crocco EA, Duara R, et al. A novel method of evaluating semantic intrusion errors to distinguish between amyloid positive and negative groups on the Alzheimer’s disease continuum. J Psychiatr Res 2004;124:131-136.
13. Loewenstein, D, Curiel, RE, DeKosky, S, et al. Recovery from proactive semantic interference and MRI volume: A replication and extension study. J Alzheimer’s Dis 2017a;59(1),131-139.
14. Loewenstein, DA, Curiel, RE, Wright, C, et al. Recovery from proactive semantic interference in mild cognitive impairment and normal aging: Relationship to atrophy in brain regions vulnerable to Alzheimer’s disease. J Alzheimer’s Dis 2017b;56(3):1119-1126.
15. Sánchez SM, Abulafia C, Duarte-Abritta B, et al. Failure to recover from proactive semantic interference and abnormal limbic connectivity in asymptomatic, middle-aged offspring of patients with late-onset Alzheimer’s disease. J Alzheimers Dis 2017;60(3):1183-1193.
16. Matias-Guiu JA, Cabrera-Martín MN, Curiel RE, et al. Comparison between FCSRT and LASSI-L to detect early stage Alzheimer’s disease. J Alzheimers Dis 2018;61(1):103-111.
17. Morris, JC. Clinical dementia rating: a reliable and valid diagnostic and staging measure for dementia of the Alzheimer type. Int psychogeriatr 1997;9:173-176.
18. Hogervorst, E, Combrinck, M, Lapuerta, P, et al. The Hopkins Verbal Learning Test and screening for dementia. Dement Geriatr Cogn Disord 2002;13,13–20.
19. Monsell, SE, Dodge, HH, Zhou, XH et al. Results from the NACC Uniform Data Set Neuropsychological Battery Crosswalk Study. Alzheimer Dis Assoc 2016;30,134–139.
20. Malek-Ahmadi, M, Small, BJ, & Raj, A. The diagnostic value of controlled oral word association test-FAS and category fluency in single-domain amnestic mild cognitive impairment. Dement Geriatr Cogn Disord 2011;32,235–240.
21. Reitan, RM. Validity of the Trail Making Test as an indicator of organic brain damage. Percept Mot Skills 1958;8,271-276.
22. Wechsler, D. (2014). Wechsler Adult Intelligence Scale–Fourth Edition (WAIS–IV). 2014. Psychological Corporation, Texas.
23. Petersen RC, Caracciolo B, Brayne C, et al. Mild cognitive impairment: a concept in evolution. J Intern Med 2014;275(3):214-228.
24. Wilkinson, GS, & Robertson, GJ. WRAT 4: Wide Range Achievement Test. 2006. Psychological Assessment Resources, Florida.
25. First MB, Williams JBW, Karg RS, et al. Structured Clinical Interview for DSM-5—Research Version (SCID-5 for DSM-5, Research Version; SCID-5-RV). 2015. American Psychiatric Association, Virginia.
26. Folstein, MF, Folstein, SE, & McHugh, PR. «Mini-mental state». A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res 1975;12(3):189-198.
27. Ebert PL, Anderson ND. Proactive and retroactive interference in young adults, healthy older adults, and older adults with amnestic mild cognitive impairment. J Int Neuropsychol Soc 2009;15(1):83-93.
28. Wilson, KE, Abulafia, CA, Loewenstein, DA, et al. Individual cognitive and depressive traits associated with maternal versus paternal family history of late-onset Alzheimer’s disease: proactive semantic interference versus standard neuropsychological assessments. J Pers Med Psychiatry 2018;11:1-6.
29. Ricci M, Graef S, Blundo C, et al. Using the Rey Auditory Verbal Learning Test (RAVLT) to differentiate Alzheimer’s dementia and behavioural variant fronto-temporal dementia. Clin Neuropsychol 2012;26(6):926-41.
30. Torres VL, Rosselli M, Loewenstein DA, et al. Types of errors on a semantic interference task in mild cognitive impairment and dementia. Neuropsychol 2019;33(5):670-684.



V. Bloniecki1,2, G. Hagman1,3, M. Ryden3, M. Kivipelto1,3,4,5,6


1. Department of Neurobiology, Caring Sciences and Society (NVS), Division of Clinical Geriatrics, Center for Alzheimer Research, Karolinska Institute, Stockholm, Sweden; 2. Dermato-Venereology Clinic, Karolinska University Hospital, Stockholm, Sweden; 3. Theme Aging, Karolinska University Hospital, Stockholm, Sweden;
4. Ageing and Epidemiology (AGE) Research Unit, School of Public Health, Imperial College London, London, UK; 5. Institute of Public Health and Clinical Nutrition, University of Eastern Finland, Kuopio, Finland, Kuopio, Finland; 6. Research and Development Unit, Stockholms Sjukhem, Stockholm, Sweden.

Corresponding Author: Victor Bloniecki, Karolinska Institute, Karolinska Uinversity Hospital, Eugeniavägen 3, SE-17176, Stockholm, Sweden. Tel.: +46 70-726 82 20; Email:

J Prev Alz Dis 2021;
Published online January 18, 2021,



Background: Due to an ageing demographic and rapid increase of cognitive impairment and dementia, combined with potential disease-modifying drugs and other interventions in the pipeline, there is a need for the development of accurate, accessible and efficient cognitive screening instruments, focused on early-stage detection of neurodegenerative disorders.
Objective: In this proof of concept report, we examine the validity of a newly developed digital cognitive test, the Geras Solutions Cognitive Test (GCST) and compare its accuracy against the Montreal Cognitive Assessment (MoCA).
Methods: 106 patients, referred to the memory clinic, Karolinska University Hospital, due to memory complaints were included. All patients were assessed for presence of neurodegenerative disorder in accordance with standard investigative procedures. 66% were diagnosed with subjective cognitive impairment (SCI), 25% with mild cognitive impairment (MCI) and 9% fulfilled criteria for dementia. All patients were administered both MoCA and GSCT. Descriptive statistics and specificity, sensitivity and ROC curves were established for both test.
Results: Mean score differed significantly between all diagnostic subgroups for both GSCT and MoCA (p<0.05). GSCT total test time differed significantly between all diagnostic subgroups (p<0.05). Overall, MoCA showed a sensitivity of 0.88 and specificity of 0.54 at a cut-off of <=26 while GSCT displayed 0.91 and 0.55 in sensitivity and specificity respectively at a cut-off of <=45.
Conclusion: This report suggests that GSCT is a viable cognitive screening instrument for both MCI and dementia.

Key words: Dementia, MCI, cognitive test, MoCA, e-medicine.


Dementia is currently a global driver of health care costs, and with an ageing demographic, the disease burden of neurodegenerative disorders will increase exponentially in the future. The prevalence is estimated to double every two decades, reaching approximately 80 million affected patients worldwide in 2030 (1). In 2016, the global costs associated with dementia were 948 billion US dollars and are currently projected to increase to 2 trillion US dollars by 2030, corresponding to roughly 2% of the world’s total current gross domestic product (GDP) (2, 3)..
Dementia, or major neurocognitive disorder (MCD), is an umbrella term for neurodegenerative disorders typically characterized by memory dysfunction with Alzheimer’s disease (AD) constituting approximately 60% of all cases. Other common forms of dementia include vascular dementia, Lewy-Body dementia and Frontotemporal dementia. Modern diagnostic tools, such as various imaging modalities and cerebrospinal fluid biomarkers (4, 5), have improved our diagnostic accuracy substantially. These methods have also provided key insights into the pathological mechanisms associated with neurodegenerative and contributed to the development of concepts such as mild cognitive impairment (MCI) and “preclinical AD” (6, 7). Preclinical AD is defined by the presence of cerebral amyloid or tau pathology, identified by positron emission tomography (PET) imaging or cerebrospinal fluid (CSF) biomarkers, before the onset of clinical symptoms (8).
Nevertheless, assessment of cognitive functions, the primary clinical outcome of interest, still largely relies on analogue “pen and paper” based tests administered to patients by health care providers (9). Although some regional differences exist, two of the most known and used cognitive tests include the Montreal Cognitive Assessment (MoCA) and the Mini-Mental State Examination (MMSE) (10, 11). Both tests assess various cognitive domains, with some inter-test differences, including for example; orientation, memory, concentration, executive functions, language, and visuospatial abilities (9) with scores ranging from 0 to 30 points. MoCA, as compared to MMSE which is mostly focused on memory deficits, includes assessment of more cognitive domains thus increasing its diagnostic accuracy. Although optimal cut-off points vary somewhat between different studies, a score lower than 26 on MoCA and 24 on MMSE are considered indicative of dementia (12–15). MoCA has in a previous meta-analysis shown to have a sensitivity and specificity of 0.94 and 0.60 respectively, at a cut-off of 26 points (16). This indicates a good ability to detect dementia, but at the cost of a high amount of false positives. MMSE has, in a meta-analysis, demonstrated a sensitivity of 0.85 and specificity of 0.9 (14). However, MMSE has limited value in detecting MCI and prodromal AD patients from healthy controls (17). Albeit, in the setting of cognitive screening tests a trade-off between sensitivity and specificity is necessary and screening instruments should favor sensitivity over specificity.
Given the current scientific consensus that potential future disease-modifying drugs for AD need to be administered early on in the disease continuum, there is a clear need to develop accurate and widely available cognitive screening tests in order to facilitate early diagnosis of MCI patients in the future. In the European Union, there are currently approximately 20 million individuals over the age of 55 with MCI, most of whom have not undergone screening for cognitive impairment (18). A previous study investigating the treatment and diagnostic capacity of six European countries (France, Germany, Italy, Spain, Sweden, United Kingdom) estimated that over 1 million patients would progress from MCI to AD due to capacity constraints within current health care systems if a disease modifying treatment were to be available in 2020 (18). As such, digital cognitive screening instruments are likely to be a part of the diagnostic process in the future, especially when considering the advancement of digitalized health care in multiple facets of modern medicine (19).
Cognitive assessment instruments are available in different settings including clinic based and at home testing (20, 21). Current cognitive evaluation methods include both pen-and-paper screening tools, which is the conventional method administrated by a clinical neuropsychologist, and computerized cognitive tests (20, 21). Increasing advances in technology has led clinical trials to move away from the conventional methods and adopt validated digital cognitive tools that are sensitive to capturing cognitive changes in early prevention stages (20, 22). Computerized cognitive assessment tools offer several benefits over the traditional instruments, enabling recording of accuracy and speed of response precisely, minimizing floor and ceiling effects and eliminating the examiner bias by offering a standardized format (20–22). Computerized cognitive assessments may also generate potential time and cost savings as the test can be administrated by the patient or other healthcare professionals than neuropsychologist, as long as appropriate professional will be responsible for the test interpretation and diagnosis (20, 22). Thus, unmonitored digital tools provide practical advantages of reduced need for trained professionals, self-administration, automated test scoring and reporting and ease of repeat adjustments, which enable administration for large-scale screening (22, 23). On the other hand, cognitive assessment tools are typically administrated to elderly population who might lack familiarity with digital tools, which can negatively affect their performance (22, 24). However, the attitude and perception of patients using a computerized cognitive assessment have been investigated in the elderly population, and individuals expressed a growing acceptance of using computerized cognitive assessments and rated them as understandable, easy to use and more acceptable than pen and paper tests (20, 22). They also perceived them as having the potential to improve patient care quality and the relationship between the patient and clinician when human intervention is involved (20).
Currently, there are a number of computerized screening instruments available, and they are either a digital version of the existing standardized tests or new computerized tests and batteries for cognitive function assessment (25). The pen-and-paper version of the MoCA test was recently transformed to an electronic version (eMoCA) (24). eMoCA was tested on a group of adults to compare its performance to MoCA, and most of the subjects performed comparably (24). For the detection of MCI, eMoCA (24, 25) and CogState (26) showed promising psychometric properties (25). Computer test of Inoue (27), CogState (26) and CANS-MCI (28) showed a good sensitivity in detecting AD (25). Unlike the other computerized cognitive screening tools, Geras Solutions is a comprehensive tool that provides, besides the cognitive test, a medical history questionnaire that is administrated by the patient, and a symptom survey that is administrated by the patient’s relatives. Thus, it has the potential to save more time and cost compared to the other digital assessment instruments by providing a more complete clinical evaluation.
The primary objective of this study is to investigate the accuracy and validity of a newly developed digital cognitive test (Geras Solutions Cognitive Test [GSCT]). The GSCT is a self-administered cognitive screening test provided by Geras Solutions predominantly based on MoCA. In this study, we intend to investigate the validity of GSCT, including psychometric properties, agreement with MoCA and diagnostic accuracy by establishing sensitivity, specificity, receiver operating characteristics (ROC), area under the curve values (AUC) and optimal cut-off levels, as well as compare performance with MoCA.


Materials and methods

Geras Solutions cognitive test

The GSCT, is a newly developed digital screening tool for cognitive impairment and is included in the Geras Solutions APP (GSA). Development of the screening tool was done in collaboration with the research and clinical team at Theme Aging, Karolinska University Hospital, Solna memory clinic and Karolinska Institutet. GSCT is developed on existing cognitive assessment methods (MoCA and MMSE) and includes additional proprietary tests developed at the memory clinic, Karolinska University Hospital Stockholm, Sweden. The test is suitable for digital administration through devices supporting iOS and Android.
The test is composed of 16 different items assessing various aspects of cognition, developed in order to screen for cognitive deterioration in the setting of dementia and to ensure suitability for administration via mobile devices. The GSCT is scored between 0-59 points in total and has six main subdomains including; memory (0-10 points), visuospatial abilities (0-11 points), executive functions (0-13 points), working memory (0-19 points), language (0-1 point) and orientation (0-5 points). Additionally, the time needed to complete the individual tasks is registered and presented as total test time and subdomain test time. The GSCT is automatically scored using a computer algorithm and results are presented as the total score as well as subdomain scores. A detailed description of the GSCT test items and scoring is provided in supplement 1.


The included study population consisted of 106 patients referred to the memory clinic at Karolinska University Hospital, Solna, predominantly by primary care practitioners due to memory complaints and suspicion of cognitive decline. All patients referred to the clinic between January 2019 and January 2020 were asked to participate in the study. No exclusion criteria were established a priori. If a patient fulfilled the criteria for inclusion (i.e. referred for investigation of suspect dementia at the memory clinic and provided informed consent) they were included in the study. A total of 106 patients accepted participation in the study. Five patients did not complete GSCT (two with MCI, two with subjective cognitive impatient [SCI] and one with dementia) and three patients displayed test scores with evident irregularities (one with MCI, one with SCI and one with dementia) and were excluded from the final analysis, thus leaving 98 complete cases. Irregularities included two patients whom started the test multiple times and one patient with a congenital cognitive deficiency resulting in test scores below 2 SD from the mean on both MoCA and GSCT.
All patients included in the study underwent the standard investigative procedure for dementia assessment as conducted at Karolinska University Hospital Memory Clinic. The investigative process is completed in its entirety in one week and includes; brain imaging, lumbar punctures for analysis of CSF biomarkers and neuropsychological assessment including administration of different cognitive rating scales, including MoCA. Patients received a dementia or MCI diagnosis according to the ICD-10 classification and the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-V) criteria were used as clinical support (29). If no evidence of neurodegeneration was observed patients were provided with an SCI diagnosis based on the ICD-10 classification (30). Final diagnosis was determined by specialist in geriatric medicine. In parallel, patients who accepted inclusion in the study completed the self-administered GSCT during the investigative process. GSCT was, in all cases, administered after MoCA, but not during the same day. Patients were provided a tablet and conducted GSCT alone with a health care provider adjacent if any technical difficulties would arise. The GSCT is a self-administered and all test instructions are provided by the platform. The test is intended to be performed in a home environment without any assistance. Information regarding patients GSCT scores, MoCA scores, age, gender and final diagnosis were collected for statistical analysis. All included patients provided informed consent, and the study was approved by the Regional Ethics Committee of Karolinska Institute, Stockholm, Sweden. Registration number: 2018/998-31/1.
The mean age for the whole included population (n=98) was 58 years. 5 patients were below 50 years of age (5%), 58 patients were between 50 and 60 years of age (59%), 34 patients were between 61 and 70 years of age (35%) and one patient over 70 years (1%) Altogether, 67% (n=65) of the patients were assessed without any signs of neurodegenerative disorder and diagnosed with SCI. 24% were diagnosed with MCI, and 9% received a dementia diagnosis. The dementia group consisted of 8 patients with AD and 1 patient with vascular dementia.


All statistical analyses were done using Statistica software (version 13). Baseline descriptive characteristics were calculated and are provided in Table 1. The rating scales (GSCT and MoCA) were treated as both continuous and dichotomous variables when identifying optimal cut-off levels based on sensitivity and specificity analysis. Both parametric and non-parametric tests were used for the analysis to validate findings and are reported if discrepancies were seen. Agreement between test measures were analyzed using standardized concordance correlation coefficient and analysis of Bland-Altman plot. Association between GSCT and MoCA was assessed using Pearson correlation. The internal consistency of GSCT was analyzed using Cronbach’s alpha index.
ANOVA was used to assess the differences in cognitive test scores categorized by diagnostic subgroups. Post-hoc analysis was conducted using Fisher’s Least Significant Difference (LSD) method. Logistic regression of total test scores was done in order to compare odds ratio between the tests.
Validation of GSCT total score against MoCA required the following to be established or calculated: ROC curves, the area under the curve (AUC) values with 95 % confidence intervals and sensitivity/specificity levels. Analyses were performed to estimate optimal cutoff values based on the best-compiled outcome from a range of sensitivity and specificity levels when testing the continuous scale against a dichotomous test of reference (SCI vs dementia and SCI vs MCI). Adjustment for multiple comparisons was done using the FDR-method. The presented p-values are adjusted values with the FDR-method. An adjusted p-level of <0.05 was defined as statistically significant.



Baseline data, psychometric properties and normative data

Baseline patient characteristics, including cognitive test scores, are provided in Table 1. The mean score for GSCT was 45 points in the SCI group, 36 points in the MCI group and 28 points in patients with dementia.

Table 1. Descriptive statistics

Descriptive data and test scores. Values are shown as means, standard deviations and minimum/maximum; A. p<0.05 compared to SCI. B. p<0.05 compared to MCI. C. p<0.05 compared to dementia; SCI = Subjective cognitive impairment. MCI = Mild cognitive impairment. GSCT = Geras Solutions cognitive test. MoCA = Montreal Cognitive Assessment.

Figure 1. Bland-Altman plot of standardized test scores

X-axis= mean of MoCA and GSCT. Y-axis = Difference in MoCA and GSCT

The correlation between GSCT and MoCA was (r(96) = 0.82, p <0.01). Standardized concordance correlation coefficient between GSCT and MoCA was 0.82, indicating a high level of agreement. Agreement between the GSCT and MoCA was also analyzed using a Bland-Altman plot with standardized values showing that 97 % of data points lie within ±2SD of the mean difference, see figure 1. Estimation of the internal consistency of GSCT showed a standardized Cronbach’s alpha index of α = 0.87.
Age was not significantly correlated with GSCT scores (r =-0.16, p=0.1). Diagnostic subgroup was significantly associated with age (F(2, 95) = 4,8 = 0.02), with post hoc test showing a significant difference between dementia and SCI patients (mean 63 vs 57 years, p=0.01) but not between SCI and MCI (mean 57 vs 60 years, p=0.08) or MCI and dementia patients (mean 60 vs 63 years, p=0.2). No differences in GSCT scores were observed depending on gender (t (96) =-0.3, p= 0.74) with males having a mean score of 41 points and females 40.4. Finally, both age, gender and education were included in an ANCOVA showing that education (F(1, 93) = 5.4, p= 0.03) was significantly associated with GSCT scores in contrast to age (F(1, 93) = 2.9, p = 0.1) and gender (F(1, 93) = 0.74, p = 0.4). Patients with more than 12 years of education showed higher mean test scores as compared to patients with 12 years or less (mean 42.2 vs 37.6 points, p = 0.05). GSCT total test time differed significantly depending on diagnostic subgroup (F(2, 95) = 36.4, p < 0.01) (Figure 2). Post-hoc tests showed that the differences in mean scores were significant between all three subgroups with SCI patients showing a mean test time of 1057 seconds compared to 1296 and 2065 seconds for MCI and dementia patients respectively (SCI vs MCI, p < 0.01) (SCI vs dementia, p < 0.01) (MCI vs dementia, p < 0.01).

Figure 2. Differences in GSCT test time depending on diagnosis

Mean GSCT total test time and a 95% confidence interval for patients with SCI, MCI and Dementia. p<0.05 between all subgroups.


Between-group differences in GSCT and MoCA

Average GSCT scores differed significantly depending on diagnostic subgroup (F(2, 95) = 20.3, p < 0.01). Post-hoc tests showed that the differences in mean scores were significant between all three subgroups (SCI vs MCI, p < 0.01) (SCI vs dementia, p < 0.01) (MCI vs dementia, p = 0.02).
Mean MoCA scores were also significantly different depending on diagnosis (F(2, 95) = 29.5, p < 0.01) and the mean scores were significantly different for all three subgroups (SCI vs MCI, p < 0.01) (SCI vs dementia, p < 0.01) (MCI vs dementia, p < 0.01) (Table 1).

Box plots for test scores for both GSCT and MoCA categorized by diagnosis can be seen in Figure 3. Odds ratios were calculated showing a one unit increase on the GSCT increased the odds of being healthy by 1.15 (CI 95% 1.07 – 1.22) while MoCA was associated with a 1.47 increase in odds (CI 95% 1.22-1.76).

Figure 3. Boxplots showing differences in test scores depending on diagnosis

Median GSCT and MoCA scores are represented by small squares. Larger squares represent interquartile range while whiskers show non-outlier range.


Accuracy and comparison with MoCA

GSCT showed very good to excellent discriminative properties at a wide range of cut-off values. When including all patients, thus coding both MCI and dementia patients into a binary classification of healthy/cognitively impaired, GSCT total score displayed an AUC value of 0.80 with 95% CI [0.70-0.90], whereas MoCA showed an AUC value of 0.80 with CI [0.70-0.90]. MoCA showed a sensitivity of 0.88 and specificity of 0.54 at a cut-off of <=26 while GSCT total score displayed 0.91 and 0.55 in sensitivity and specificity respectively at a cut-off of <=45. Figure 4 shows respective AUC curves and Table 2 presents the respective summary statistics.

Figure 4. Comparison of ROC curves between cognitive tests

Receiver operating characteristics curves for GSCT and MoCA in; top left SCI vs (MCI + dementia); Top right SCI vs MCI. Bottom left SCI vs Dementia.

When assessing the accuracy in discriminating between SCI and MCI patients GSCT showed an AUC value of 0.74 with 95% CI [0.62-0.85] whereas MoCA showed an AUC value of 0.74 with 95% CI [0.61-0.85]. Sensitivity and specificity at a cut-off level of <=45 was 0.88 and 0.55, respectively for GSCT total score. Whereas MoCA, at the traditional cut-off of <=26, displayed a sensitivity of 0.83 and specificity of 0.54 (Table 2). Both tests were excellent at discriminating dementia patients from SCI. GSCT showed an AUC score of 0.96 with 95% CI [0.92-0.1] while MoCA had an AUC score of 0.98 with 95% CI [0.95-0.1]. At the traditional MoCA cut-off of <= 26, sensitivity and specificity scores were 1 and 0.54, respectively whereas GSCT using a cut-off of <=35.5 showed a sensitivity of 1 and specificity of 0.9. As seen in Figure 5, both tests show good capabilities in discriminating between different diagnostic subgroups in this material, although some overlap between MCI and SCI patients existed for both tests. GSCT was marginally better at discriminating MCI from SCI patients as compared to MoCA. No patients with dementia scored within the normal range for either test.

Figure 5. Scatterplot of cognitive test scores depending on diagnosis

Scatter plot of GSCT and MoCA categorized by diagnosis. Marked lines represent cut-off points.

Table 2. Summary of accuracy for both tests

Summary statistics ROC



In this study, we present the first results on a newly developed digital cognitive test provided by Geras Solutions. GSCT displayed good agreement with MoCA based on concordance correlation analysis and Bland-Altman plot indicating that both tests measure similar cognitive domains. Additionally, normative data regarding the influence of age, gender and education was analyzed showing that education, but not age and gender, affected test scores. Individuals with more than 12 years of education had higher mean GSCT scores as compared to individuals with 12 years or less of education providing valuable information regarding scoring analysis in different demographic groups. GSCT showed equally good discriminative properties compared to the MoCA test. Both tests were excellent at discriminating dementia patients from SCI patients with a sensitivity of 1 for both GSCT and MoCA while showing a specificity of 0.9 and 0.56, respectively. This result is similar to the differential capabilities of other digital cognitive test showing sensitivity and specificity scores ranging from 0.85-1 and 0.81-1 respectively (31–33). Both tests also showed similar capabilities when discriminating between SCI and MCI patients with AUC scores of 0.74. GSCT was in this study slightly better in correctly identifying cognitive deterioration in MCI patients with a sensitivity of 0.88 compared to 0.83 for MoCA while both tests showed similar specificity of 0.55 and 0.54 receptively. The GSCT showed somewhat better sensitivity in detecting MCI patients compared to other digital screening tools, such as CogState, which previously reported sensitivity scores ranging between 0.63 and 0.84, albeit those test demonstrated higher specificity (31, 33, 34). Since GSCT is intended as a screening tool used early in the diagnostic process we believe that focus on high sensitivity is of more importance and must come at the cost of lower specificity.
Both tests demonstrated significant differences in mean test scores between all diagnostic subgroups. Additionally, the total GSCT time was also significantly different between all subgroups providing further valuable clinical information as compared to current paper and pen based cognitive screening instruments. GSCT showed very good internal consistency (α = 0.87). Based on this study, we suggest a cut-off level of <=45 for detection of MCI while values <=35.5 indicate manifest dementia.

Overall, GSCT performed at least as well as compared to currently available screening tools for dementia disorders (MoCA) while simultaneously providing several advantages. First, the test is administered via a digital device, thus eliminating the time-consuming need for testing provided by health care practitioners while also increasing the availability of cognitive screening. Given the earlier reported estimated increase in dementia prevalence combined with possible disease-modifying drugs, there is an urgent need for increased accessibility. Additionally, the digital set up of the test eliminates administration bias from health care providers and creates a more homogenous diagnostic tool. Albeit, future studies are needed to test the device in a setting without health care providers nearby. Furthermore, the possibility to register total and domain-specific test time may provide valuable clinical information potentially increasing the diagnostic capabilities, a hypothesis needing further testing in future research. Due to current trends, the development of an effective and accurate digital screening tool for cognitive impairment is of utter importance. Given a sufficiently accurate test, patients scoring in the normal range would not need to undergo further examination in the hospital setting. Instead, this digital screening instrument could identify the proper individuals in need of expanded testing e.g. MRI, CSF analysis and detailed neuropsychological testing, thus saving resources for the health care system and allocating interventions for those in need.


In this initial study we were not able to include healthy subjects. Instead, SCI patients were used as “healthy controls”. Although these patients have a self-reported presence of cognitive dysfunction, no objective findings for the presence of an ongoing neurodegenerative process could be identified. Future studies should include healthy patients without any subjective symptoms. Additionally, future larger normative studies are required to investigate how factors such as age, gender and education affect GSCT performance in order to increase validity and diagnostic accuracy. Another limitation of the test is the lack of information regarding test-retest reliability. In this preliminary trial, we were unable to obtain longitudinal data thus hindering such analysis. Future studies must include longitudinal measurements in order to determine the test-retest reliability of GSCT.
Another limitation of this study is the small sample size, especially in the MCI and dementia subgroups. These findings should be interpreted with caution and future studies, including more patients with MCI and dementia disorders, are necessary to improve the accuracy of the test. Albeit the low sample size increases the risk of type 2 errors, we found significant differences for all groups in mean GSCT scores, further supporting the robustness of the findings. Continuous collection of data from new individuals will improve test performance and provide normative information. Another limitation is the fact that patients were administered GSCT during the same week as MoCA, which could potentially generate practice effects. Furthermore, all testing in the study was conducted in Swedish and all included patients were living in close proximity to Stockholm, Sweden. Thus, there may be a potential bias in the selection of the study population and future studies should investigate whether GSCT scores are affected by regional differences as well as examine the suitability of different language versions in order to improve accessibility.



Overall, the Geras Solutions Cognitive Test performed very well with diagnostic capabilities equal to MoCA when tested on this study population.
This report suggests that GSCT could be a viable cognitive screening instrument for both MCI and dementia. Continued testing and the collection of normative data and test-retest reliability analysis is needed to improve the validity and diagnostic accuracy of the test. Additionally, future studies should explore the diagnostic value of total test time as well as item specific test time.

Funding: Theme Aging Research Unit had research collaboration with Geras Solutions during the study and a grant from Geras Solutions was provided to support conducting the study. The study was conducted independently at the memory clinic, Karolinska University Hospital, and the funding organizations had not been involved in analyses and writing. Other research support: Joint Program of Neurodegenerative Disorders, IMI, Knut and Alice Wallenberg Foundation, Center for Innovative Medicine (CIMED) Stiftelsen Stockholms sjukhem, Konung Gustaf V:s och Drottning Victorias Frimurarstiftelse, Alzheimer’s Research and Prevention Foundation, Alzheimerfonden, Region Stockholm (ALF and NSV grants). Advisory board (MK): Geras Solutions, Combinostics, Roche. GH: Advisory board: Gears Solutions. VB: Consultant for Geras Solutions.

Conflict of Interest: MK: Advisory board: Combinostics, Roche; GH: Advisory board: Gears Solutions; VB: Consultant for Geras Solutions.

Ethical Standards: The study was approved by the Regional Ethics Committee of Karolinska Institute, Stockholm, Sweden. Registration number: 2018/998-31/1.

Open Access: This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits use, duplication, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.



1. Prince M, Wimo A, Guerchet M et al (2015) World alzheimer report. The global impact of dementia. An analysis of prevalance, incidence, cost and trends. Alzheimer’s Disease International, London.
2. Wimo A, Guerchet M, Ali GC, Wu YT, Prina AM, Winblad B, et al. The worldwide costs of dementia 2015 and comparisons with 2010. Alzheimer’s Dement. 2017;13(1):1–7.
3. Xu J, Zhang Y, Qiu C, Cheng F. Global and regional economic costs of dementia: a systematic review. Lancet. 2017;390:S47.
4. Blennow K, Zetterberg H. Biomarkers for Alzheimer’s disease: current status and prospects for the future. J Intern Med. 2018;284(6):643–63.
5. Jack CR, Bennett DA, Blennow K, Carrillo MC, Dunn B, Haeberlein SB, et al. 2018 National Institute on Aging-Alzheimer’s Association (NIA-AA) Research Framework NIA-AA Research Framework: Toward a biological definition of Alzheimer’s disease. Alzheimer’s Dement. 2018;14(1):535–62.
6. Lopez OL. Mild cognitive impairment. Continuum (Minneap Minn). 2013;19(2 Dementia):411–24.
7. Sperling RA, Aisen PS, Beckett LA, Bennett DA, Craft S, Fagan AM, et al. Toward defining the preclinical stages of Alzheimer’s disease: Recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimer’s Dement. 2011;7(3):280–92.
8. Dubois B, Hampel H, Feldman HH, Scheltens P, Aisen P, Andrieu S, et al. Preclinical Alzheimer’s disease: Definition, natural history, and diagnostic criteria. Vol. 12, Alzheimer’s and Dementia. Elsevier Inc.; 2016. p. 292–323.
9. Sheehan B. Assessment scales in dementia. Ther Adv Neurol Disord. 2012;5(6):349–58.
10. Nasreddine ZS, Phillips NA, Bédirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: A Brief Screening Tool For Mild Cognitive Impairment. J Am Geriatr Soc. 2005;53(4):695–9.
11. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975;12(3):189–98.
12. Carson N, Leach L, Murphy KJ. A re-examination of Montreal Cognitive Assessment (MoCA) cutoff scores. Int J Geriatr Psychiatry. 2018;33(2):379–88.
13. Milani SA, Marsiske M, Cottler LB, Chen X, Striley CW. Optimal cutoffs for the Montreal Cognitive Assessment vary by race and ethnicity. Alzheimer’s Dement Diagnosis, Assess Dis Monit. 2018;10:773–81.
14. Creavin ST, Wisniewski S, Noel-Storr AH, Trevelyan CM, Hampton T, Rayment D, et al. Mini-Mental State Examination (MMSE) for the detection of dementia in clinically unevaluated people aged 65 and over in community and primary care populations. Cochrane Database Syst Rev. 2016;2016(1):CD011145.
15. O’Bryant SE, Humphreys JD, Smith GE, Ivnik RJ, Graff-Radford NR, Petersen RC, et al. Detecting dementia with the mini-mental state examination in highly educated individuals. Arch Neurol. 2008;65(7):963–7.
16. Davis DH, Creavin ST, Yip JL, Noel-Storr AH, Brayne C, Cullum S. Montreal Cognitive Assessment for the diagnosis of Alzheimer’s disease and other dementias. Cochrane Database Syst Rev. 2015;
17. Mitchell AJ. A meta-analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res. 2009;43(4):411–31.
18. Hlavka JP, Mattke S, Liu JL. Assessing the Preparedness of the Health Care System Infrastructure in Six European Countries for an Alzheimer’s Treatment. Rand Heal Q. 2019;8(3):2.
19. Meskó B, Drobni Z, Bényei É, Gergely B, Győrffy Z. Digital health is a cultural transformation of traditional healthcare. mHealth. 2017;3:38–38.
20. Robillard JM, Lai JA, Wu JM, Feng TL, Hayden S. Patient perspectives of the experience of a computerized cognitive assessment in a clinical setting. Alzheimer’s Dement Transl Res Clin Interv. 2018;
21. Kim H, Hsiao CP, Do EYL. Home-based computerized cognitive assessment tool for dementia screening. J Ambient Intell Smart Environ. 2012;
22. Wild K, Howieson D, Webbe F, Seelye A, Kaye J. Status of computerized cognitive testing in aging: A systematic review. Alzheimer’s and Dementia. 2008.
23. Morrison RL, Pei H, Novak G, Kaufer DI, Welsh-Bohmer KA, Ruhmel S, et al. A computerized, self-administered test of verbal episodic memory in elderly patients with mild cognitive impairment and healthy participants: A randomized, crossover, validation study. Alzheimer’s Dement Diagnosis, Assess Dis Monit. 2018;
24. Berg JL, Durant J, Léger GC, Cummings JL, Nasreddine Z, Miller JB. Comparing the Electronic and Standard Versions of the Montreal Cognitive Assessment in an Outpatient Memory Disorders Clinic: A Validation Study. J Alzheimer’s Dis. 2018;
25. De Roeck EE, De Deyn PP, Dierckx E, Engelborghs S. Brief cognitive screening instruments for early detection of Alzheimer’s disease: A systematic review. Alzheimer’s Research and Therapy. 2019.
26. Maruff P, Lim YY, Darby D, Ellis KA, Pietrzak RH, Snyder PJ, et al. Clinical utility of the cogstate brief battery in identifying cognitive impairment in mild cognitive impairment and Alzheimer’s disease. BMC Psychol. 2013;
27. Inoue M, Jinbo D, Nakamura Y, Taniguchi M, Urakami K. Development and evaluation of a computerized test battery for Alzheimer’s disease screening in community-based settings. Am J Alzheimers Dis Other Demen. 2009;
28. Memória CM, Yassuda MS, Nakano EY, Forlenza O V. Contributions of the Computer-Administered Neuropsychological Screen for Mild Cognitive Impairment (CANS-MCI) for the diagnosis of MCI in Brazil. Int Psychogeriatrics. 2014;
29. Diagnostic and statistical manual of mental disorders : DSM-5 [Internet]. Fifth edition. Arlington, VA : American Psychiatric Publishing, [2013] ©2013;
30. The ICD-10 Classification of Mental and Behavioural Disorders Clinical descriptions and diagnostic guidelines World Health Organization.
31. Scharre DW, Chang SI, Nagaraja HN, Vrettos NE, Bornstein RA. Digitally translated Self-Administered Gerocognitive Examination (eSAGE): Relationship with its validated paper version, neuropsychological evaluations, and clinical assessments. Alzheimer’s Res Ther. 2017;9(1).
32. Onoda K, Yamaguchi S. Revision of the cognitive assessment for dementia, iPad version (CADi2). PLoS One. 2014;9(10).
33. Possin KL, Moskowitz T, Erlhoff SJ, Rogers KM, Johnson ET, Steele NZR, et al. The Brain Health Assessment for Detecting and Diagnosing Neurocognitive Disorders. J Am Geriatr Soc. 2018;66(1):150–6.
34. de Jager CA, Schrijnemaekers ACMC, Honey TEM, Budge MM. Detection of MCI in the clinic: Evaluation of the sensitivity and specificity of a computerised test battery, the Hopkins Verbal Learning Test and the MMSE. Age Ageing. 2009;38(4):455–60.


R.E. Curiel Cid1, E.A. Crocco1, M. Kitaigorodsky1, L. Beaufils2, P.A. Peña2, G. Grau1, U. Visser2, D.A. Loewenstein1

1. Center for Cognitive Neuroscience and Aging, Department of Psychiatry and Behavioral Sciences, University of Miami Miller School of Medicine, 1695 NW 9th Avenue, Miami, Florida, 33136. U.S.A; 2. Department of Computer Science, University of Miami, 1365 Memorial Drive, Coral Gables, Florida 33146, U.S.A.

Corresponding Author: Rosie E. Curiel, Psy.D., Associate Professor of Psychiatry & Behavioral Sciences, University of Miami Miller School of Medicine, 1695 NW 9th Avenue, Suite 3202, Miami, FL 33136.

J Prev Alz Dis 2021;
Published online January 19, 2021,



BACKGROUND: The Loewenstein Acevedo Scales of Semantic Interference and Learning (LASSI-L) is a novel and increasingly employed instrument that has outperformed widely used cognitive measures as an early correlate of elevated brain amyloid and neurodegeneration in prodromal Alzheimer’s Disease (AD). The LASSI-L has distinguished those with amnestic mild cognitive impairment (aMCI) and high amyloid load from aMCI attributable to other non-AD conditions. The authors designed and implemented a web-based brief computerized version of the instrument, the LASSI-BC, to improve standardized administration, facilitate scoring accuracy, real-time data entry, and increase the accessibility of the measure.
Objective: The psychometric properties and clinical utility of the brief computerized version of the LASSI-L was evaluated, together with its ability to differentiate older adults who are cognitively normal (CN) from those with amnestic Mild Cognitive Impairment (aMCI).
Methods: After undergoing a comprehensive uniform clinical and neuropsychological evaluation using traditional measures, older adults were classified as cognitively normal or diagnosed with aMCI. All participants were administered the LASSI-BC, a computerized version of the LASSI-L. Test-retest and discriminant validity was assessed for each LASSI-BC subscale.
Results: LASSI-BC subscales demonstrated high test-retest reliability, and discriminant validity was attained.
Conclusions: The LASSI-BC, a brief computerized version of the LASSI-L is a valid and useful cognitive tool for the detection of aMCI among older adults.

Key words: Computerized test, mild cognitive impairment, Alzheimer’s disease, semantic intrusion errors, semantic interference, clinical trials.



Alzheimer’s disease (AD) is a devastating condition that is expected to significantly impact the rapidly aging population. Important advancements have been made to identify novel candidate biomarkers of AD, and a research framework to stage the disease from its preclinical stage onward has been proposed, with the aim of establishing a biological definition of the disease (1). Despite these formidable advances, neuropsychological assessment remains an essential component of the evaluative process because cognitive impairment is a fundamental defining symptom of AD that emerges early, at a certain point in the transition from the preclinical to clinically symptomatic stages of the disease. Further, cognitive changes are used to detect and track disease progression over time and a measurable change in cognitive ability represents a potentially meaningful clinical outcome (2). Thus, the identification of cognitive markers that are sensitive to detecting early disease states and converge with biological markers of AD pathology, have become increasingly necessary in terms of identifying individuals at risk, monitoring disease progression, and ascertaining treatment efficacy (3).
Traditional paper-and-pencil cognitive measures employed for the detection of AD-related Mild Cognitive Impairment (MCI) are often insensitive to detecting subtle cognitive changes that occur during preclinical or prodromal disease states (5, 6). There is a developing body of literature, however, that cognitive stress paradigms can measure subtle deficiencies that are highly implicated in early AD disease states among older adults. One such paradigm that measures semantic interference in memory, the Loewenstein-Acevedo Scales for Semantic Interference and Learning (LASSI-L), was sensitive enough to differentiate older adults who are cognitively unimpaired from those with subjective memory complaints, and early amnestic MCI (7, 8). On this memory measure, proactive semantic interference (PSI) deficits and particularly, the inability to recover from PSI (frPSI) was also highly associated with brain amyloid load in older adults with otherwise normal performance on a traditional battery of cognitive tests (9). The LASSI-L has outperformed other widely used memory measures in detecting prodromal AD in both English and Spanish (10, 11), and has been found to be useful in different cultural/language groups (7, 11, 12). In addition to measuring the total number of correct targets recalled on individual LASSI-L subscales, there is evidence that semantic intrusion errors may have specific utility in the assessment of prodromal AD. Loewenstein and colleagues (4) found that semantic intrusion errors sensitive to PSI and frPSI on the LASSI-L could differentiate amyloid positive aMCI groups from amyloid negative aMCI groups with non-AD diagnoses.
While it is recognized that intrusion errors represent early manifestations of neurodegenerative brain disease, a potential limitation of previous approaches is that the number of intrusion errors are often highly dependent on an individual’s total responses on a particular trial. Thus, even a seemingly modest number of intrusion errors may actually represent an at-risk cognitive profile, depending on the total number of responses that are correct. For example, an individual may make a minimal number of intrusion errors on a given trial, which may appear to be clinically insignificant. However, if the number of total responses is low, even a modest number of intrusion errors may indicate impaired inhibitory processes and underlying brain pathology. As a result, we recently developed a novel method to evaluate semantic intrusion errors utilizing the percentage of intrusion errors (PIE) in relation to total correct responses (13). This method takes into account the observation that the number of intrusion errors a person makes is often highly dependent on their total responses on a particular trial. Thus, even a seemingly modest number of intrusion errors may represent an at-risk cognitive profile. PIE demonstrated high levels of sensitivity and specificity in differentiating CN from amyloid positive persons with preclinical AD and preliminary work suggests that it is a novel and sensitive index of early memory dysfunction (11, 13).
Traditional paper and pencil neuropsychological assessments are lengthy, require a skilled examiner, are vulnerable to human error in administration and scoring, and associated with practice effects. Moreover, some of these measures have been found to be biased among diverse ethnic/cultural and language groups. To address some of these concerns, computerized testing batteries have been developed to explore a more suitable option to mitigate some of the abovementioned limitations (14-17). However, these too have limitations in early detection of AD-associated cognitive impairment. For example, many of these computerized batteries are relatively successful at distinguishing between older adults with normal cognition and those with dementia or late stage MCI, but lack the predictive power needed to move the field forward, which is to correctly classify individuals with MCI and/or earlier on the disease continuum, and do so in a manner that is validated for use among different ethnic/cultural and language groups. This highlights a major problem with many traditional computerized batteries; they are automated versions of traditional paper-and-pencil cognitive assessment paradigms that lack sensitivity to detect AD-associated cognitive decline, and employ the same paradigms originally developed for the assessment of dementia or traumatic brain injury (17).
Recent work by Curiel and associates (5-12) led to the development of a brief computerized version of the LASSI-L, the LASSI-BC, which incorporates all the elements of this well-established cognitive stress test. The LASSI-BC is currently being studied extensively in a longitudinal study of at-risk aging adults. This novel computerized version of the instrument does not require a skilled examiner, is web-based and can remotely run on most browser-capable devices. Moreover, it is intuitive and appropriate for use among older adults that are either predominantly English or Spanish-speaking and who have varying ethnic/cultural backgrounds including Hispanics and African Americans.
In this first validation study, we examine the psychometric properties of the LASSI-BC. We also evaluate the clinical utility several LASSI-BC subscales as it relates to their ability to differentiate older adults with normal cognition from those with aMCI on measures of: i) proactive semantic interference, ii) the failure to recover from proactive semantic interference, iii) retroactive semantic interference and iv) the percentage of intrusion errors in relation to total cued recall responses by the participant. Performance on these specific subscales were selected a priori because, as noted above, our previous work using the paper-and-pencil LASSI-L has robustly demonstrated that these particular subscales are the most sensitive to cognitive breakdowns associated with MCI due to preclinical and prodromal AD.



This study included 64 older adults that were evaluated as part of an IRB-approved longitudinal investigation funded by the National Institute on Aging. An experienced clinician administered a standard clinical assessment protocol, which included the Clinical Dementia Rating Scale (CDR) (18), and the Mini-Mental State Examination (MMSE) (19). Subsequently, a neuropsychological battery was independently administered in either Spanish or English dependent on the participant’s dominant and preferred language. Spanish language evaluations were completed with equivalent standardized neuropsychological tests and appropriate age, education, and cultural/language normative data (20-23). Proficient bilingual (Spanish/English) psychometricians performed all the testing.
Diagnostic groups were classified using the following criteria:

Amnestic MCI group (aMCI) (n=25)

Participants met Petersen’s criteria (24)) for MCI and evidenced all of the following: a) subjective cognitive complaints by the participant and/or collateral informant; b) evidence by clinical evaluation or history of memory or other cognitive decline; c) Global Clinical Dementia Rating scale of 0.5 (18); d) below expected performance on delayed recall of the HVLT-R (23) or delayed paragraph recall from the National Alzheimer’s Coordinating Center -Unified Data Set (NACC-UDS) (25) as measured by a score that is 1.5 SD or more below the mean using age, education, and language-related norms.

Cognitively Normal Group (n=39)

Participants were classified as cognitively normal if all of the following criteria were met: a) no subjective cognitive complaints made by the participant and a collateral informant; b) no evidence by clinical evaluation or history of memory or other cognitive decline after an extensive interview with the participant and an informant; c) Global CDR score of 0; d) performance on all traditional neuropsychological tests (e.g.: Category Fluency (26), Trails A and B (27), WAIS-IV Block Design subtest (28) was not more than 1.0 SD below normal limits for age, education, and language group.

Loewenstein-Acevedo Scales for Semantic Interference and Learning, Brief Computerized Version (LASSI-BC)

The LASSI-BC was not used for diagnostic determination in this study. This computerized cognitive stress test is a novel computerized measure that is briefer than the paper-and-pencil LASSI-L, taking approximately 10 to 12 minutes to complete. The LASSI-BC contains the elements of the original LASSI-L which demonstrated the greatest differentiation between aMCI, PreMCI and CN older adults in previous studies. For example, free recall preceding the cued recall trials of the LASSI-L added time to the administration but was never as effective as cued recall in distinguishing among diagnostic groups. Developed in collaboration with the University of Miami Department of Computer Science, the LASSI-BC is a remotely accessible test available in both English and Spanish. As a web application, it can be run on devices that can run Google Chrome, including desktop computers, laptops, tablets, or even smartphones. While the LASSI-BC is a fully self-administered test with all verbal responses recorded and scored by the computer, for the purposes of this validation study, a trained study team member was present for each administration to systematically record responses, which provided a double check on the accuracy of data. The LASSI-BC utilizes Google Cloud Speech API , which has been successfully implemented for use with older adults. The test leverages Google Cloud’s Speech to Text software in conjunction with a backup lexicon for understanding the participants’ spoken words. The lexicon is designed to account for variations in participant’s pronunciation by allowing for words that the computer “mishears” to serve as alternatives to the actual word being spoken. Lexicons were chosen based on observations from participants during the test.
Upon initiating the examination, the participant is instructed in both audio and visual formats. They will see 15 words belonging to one of three semantic categories: fruits, musical instruments, or articles of clothing (five words per category). The words are then individually presented on the screen and audio for a 6-second interval. This presentation facilitates optimal encoding and storage of the to-be-remembered information. Further, this instruction style has been easily understood and accepted by older adults during pilot studies in the course of developing the LASSI-BC. After the computer presents all 15 words, participants are presented with each category cue (e.g., fruits) and asked to recall the words that belonged to that category. Participants are then presented with the same target stimuli for a second learning trial with subsequent cued recall to strengthen the acquisition and recall of the List A targets. The exposure to the semantically related list (i.e., List B) is then conducted in the same manner as exposure to List A. List B consists of 15 words different from List A, all of which belong to each of the three categories used in List A (i.e., fruits, musical instruments, and articles of clothing). Following the presentation of the List B words, the person is asked to recall each of the List B words that belonged to each of the categories. List B words are presented again, followed by a second category-cued recall trial. Finally, to assess retroactive semantic interference, participants are asked to free recall the original List A words. Primary measures used in this study are the second cued recall score for List A (maximum learning), first cued recall score for List B (susceptibility to proactive semantic interference), second cued recall of List B (failure to recover from proactive semantic interference), and the third cued recall of List A (retroactive semantic interference). In addition, we evaluated the novel ratio used with the LASSI-L, that takes into account the percentage of intrusion errors (PIE) as a function of total responses on subscales that measure proactive semantic interference and the failure to recover from proactive semantic interference. Specifically, the ratio is denoted as follows: Total Intrusion Errors/ (Total Intrusion Errors + Total Correct Responses) for LASSI-BC Cued B1 (a measure of susceptibility to proactive semantic interference) and LASSI-BC Cued B2 recall (a measure of recovery from proactive semantic interference).



The computerized version of the LASSI-BC had psychometric properties that compared favorably to the test-retest reliabilities obtained on the original paper-and-pencil LASSI-L (7). As depicted in Table 1, CN (n=39) and aMCI (n=25) groups did not differ in terms of age, sex, or language of evaluation. Individuals diagnosed as aMCI, although well educated (Mean =14.26; SD=3.5), had less educational attainment relative to their cognitively normal counterparts (Mean =16.32; SD=2.3). As expected, aMCI participants also had lower mean MMSE scores (Mean =26.04; SD=2.3).

Table 1. Demographic Characteristics and Computerized LASSI-BC Scores among Participants who are Cognitively Normal and with Amnestic Mild Cognitive Impairment


Test-retest reliability

Test-retest reliability data was obtained on a subset of 15 older adults diagnosed with aMCI using Petersen’s criteria (24) for each of the LASSI-BC subscales. The mean age was 73.4 (SD=6.3); education 15.4 (SD=3.6); and the mean MMSE score for this group was 26.6 (SD=2.2). These individuals (60% primary English-speakers and 60% female) were administered the LASSI-BC on two occasions, within a 4 to 39-week interval (Mean =13.9.; SD=10.6 weeks). In our pilot work, we found robust test-retest correlations ranging from 0.55 to 0.721 on the subscales that have shown to be the most sensitive measures of cognitive decline in the original paper-and-pencil version. In this study, test-retest comparisons were conducted for Cued Recall A2 (measures maximum learning), Cued Recall B1 (measures proactive semantic interference), and Cued Recall B2 (measures the failure to recover from proactive semantic interference). One-tailed Pearson Product Moment Correlation Coefficients were obtained given the directional hypotheses concerning test-retest relationships. High, statistically significant test-retest reliabilities were obtained for Cued A2 Recall (r=.726; p<.001); Cued Recall B1 (r=.529; p=0.021); Cued Recall B2 (r=.555; p=0.016).

Discriminant validity

As depicted in Table 1, LASSI-BC scales sensitive to maximum learning (Cued A2), vulnerability to proactive semantic interference (Cued B1) and the failure to recover from proactive semantic interference (Cued B2) were statistically significant in discriminating between older adults with amnestic MCI and cognitively normal counterparts. These results were identical when demographic variables such as education were entered in the model as covariates
We then calculated areas under the Receiver Operating Characteristic (ROC) curve for LASSI-BC correct responses as well as the PIE indices for Cued B1 and Cued B2 subscales. We selected these measures a priori given that performance on these specific subscales have traditionally been the most discriminant measures on the paper-and-pencil form of the LASSI-L.
As shown in Table 2, an optimal cut-point of 5 by Youden’s criteria on correct responses for Cued Recall B1, yielded a sensitivity of 84.6% and a specificity of 86.8%. An optimal cut-point of 9 by Youden’s criteria on correct responses provided on Cued Recall B2, yielded statistically significant areas under the ROC curve of .868 (SE=0.88) and .824 (SE=.051), respectively.

Table 2. Classification of aMCI versus Cognitively Normal Participants on the LASSI-BC


We subsequently examined an optimal cut-point for PIE on the Cued Recall B1 and the Cued Recall B2 subscales. For PIE on Cued Recall B1, the area under the ROC was .879 (SE=.06) with a sensitivity of 92.9% and specificity of 80%, respectively using an optimal cut-point of .2540. For PIE on Cued Recall B2, the area under the ROC was .801 (SE=.07), using an optimal cut-point of .2159, which yielded a sensitivity of 78.6% and specificity of 68.0%. We selected these specific subscales because they have shown to be the strongest predictors of aMCI in the paper-and-pencil form of the LASSI-L.
We subsequently entered the statistically significant LASSI-BC subscales (Cued Recall B1 and Cued Recall B2) into a stepwise logistic regression. As seen in Table 3, the first variable to enter the logistic regression model was PIE on Cued B1 [B=6.86 (SE=1.67) Wald=17.07, p<0.001)]. On the second step of the logistic regression model, correct responses on Cued Recall B2 entered the model [B=-.34 (SE=.128), Wald= 17.1 (p=.008)]. Combining PIE Cued Recall B1 and correct responses on Cued Recall B2, yielded an overall sensitivity of 80% and specificity of 89.7%. It should be noted logistic regression weighs overall classification in a manner that favors the largest diagnostic group (in this case CN participants). Nonetheless, ROC and stepwise regression models yielded similar results indicating excellent discriminative ability.
In sum, our findings support that the LASSI-BC has equal or better psychometric properties than the original paper-and-pencil LASSI-L and demonstrates that computerized administration is both feasible, well accepted, and has excellent discriminant properties.

Table 3. Step-wise Logistic Regression Using Proactive Semantic Interference Measures on the Computerized LASSI-BC



The present study was designed to examine the psychometric properties of the LASSI-BC, the brief computerized version of the LASSI-L, a cognitive stress test that utilizes a novel cognitive assessment paradigm based on semantic interference in memory. In studies conducted in the United States and abroad, the LASSI-L has shown great utility in detecting cognitive changes among individuals during the preclinical and prodromal stages of AD (4, 29) and has been found to be appropriate for use among diverse ethnic/cultural and language groups (11, 30, 12). The paradigm that this measure employs is unique in that it explicitly and from the outset organizes the examinee’s learning around specific semantic categories, which promotes active encoding, reduces the use of individualized learning strategies that can help or hinder performance, increases depth of initial learning, and is designed to tap an individual’s vulnerability to semantic interference.
The current investigation examined all salient subscales of the LASSI-BC, which were selected based on previous work with the paper-and-pencil versions. The computerized version evidenced good test-retest reliability for participants diagnosed with aMCI. Scores on all LASSI-BC subscales were higher for cognitively normal older adults, relative to aMCI participants. In addition, high levels of discriminant validity were obtained in differentiating aMCI from cognitively normal groups based on ROC analyses and logistic regression.
A potential limitation of this first validation study is that we employed modest numbers of participants who were tested in either English or Spanish on the LASSI-BC. Although, our overall findings were highly significant and the paper-and-pencil LASSI-L has been validated in different languages (i.e.- Spanish speakers in Argentina, Spanish speakers in Spain, Spanish speakers from Mexico, etc.) and with different ethnic/cultural groups (European Americans, Hispanics and African Americans), such future comparisons should be made with the LASSI-BC. Further, additional studies with the LASSI-BC will include evaluating the diagnostic utility of this computerized cognitive stress test to differentiate older adults earlier on the preclinical continuum of AD, and relate performance to biomarkers of AD pathology, as well as compare it to other traditional and widely used cognitive measures in the field.
There has been an increase in the number of computerized tests developed including the CogState (31) and the Cognition Battery from the NIH Toolbox (16), but limitations exist. For example, one of the most widely-used computerized cognitive batteries for the assessment of MCI is the CogState. As part of the Mayo Clinic Study on Aging, Mielke and associates (32) administered the CogState to eighty-six participants diagnosed with MCI who were found to have worse performance than cognitively healthy individuals; however, it is likely that individuals classified as MCI ranged from early states of MCI to late MCI, the latter of which is more cognitively similar to early dementia in terms of neuropsychological test performance, limiting evidence that this measure in sensitive to preclinical cognitive change. Further, the authors noted that their results are not generalizable to other ethnicities due to the demographic makeup of the region (Minnesota, USA). Another study conducted by Mielke and colleagues (33) aimed to examine performance on the CogState with neuroimaging biomarkers (MRI, FDG PET, and amyloid PET) among cognitively normal participants aged 51-71; however, only weak associations were found between CogState subtests and biomarkers of neurodegeneration.
With the rapidly aging population, early detection of cognitive decline in individuals at risk for AD and related disorders has become a global priority. Accurately identifying at risk individuals through the detection and monitoring of subtle, albeit sensitive cognitive changes that transpire early in the disease course is an important initiative and computerized cognitive outcome measures have the potential to greatly reduce burden for participants, clinical researchers and clinicians.
The development of computerized cognitive tests for older adults has significantly increased during the past decade. In fact, available systematic reviews have identified more than a dozen computerized measures designed to detect dementia or MCI (34, 35, 36). Moreover, the use of computerized assessments with older adults has been found to be feasible and reliable (37, 38). A recent meta-analysis has shown relatively good diagnostic accuracy, and authors further concluded that their performance distinguishing individuals with MCI and dementia is comparable with traditional paper-pencil neuropsychological measures (35). It is anticipated that as technology advances, clinical trials will include validated computerized testing to sensitively capture cognitive performance, particularly in large-scale secondary prevention efforts (39). The impact of this technological advancement in computerized, web-based cognitive testing has the potential to facilitate remote deliverability, allow for real-time data entry, improves standardization, and reduces administration and scoring errors. Moreover, computerized assessment can more readily monitor longitudinal cognitive changes for each individual, facilitating a precision-based approach. It is critical; however, that emerging cognitive tests move beyond simply computerizing outdated, insensitive cognitive paradigms and instead invest in the development and validation of cognitive paradigms that are sensitive and specific to early cognitive breakdowns that occur during the preclinical stages of AD. These too should exhibit sensitivity to biomarkers of AD (e.g., amyloid load, tau deposition, and neurodegeneration in AD-prone regions). Doing so may address some of the most critical challenges facing clinical trials including proper selection of at-risk participants, and monitoring meaningful cognitive change over time.

Funding: This research was funded by the National Institute of Aging Grant 1 R01 AG047649-01A1 (David Loewenstein, PI), 1 R01 AG047649-01A1 (Rosie Curiel Cid, PI) 5 P50 AG047726602 1Florida Alzheimer’s Disease Research Center (Todd Golde, PI), 8AZ. The sponsors had no role in the design and conduct of the study; in the collection analysis, and interpretation of data; in the preparation of the manuscript; or in the review or approval of the manuscript.

Ethical standard: This research study was conducted in alignment with the Declaration of Helsinki and through the approval of the University of Miami Institutional Review Board.
Conflict of interest: Drs. Curiel and Loewenstein have intellectual property used in this study.


1. Jack Jr CR, Bennett DA, Blennow K, et al. NIA-AA research framework: toward a biological definition of Alzheimer’s disease. Alzheimer’s & Dementia. 2018 Apr;14(4):535-62.
2. Harvey PD, Cosentino S, Curiel R, et al. Performance-based and observational assessments in clinical trials across the Alzheimer’s disease spectrum. Innovations in clinical neuroscience. 2017 Jan; 14(1-2):30.
3. Edmonds EC, Delano-Wood L, Galasko DR, et al. Subtle cognitive decline and biomarker staging in preclinical Alzheimer’s disease. Journal of Alzheimer’s disease. 2015 Jan 1;47(1):231-42.
4. Loewenstein DA, Curiel RE, Duara R, et al. Novel cognitive paradigms for the detection of memory impairment in preclinical Alzheimer’s disease. Assessment. 2018 Apr25;(3):348-59.
5. Brooks L, Loewenstein D. Assessing the progression of mild cognitive impairment to Alzheimer’s disease: current trends and future directions. Alzheimer’s Research & Therapy. 2010;2(28):28-28.
6. Crocco E, Curiel RE, Acevedo A, et al. An evaluation of deficits in semantic cueing and proactive and retroactive interference as early features of Alzheimer’s disease. The American Journal of Geriatric Psychiatry. 2014 Sep 1;22(9):889-97.
7. Curiel RE, Crocco E, Acevedo A, et al. A new scale for the evaluation of proactive and retroactive interference in mild cognitive impairment and early Alzheimer’s disease. Aging. 2013;1(1):1000102.
8. Loewenstein DA, Curiel RE, Greig MT, et al. A novel cognitive stress test for the detection of preclinical Alzheimer disease: discriminative properties and relation to amyloid load. The American Journal of Geriatric Psychiatry. 2016 Oct 1;24(10):804-13.
9. Matías-Guiu JA, Curiel RE, Rognoni T, Valles-Salgado M, Fernández-Matarrubia M, Hariramani R, Fernández-Castro A, Moreno-Ramos T, Loewenstein DA, Matías-Guiu J. Validation of the Spanish version of the LASSI-L for diagnosing mild cognitive impairment and Alzheimer’s disease. Journal of Alzheimer’s Disease. 2017 Jan 1;56(2):733-42.
10. Rosselli M, Loewenstein DA, Curiel RE, et al. Effects of bilingualism on verbal and nonverbal memory measures in mild cognitive impairment. Journal of the International Neuropsychological Society. 2019 Jan;25(1):15-28.
11. Capp KE, Curiel Cid, RE, Crocco EA, et al. Semantic Intrusion Error Ratio Distinguishes Between Cognitively Impaired and Cognitively Intact African American Older Adults. Journal of Alzheimer’s Disease. 2019 Dec 23(Preprint):1-6.
12. Matias-Guiu JA, Cabrera-Martín MN, Curiel RE, et al. Comparison between FCSRT and LASSI-L to detect early stage Alzheimer’s disease. Journal of Alzheimer’s Disease. 2018 Jan 1;61(1):103-11.
13. Crocco, E, Curiel RE, Grau, G. Percentage of intrusion errors predicts patterns of cognitive change in older adults. (Under Review). Journal of Alzheimer’s Disease.
14. Beaumont JL, Havlik R, Cook KF, et al. Norming plans for the NIH Toolbox. Neurology. 2013;80(11 Suppl 3):S87–S92.
15. Saxton J, Morrow L, Eschman A, et al. Computer assessment of mild cognitive impairment. Postgraduate medicine. 2009 Mar 1;121(2):177-85.
16. Weintraub S, Dikmen SS, Heaton RK, et al. Cognition assessment using the NIH Toolbox. Neurology. 2013 Mar 12;80(11 Supplement 3):S54-64.
17. Parsons TD, Courtney CG, Arizmendi BJ, et al. Virtual reality stroop task for neurocognitive assessment. InMMVR 2011 Feb 16 (pp. 433-439).
17. Morris JC. Clinical dementia rating: a reliable and valid diagnostic and staging measure for dementia of the Alzheimer type. International psychogeriatrics. 1997 Dec;9(S1):173-6.
18. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”: a practical method for grading the cognitive state of patients for the clinician. Journal of psychiatric research. 1975 Nov 1;12(3):189-98.
19. Arango-Lasprilla JC, Rivera D, Aguayo A, et al. Trail making test: Normative data for the Latin American Spanish speaking adult population. NeuroRehabilitation. 2015 Jan 1;37(4):639-61.
20. Arango-Lasprilla JC, Rivera D, Garza MT, et al. Hopkins verbal learning test–revised: Normative data for the Latin American Spanish speaking adult population. NeuroRehabilitation. 2015 Jan 1;37(4):699-718.
21. Benson G, de Felipe J, Sano M. Performance of Spanish-speaking community-dwelling elders in the United States on the Uniform Data Set. Alzheimer’s & Dementia. 2014 Oct;10:S338-43.
22. Peña-Casanova J, Quinones-Ubeda S, Gramunt-Fombuena N,et al. Spanish Multicenter Normative Studies (NEURONORMA Project): norms for verbal fluency tests. Archives of Clinical Neuropsychology. 2009 Jun 1;24(4):395-411.
23. Brandt J. The Hopkins Verbal Learning Test: Development of a new memory test with six equivalent forms. The Clinical Neuropsychologist. 1991 Apr 1;5(2):125-42.
24. Petersen RC. Mild cognitive impairment as a diagnostic entity. Journal of internal medicine. 2004 Sep;256(3):183-94.
25. Beekly DL, Ramos EM, Lee WW, Deitrich WD, Jacka ME, Wu J, Hubbard JL, Koepsell TD, Morris JC, Kukull WA. The National Alzheimer’s Coordinating Center (NACC) database: the uniform data set. Alzheimer Disease & Associated Disorders. 2007 Jul 1;21(3):249-58.
26. Binetti G, Magni E, Cappa SF, et al. Semantic memory in Alzheimer’s disease: an analysis of category fluency. Journal of Clinical and Experimental Neuropsychology. 1995 Feb 1;17(1):82-9.
27. Reitan RM. Validity of the Trail Making Test as an indicator of organic brain damage. Perceptual and motor skills. 1958 Dec;8(3):271-6.
28. Wechsler D. Wechsler adult intelligence scale–Fourth Edition (WAIS–IV). San Antonio, TX: NCS Pearson. 2008;22(498):816-27.
29. Crocco EA, Loewenstein DA, Curiel RE, et al. A novel cognitive assessment paradigm to detect Pre-mild cognitive impairment (PreMCI) and the relationship to biological markers of Alzheimer’s disease. Journal of psychiatric research. 2018 Jan 1;96:33-8.
30. Curiel Cid RE, Loewenstein DA, Rosselli M, et al. A cognitive stress test for prodromal Alzheimer’s disease: Multiethnic generalizability. Alzheimer’s & Dementia: Diagnosis, Assessment & Disease Monitoring. 2019 Dec;11(C):550-9.
31. Darby D, Collie A, McStephen M, Maruff P. Reliable detection of asymptomatic longitudinal cognitive decline in healthy community dwelling volunteers. Journal of the American Geriatrics Society. 2004 Apr;52.
32. Mielke MM, Machulda MM, Hagen CE, Edwards KK, Roberts RO, Pankratz VS, Knopman DS, Jack Jr CR, Petersen RC. Performance of the CogState computerized battery in the Mayo Clinic Study on Aging. Alzheimer’s & Dementia. 2015 Nov 1;11(11):1367-76.
33. Mielke MM, Weigand SD, Wiste HJ, Vemuri P, Machulda MM, Knopman DS, Lowe V, Roberts RO, Kantarci K, Rocca WA, Jack Jr CR. Independent comparison of CogState computerized testing and a standard cognitive battery with neuroimaging. Alzheimer’s & Dementia. 2014 Nov 1;10(6):779-89.
34. Zygouris S, Tsolaki M. Computerized cognitive testing for older adults: a review. American Journal of Alzheimer’s Disease & Other Dementias®. 2015 Feb;30(1):13-28.
35. Chan JY, Kwong JS, Wong A, Kwok TC, Tsoi KK. Comparison of computerized and paper-and-pencil memory tests in detection of mild cognitive impairment and dementia: A systematic review and meta-analysis of diagnostic studies. Journal of the American Medical Directors Association. 2018 Sep 1;19(9):748-56.
36. De Roeck EE, De Deyn PP, Dierckx E, Engelborghs S. Brief cognitive screening instruments for early detection of Alzheimer’s disease: a systematic review. Alzheimer’s research & therapy. 2019 Dec 1;11(1):21.
37. Wild K, Howieson D, Webbe F, Seelye A, Kaye J. Status of computerized cognitive testing in aging: a systematic review. Alzheimer’s & Dementia. 2008 Nov 1;4(6):428-37.
38. Pankratz VS, Roberts RO, Mielke MM, Knopman DS, Jack CR, Geda YE, Rocca WA, Petersen RC. Predicting the risk of mild cognitive impairment in the Mayo Clinic Study of Aging. Neurology. 2015 Apr 7;84(14):1433-42.
39. Buckley RF, Sparks KP, Papp KV, Dekhtyar M, Martin C, Burnham S, Sperling RA, Rentz DM. Computerized cognitive testing for use in clinical trials: a comparison of the NIH Toolbox and Cogstate C3 batteries. The journal of prevention of Alzheimer’s disease. 2017;4(1):3.


X. Fu1,*, W. Yu2,*, M. Ke2, X. Wang1, J. Zhang1, T. Luo1, P.J. Massman3,4, R.S. Doody3, Y. Lü1,*

1. Department of Geriatrics, The First Affiliated Hospital of Chongqing Medical University, Chongqing 400016, China; 2. Institute of Neuroscience, Chongqing Medical University, Chongqing 400016, China; 3. Department of Neurology, Baylor College of Medicine, Houston, TX USA at the time this work was done. Now Genentech/Roche, Basel, Switzerland; 4. Department of Psychology, University of Houston, Houston, TX USA; *Authors contributed equally and are co-first authors of the study.

Corresponding Authors: Prof. Yang Lü, 1 Youyi Road, Yuzhong District, Chongqing 400016, China, Tel: +86-23-89011622, Fax: +86-23-68811487, E-mail:

J Prev Alz Dis 2020;
Published online December 21, 2020,



BACKGROUND: A specialized instrument for assessing the cognition of patients with severe Alzheimer’s disease (AD) is needed in China.
Objectives: To validate the Chinese version of the Baylor Profound Mental Status Examination (BPMSE-Ch).
Design: The BPMSE is a simplified scale which has proved to be a reliable and valid tool for evaluating patients with moderate to severe AD, it is worthwhile to extend the use of it to Chinese patients with AD.
Setting: Patients were assessed from the Memory Clinic Outpatient.
Participants: All participants were diagnosed as having probable AD by assessment.
Measurements: The BPMSE was translated into Chinese and back translated. The BPMSE-Ch was administered to 102 AD patients with a Mini-Mental State Examination (MMSE) score below 17. We assessed the internal consistency, reliability, and construct validity between the BPMSE-Ch and MMSE, Severe Impairment Battery (SIB), Global Deterioration Scale (GDS-1), Geriatric Depression Scale(GDS-2), Instrumental Activities of Daily Living (IADL), Physical Self-Maintenance Scale (PSMS), Neuropsychiatric Inventory (NPI) and Clinical Dementia Rating (CDR).
Results: The BPMSE-Ch showed good internal consistency (α = 0.87); inter-rater and test-retest reliability were both excellent, ranging from 0.91 to 0.99. The construct validity of the measure was also supported by significant correlations with MMSE, SIB. Moreover, as expected, the BMPSE-Ch had a lower floor effect than the MMSE, but a ceiling effect existed for patients with MMSE scores above 11.
Conclusions: The BPMSE-Ch is a reliable and valid tool for evaluating cognitive function in Chinese patients with severe AD.

Key words: Alzheimer’s disease, Baylor Profound Mental Status Examination, Chinese version, severe dementia, validation.

Abbreviations: AD: Alzheimer’s disease; ADAS-cog: Alzheimer’s Disease Assessment Scale-Cognitive section; ANOVA: A one-way analysis of variance; BPMSE: Baylor Profound Mental Status Examination; BPMSE-Ch: Chinese version of the Baylor Profound Mental Status Examination; BPMSE-Ch-cog: Cognition subscale of Chinese version of the Baylor Profound Mental Status Examination; BPMSE-Ch-behav: Behavior subscale of Chinese version of the Baylor Profound Mental Status Examination; CDR: Clinical Dementia Rating; FAST: Functional Assessment Staging; GDS-1: Global Deterioration Scale; GDS-2: Geriatric Depression Scale: IADL, Instrumental Activities of Daily Living; MMSE: Mini-Mental State Examination; NPI: Neuropsychiatric Inventory; PSMS: physical self-maintenance scale; SIB: Severe Impairment Battery.



Alzheimer’s disease (AD) is a common neurodegenerative disorder among mainly elderly persons worldwide. The manifestations of AD include deterioration in cognition, memory and activities of daily living. It is usually accompanied by behavioral and psychological symptoms (1).
Currently, China is facing serious issues related to having an aging population. Persons aged 60 or older account for 17.3% of the total population (2). The prevalence of all-cause dementia over age 65 is about 6% in China, and AD makes up about 65% of all cases (3, 4). The rough prevalence of AD in China has reported to ranges from 7 per 1000 people to 66 per 1000 individuals (5). In a population-based cross-sectional survey, 10276 residents aged 65 year or older were drawn from Beijing (northern-eastern), Zhengzhou (northern-central), Guiyang (southern-western) and Guangzhou (southern-eastern). This survey showed that the prevalence of AD was 3.21% in a total of 10276 residents (6). Despite the fact that China has the relatively high AD prevalence, few studies of AD were conducted to research excellent methods for AD diagnosing and evaluating.
It seems unquestionable that AD is gradually evolving into a crucial social problem and presents a major challenge for health-care in China. However, awareness of AD and dementia in general is inadequate in China, leading to delayed diagnosis and initiation of treatment (7, 8).Therefore, many patients do not get evaluated until moderate to severe stages of the disease (9, 10). Moreover, once these patients present for an evaluation, tools to assess them are limited (11). Hence, better instruments are needed for the accurate assessment of patients with advanced AD.
A variety of neuropsychological and functional measures have been utilized to assess mental status and dementia severity both cross-sectionally and longitudinally. Frequently-used instruments include the Mini-Mental Status Examination (MMSE) (12), Severe Impairment Battery (SIB) (13), Alzheimer’s Disease Assessment Scale-Cognitive section (ADAS-Cog) (14), Geriatric Deterioration Scale (GDS-1) (15), Functional Assessment Staging Tool(FAST) (16) and Clinical Dementia Rating (CDR) (17). However, these scales show some limitations in patients with moderate to severe AD. The MMSE and ADAS-cog are not optimal for evaluating patients with severe AD because both contain a lot of verbal information and; therefore, the results may be confounded by language disorders and/or low level of education. SIB is a suitable tool to evaluate patients with severe dementia. However, this test takes more than 30 minutes to administer, which often exceeds the attention capacity of most patients with severe AD (18). The NPI is usually used to evaluate neuropsychiatric symptoms, but it is largely dependent on the description from caregivers (19). Overall, it is clear that a convenient and effective assessment instrument for measuring cognitive function in patients with severe AD is highly needed.
The Baylor Profound Mental State Examination (BPMSE) developed by Doody RS et al, is a simplified scale which has proved to be a reliable and valid tool for evaluating patients with moderate to severe AD (20). And in Doody’s study, European American accounted for about 82% of the original population. Thus, it is worthwhile to extend the use of the BPMSE to Severely demented patients from different cultural backgrounds. To date, there have been three translated versionsof the BPMSE, including Korean, Danish and Spanish (21-23). A study of the Korean version has shown that the BPMSE is a rapid, easy and valid scale for measuring cognitive function in patients with moderate to severe AD, particularly in patients with MMSE below 12. Similarly, a study utilizing the Danish version indicated that the BPMSE is a stable and strong instrument, and was recommended as an appropriate measure of dementia severity in patients with more sever impairment. Adaptation of the Spanish version revealed that BPMSE that the BPMSE is a useful tool for assessing cognitive function, even in daily medical practice focusing on patients with severe AD.
In China, there is no applicable scale for assessing patients with severe AD. Therefore, the aim of our study was to develop a Chinese version of the BPMSE (BPMSE-Ch) and to evaluate the psychometric properties of this version in Chinese patients with AD.




The original version of BPMSE consists of three parts, including the cognition subscale which includes 25 questions, the behavior subscale which includes 10 items to rate the presence or absence of behavioral problems, and 2 qualitative observations of language and social interaction. The cognition subscale assesses four areas: language, orientation, attention, and motor skills. The BPMSE total cognition subscale has a score between 0and 25: maximum 5 scores for orientation, 11 scores for language, 4scores for attention and 5 scores for motor skills. BPMSE behavior subscale score has a score between 0 (no behavioral disturbances) and 10 (all behavioral disturbances). In present study, we did not study the 2 qualitative observations about communication and social interactions.
Firstly, the original version of BPMSE was translated into Chinese with Mandarin by two bilingual translators whose mother tongue was Chinese. Then, the two Chinese versions were discussed by our team with gerontologists, a neurologist, a psychologist and an English expert, and the final Chinese version was formulated based on this input. Finally, two other translators of English philology back translated the final Chinese version into English to confirm consistency with the original version.


Patients were recruited from the Memory Clinic, Department of Geriatrics, The First Affiliated Hospital of Chongqing Medical University.

Enrollment criteria

(a) All participants were diagnosed as having probable AD according to National Institute of Neurological and Communicative Diseases and Stroke/Alzheimer’s Disease and Related Disorders Association criteria (NINCDS-ADRDA) (24); (b) Patients with MMSE <17 were included; (c) This study was approved by the Ethical Committee of The First Affiliated Hospital of Chongqing Medical University on human research; (d) Informed consent was obtained from all participants or their family members.

Exclusion criteria

(a) Patients were excluded if they had other neurological or psychiatric disorders or clinically significant medical conditions (e.g., acute infections, cancer, organ failure etc.,); (b) Patients had severely impaired communication abilities (e.g., global aphasia, deafness, blindness, muteness etc.,); (c) Patients had a history of head trauma, sedative drugs use or substance abuse.


The following measures were administered to all enrolled patients: BPMSE-Ch, MMSE, SIB, GDS-1, GDS-2, IADL, PSMS, NPI, and CDR. All tests were given on the same day. Two trained physicians in our clinic administered the BPMSE-Ch to evaluate a subset of enrolled patients consecutively and independently in order to examine inter-rater reliability. Finally, to investigate test-retest reliability, some patients were randomly chosen to be given the BPMSE-Cha second time within 30days of the first administration. It took 5 minutes on average to administer the BPMSE-Ch.

Statistical analyses

Internal consistency was assessed by computing coefficient α. Inter-rater reliability was assessed by correlation and paired t-test of the two scores obtained by different professionals on the same day. And the test-retest reliability was also calculated with correlational and paired t-test analyses using scores obtained on the same patient within 30 days. The correlations between the BPMSE-Ch and other measures including the SIB, MMSE, GDS-1, GDS-2, IADL, PSMS, NPI and CDR were calculated with Pearson correlations in order to evaluate construct validity. In addition, patients were divided into dementia severity groups using the MMSE and CDR, and differences between those groups were analyzed by conducting a one-way analysis of variance (ANOVA) and Scheffé’s test. Statistical analyses were performed with SPSS 20.0 for Windows.



Demographic characteristics and test performances

102 patients (male: 35, female: 67) were included in our study, the mean age of the patients was 77.76, ranging between 64 and 93. The mean years of education was 7.95, ranging from 0 to 16 years. The specific variations were showed in Table 1.

Table 1. Demographic characteristics and Scores on Instruments

Abbreviations: MMSE, Mini-Mental State Examination; BPMSE-Ch-cog, Cognition subscale of Chinse version of the Baylor Profound Mental Status Examination; BPMSE-Ch-behav, Behavior subscale of Chinse version of the Baylor Profound Mental Status Examination; SIB, Severe Impairment Battery; NPI, Neuropsychiatric Inventory; SD, standard deviation.



In our study, the coefficient α which could reflect the inter-correlations for items on the BPMSE-Ch cognition (BPMSE-Ch-cog) subscale, was 0.87. Furthermore, significant correlations were found among all the BPMSE-Ch-cog components, as seen in Table 2. Inter-rater and test-retest reliability were showed in Table 3.

Table 2. Correlations among BPMSE-Ch-cogsubscales

Correlation coefficients by Pearson correlation. * p< 0.001.


Table 3. Inter-rater and test-retest reliability

Abbreviations: BPMSE-Ch-cog, Cognition subscale of Chinse version of the Baylor Profound Mental Status Examination; BPMSE-Ch-behav, Behavior subscale of Chinse version of the Baylor Profound Mental Status Examination.
Correlation coefficients by Pearson correlation. n = Number of patients. All p values <0.001.


52 patients were tested twice by two trained doctors simultaneously and independently to determine the inter-rater reliability. The correlation between two total cognition subscale scores was 0.99 (p < 0.001) and there was no significant difference (paired t (51) = +1.84, p > 0.05) between the two scores (Mean = 0.17, SD = 0.68). The correlation between two behavior subscale scores was 0.92 (p < 0.001).
42 patients were tested twice by a same doctor within 30 day-interval for the test-retest reliability. The test-retest correlation between two total cognition scores was 0.99 (p < 0.001). Similarly, there was no significant difference (paired t (41) = +1.18, p > 0.05) between the two scores obtained at two time points (Mean = 0.14, SD = 0.78). The test-retest correlation between two behavior scores was 0.94 (p < 0.001).


Construct validity of the BPMSE-Ch was showed in Table 4. The correlations between the BPMSE-Ch-cog and MMSE (0.76), SIB (0.78), GDS-1 (-0.26), GDS-2 (0.16), PSMS (-0.26), IADL (-0.36), NPI (-0.41), CDR (-0.54) were calculated by Pearson correlation. The results showed that the construct validity of BPMSE-cog was very good (r=0.78) for SIB and good for MMSE (0.76). In addition, the relationship between BPMSE-Ch behavior subscale (BPMSE-Ch-behav) and NPI was analyzed (0.54, p < 0.001, Table 4).

Table 4. Concurrent validity of BPMSE-Ch

Abbreviations: BPMSE-Ch, Chinse version of the Baylor Profound Mental Status Examination; BPMSE-Ch-cog, Cognition subscale of Chinse version of the Baylor Profound Mental Status Examination; BPMSE-Ch-behav, Behavior subscale of Chinse version of the Baylor Profound Mental Status Examination; MMSE, Mini-Mental State Examination; SIB, Severe Impairment Battery; GDS1, Global Deterioration Scale; GDS2, Geriatric Depression Scale; IADL, Instrumental Activities of Daily Living; PSMS, physical self-maintenance scale; CDR, Clinical Dementia Rating; NPI, Neuropsychiatric Inventory.


Ceiling and floor effects

The relationship between BPMSE-Ch-cog and MMSE was revealed on a scatterplot (Supplementary Figure 1A). The range of 0 to 5 scores on the MMSE corresponded to a substantial range of 2 to 24 scores on BPMSE-Ch-cog, indicating that the BPMSE-Ch had no floor effect. In addition, it was found that patients scoring 12 to 16 on MMSE had the BPMSE-Ch-cog scores ranging from 20 to 25 (Mean: 23.08, SD: 1.08, Table 5). This demonstrated that BPMSE-Ch showed a ceiling effect among patients who were at a relative moderate level of dementia.


The relationship between BPMSE-Ch-cog and SIB scores is displayed (Supplementary Figure 1B). The relatively highR2=0.61 indicated that BPMSE-Ch-cog showed a strong association with the SIB, which demonstrated that the BPMSE-Ch was a sensitive tool for assessing patients with severe AD.

BPMSE-Ch-cog score stratified by MMSE levels

Table 5 presented that BPMSE-Ch-cog differentiated all the enrolled patients belonging to different severity groups according to the MMSE scores (F = 56.7, p <0.001). Patients in the MMSE Group 1 (range 16-12) had a BPMSE-Ch score of 23.08 ± 1.08, patients in the MMSE Group 2 (range 7-11) had a BPMSE-Ch score of 21.25 ± 3.53, and patients in the MMSE Group 3 (range 0-6) had a further reduced BPMSE-Ch score of 12.50 ± 6.69. From the results of Table 5, it was found that the differences in total BPMSE-Ch-cog score as well as in its four subcomponents scores between the Group 2 and Group 3 was significant (p < 0.001).

Table 5. Three severity groups according to the MMSE

Abbreviations: MMSE, Mini-Mental State Examination; BPMSE-Ch-cog, Cognition subscale of Chinse version of the Baylor Profound Mental Status Examination; SD, standard deviation; n = Number of patients. One-way ANOVA test. NS = Nonsignificant; 1. By Scheffé’s analysis


BPMSE-Ch-cog score stratified by CDR levels

BPMSE-Ch-cog differentiates the patients into different groups according to the CDR stage (F = 16.0, p < 0.001) (Supplementary Table 1). It was observed that the mean BPMSE-Ch-cog and subcomponents scores declined as the CDR stage increased (Supplementary Table 1). Furthermore, at Group 1 (CDR = 0.5), the total score of BPMSE-Ch-cog ranged from 23 to 25(Mean = 24.33, SD = 1.15); at Group 2 (CDR = 1), the total score of BPMSE-Ch-cog ranged from 19 to 25 (Mean = 22.86, SD = 1.42); at Group 3 (CDR = 2), the total BPMSE-Ch-cog score ranged from 2 to 25 (Mean = 19.82, SD = 5.59); at Group 4 (CDR = 3), the total BPMSE-Ch-cog score ranged from 2 to 24 (Mean = 12.50, SD = 7.29). It was observed that as the CDR stage increased, the corresponding range of BPMSE-Ch-cog became wide. Moreover, it was also shown that significant differences of total BPMSE-Ch-cog score and subcomponents scores existed between Group 3 and Group 4(Supplementary Table 1). All above suggested that BPMSE-Ch measured in a way different from CDR, and could differentiate levels of cognition at high CDR stages. Discussion The present study shows that BPMSE-Ch is a reliable, stable and valid instrument for assessing cognition in patients with severe AD. Internal consistency is robust, inter-rater reliability is near-perfect for both the BPMSE-Ch-cog and BPMSE-Ch-behav subscales, and test-retest reliability is also excellent. Furthermore, excellent construct validity was found referring to significant correlations with SIB (r=0.78), MMSE (r=0.76). These findings are consistent with the results of previous adoptions of Korean, Spanish, and Danish versions of the BPMSE. BPMSE-Ch-cog scores were strongly associated with MMSE, SIB ratings, indicating that the BPMSE-Ch-cog can differentiate well among patients with AD with differing degrees of cognitive impairment, particularly in the more severe end of the dementia spectrum, which of course is its primary intended use. In this regard, BPMSE-Ch-cog do not display floor effects in severely demented patients, as measured by the MMSE. Also, BPMSE-Ch-cog scores are strongly associated with SIB scores (while displaying a lower floor than the SIB), and its administration time is much shorter (only 5 minutes on average versus 30 minutes for the SIB). It further suggests that BPMSE-Ch is an efficient tool. Relative low correlations are also shown between BPMSE-Ch-cog scores and PSMS and IADL functional scores, demonstrating that the BPMSE-Ch can only partly measure cognitive abilities relevant to the abilities needed to function in daily life. We thought the possible reason is that the most enrolled patients would have reached maximum impairment of activities of daily living. It supposed that a certain degree of ceiling effects existed in IADL and PSMS tests. AlthoughGDS-1 is an available tool used to evaluate not only cognition but also the abilities to maintain daily life, participation in adverse activities and it is useful for the severe AD cases (25-27), it is a synthetic grade evaluation tool. The forced-choice format would place most enrolled patients into high stages. This might be the reason that the correlation between BPMSE-Ch and GDS-1 is low. Behavioral and psychological symptoms of dementia (BPSD) in patients with Alzheimer’s disease have a strong correlation with cognitive impairment and impairment in activities of daily living. NPI is a common tool for BPMSD evaluating. The BPMSE-Ch-behav selectively focused on disruptive behaviors. In this study, it has been found that there is a moderate correlation between BPMSE-Ch-behavand NPI. While the NPI is obtained by questions to the primary caregiver and is a complex and time-consuming process. Therefore, it indicated that BPMSE-Ch is also a relative practicable instrument to evaluate the behavioral and psychological symptoms in patients with severe dementia. The correlation between BPMSE-Ch-cog and GDS-2 is not significant (r = 0.16, p > 0.001). There are two possible reasons. Firstly, BPMSE-Ch-cog does not involve questions directed against depressive symptoms and is not intended to evaluate for depression. Secondly, it has been reported that patients with moderate-severe AD have relatively low GDS-2 scores (28), which is similar to our study. It suggests that patients with moderate-severe AD have no obvious depression symptoms. In our study, the highest GDS-2score seen was 24; therefore, GDS-2 sometimes shows a good complementary assessment for depression. Because the BPMSE measures clinical features distinct from the GDS-2 the absence of correlation is not surprising.
Regarding its suitability for use with severely impaired patients, it has been observed that the BPMSE-Ch-cog differentiates well between patients with MMSE scores 0-6 and those with MMSE scores 7-11, but not as well between patients with scores of 12-16 and those with scores 7-11. This indicates that the BPMSE-Ch, like its versions in other languages, is most appropriate to use with patients who are more severely impaired (with MMSE score of 11 or below). Similarly, analyses of patients in different CDR stages reveals that the total BPMSE-Ch score and subcomponent scores differ significantly between patients in CDR stage 2 versus those in CDR stage 3, and patients in both of these more severely impaired CDR stages exhibited a wide range of scores, with substantial variability. These results lend further support to the use of the BPMSE-Ch with severely impaired patients.
In conclusion, the BPMSE-Ch is a convenient, stable, reliable and valid scale to assess cognition in patients with moderate-severe AD, and is most appropriately used with patients who have MMSE scores 11 or below. And in future work, we should popularize the BPMSE-Ch in other areas of China including rural areas to research the properties about BPMSE. We believe that it would be beneficial for this instrument to be widely used for evaluating cognitive functioning of patients with severe AD in China.


Acknowledgments: Funding Information: This study was supported by grants from National Key R&D Program of China (2018YFC2001700), General Project of Technological Innovation and Application Development of Chongqing Science & Technology Bureau (cstc2019jscx-msxmX0239), Key project of Social undertakings and people’s livelihood security of Chongqing Science & Technology Commission (cstc2017shms-zdyfX0009) and Postgraduate Research Innovation Project of Chongqing(CYS16122), Particularly, we greatly thank Dr. Sergio Salmerón (Department of Geriatrics, Hospital General de Villarrobledo, Albacete, Spain) for the assistance in making a translation of BPMSE.

Conflict of Interest: The authors declare that they have no potential competing interests

Ethics approval and consent to participate: The study was approved by the Ethics Committee of The First Affiliated Hospital of Chongqing Medical University and has been performed in accordance with the ethical standards laid down in the Declaration of Helsinki and its later amendments.

Authors’ Contributions: Rachelle S. Doody and Yang Lü designed the study. Xue Fu, Weihua Yu and Yang Lü collected the data and wrote the paper. Yang Lü, Paul J. Massman and Rachelle S. Doody revised the manuscript: Xia Wang, Jia Zhang and Tao Luo analyzed data and assisted with writing the article.




1. J.C. Morris, K. Blennow, L. Froelich, et al. Harmonized diagnostic criteria for Alzheimer’s disease: recommendations, Journal of internal medicine2014; 275(3): 204-13.
2. F. Li, S. Chen, C. Wei, J. Jia. Monetary costs of Alzheimer’s disease in China: protocol for a cluster-randomised observational study, BMC neurology 2017; 17(1):15.
3. J. Jia, A. Zhou, C. Wei, et al. The prevalence of mild cognitive impairment and its etiological subtypes in elderly Chinese, Alzheimer’s & dementia : the journal of the Alzheimer’s Association 2014; 10(4): 439-47.
4. Y. Zhang, Y. Xu, H. Nie, et al. Prevalence of dementia and major dementia subtypes in the Chinese populations: a meta-analysis of dementia prevalence surveys, 1980-2010, Journal of clinical neuroscience : official journal of the Neurosurgical Society of Australasia 2012;19(10): 1333-7.
5. K.Y. Chan, W. Wang, J.J. Wu, et al. Epidemiology of Alzheimer’s disease and other forms of dementia in China, 1990-2010: a systematic review and analysis, Lancet (London, England) 2013; 381(9882): 2016-23.
6. J. Jia, F. Wang, C. Wei, et al. The prevalence of dementia in urban and rural areas of China, Alzheimer’s & dementia : the journal of the Alzheimer’s Association 2014, 10(1): 1-9.
7. D. Liu, G. Cheng, L. An, et al. Public Knowledge about Dementia in China: A National WeChat-Based Survey, International journal of environmental research and public health2019; 16(21).
8. X. Li, W. Fang, N. Su, Y. Liu, S. Xiao, Z. Xiao, Survey in Shanghai communities: the public awareness of and attitude towards dementia, Psychogeriatrics : the official journal of the Japanese Psychogeriatric Society2011; 11(2): 83-9.
9. M. Zhao, X. Lv, M. Tuerxun, et al. Delayed help seeking behavior in dementia care: preliminary findings from the Clinical Pathway for Alzheimer’s Disease in China (CPAD) study, International psychogeriatrics 2016; 28(2): 211-9.
10. D. Peng, Z. Shi, J. Xu, et al. Demographic and clinical characteristics related to cognitive decline in Alzheimer disease in China: A multicenter survey from 2011 to 2014, Medicine2016; 95(26): e3727.
11. F.A. Schmitt, W. Ashford, C. Ernesto, et al. The severe impairment battery: concurrent validity and the assessment of longitudinal change in Alzheimer’s disease. The Alzheimer’s Disease Cooperative Study, Alzheimer disease and associated disorders1997; 11 Suppl 2: S51-6.
12. M.F. Folstein, S.E. Folstein, P.R. McHugh, «Mini-mental state». A practical method for grading the cognitive state of patients for the clinician, Journal of psychiatric research 1975; 12(3): 189-98.
13. J. Saxton, A.A. Swihart, Neuropsychological assessment of the severely impaired elderly patient, Clinics in geriatric medicine 1989; 5(3): 531-43.
14. S.J. Cano, H.B. Posner, M.L. Moline, et al. The ADAS-cog in Alzheimer’s disease clinical trials: psychometric evaluation of the sum and its parts, Journal of neurology, neurosurgery, and psychiatry2010; 81(12): 1363-8.
15. B. Reisberg, S.H. Ferris, M.J. de Leon, T. Crook. The Global Deterioration Scale for assessment of primary degenerative dementia, The American journal of psychiatry1982; 139(9): 1136-9.
16. B. Reisberg. Functional assessment staging (FAST), Psychopharmacology bulletin 1988; 24(4): 653-9.
17. C.P. Hughes, L. Berg, W.L. Danziger, L.A. Coben, R.L. Martin, A new clinical scale for the staging of dementia, The British journal of psychiatry : the journal of mental science 1982; 140: 566-72.
18. G.M. Peavy, D.P. Salmon, V.A. Rice, et al. Neuropsychological assessment of severely demeted elderly: the severe cognitive impairment profile, Archives of neurology 1996; 53(4): 367-72.
19. K.L. Lanctot, J. Amatniek, S. Ancoli-Israel, et al. Neuropsychiatric signs and symptoms of Alzheimer’s disease: New treatment paradigms, Alzheimer’s & dementia (New York, N. Y.)2017; 3(3): 440-449.
20. R.S. Doody, S.L. Strehlow, P.J. Massman, E.P. Feher, C. Clark, J.R. Roy, Baylor profound mental status examination: a brief staging measure for profoundly demented Alzheimer disease patients, Alzheimer disease and associated disorders 1999; 13(1): 53-9.
21. A. Korner, A. Brogaard, I. Wissum, U. Petersen, The Danish version of the Baylor Profound Mental State Examination, Nordic journal of psychiatry2012; 66(3): 198-202.
22. H.R. Na, S.H. Lee, J.S. Lee, R.S. Doody, S.Y. Kim, Korean version of the Baylor Profound Mental Status Examination: a brief staging measure for patients with severe Alzheimer’s disease, Dementia and geriatric cognitive disorders 2009; 27(1): 69-75.
23. S. Salmeron, I. Huedo, M. Lopez-Utiel, et al. Validation of the Spanish version of the Baylor Profound Mental Status Examination, Journal of Alzheimer’s disease 2016; 49(1): 73-8.
24. G. McKhann, D. Drachman, M. Folstein, R. Katzman, D. Price, E.M. Stadlan, Clinical diagnosis of Alzheimer’s disease: report of the NINCDS-ADRDA Work Group under the auspices of Department of Health and Human Services Task Force on Alzheimer’s Disease, Neurology1984; 34(7): 939-44.
25. R.H. Paul, R.A. Cohen, D.J. Moser, et al. The global deterioration scale: relationships to neuropsychological performance and activities of daily living in patients with vascular dementia, Journal of geriatric psychiatry and neurology 2002; 15(1): 50-4.
26. J.S. Kim, C.W. Won, B.S. Kim, H.R. Choi, Predictability of various serial subtractions on global deterioration scale according to education level, Korean journal of family medicine2013; 34(5): 327-33.
27. S.H. Choi, B.H. Lee, S. Kim, et al. Interchanging scores between clinical dementia rating scale and global deterioration scale, Alzheimer disease and associated disorders2003; 17(2): 98-105.
28. A.J. Midden, B.T. Mast, Differential item functioning analysis of items on the Geriatric Depression Scale-15 based on the presence or absence of cognitive impairment, Aging & mental health 2017; 1-7.


I. McRae1, L. Zheng2,4, S. Bourke3, N. Cherbuin1, K.J. Anstey2,4

1. Centre for Research on Ageing Health and Wellbeing, Research School of Population Health, The Australian National University, Canberra, ACT, Australia; 2. Neuroscience Research Australia, Margarete Ainsworth Building, Barker Street, Randwick, Sydney NSW, Australia; 3..Department of Health Services Research and Policy, Research School of Population Health, The Australian National University, Canberra, ACT, Australia; 4. Ageing Futures Institute, School of Psychology, University of New South Wales, Sydney, NSW, Australia

Corresponding Author: Dr Ian McRae, Centre for Research on Ageing Health and Wellbeing, Research School of Population Health, The Australian National University, Canberra, ACT 2600, Australia, Email:, Ph: +61 431 929 750

J Prev Alz Dis 2020;
Published online December 15, 2020,



Background: Assessment of cost-effectiveness of interventions to address modifiable risk factors associated with dementia requires estimates of long-term impacts of these interventions which are rarely directly available and must be estimated using a range of assumptions.
OBJECTIVES: To test the cost-effectiveness of dementia prevention measures using a methodology which transparently addresses the many assumptions required to use data from short-term studies, and which readily incorporates sensitivity analyses.
DESIGN: We explore an approach to estimating cost-effective prices which uses aggregate data including estimated lifetime costs of dementia, both financial and quality of life, and incorporates a range of assumptions regarding sustainability of short- term gains and other parameters.
SETTING: The approach is addressed in the context of the theoretical reduction in a range of risk factors, and in the context of a specific small-scale trial of an internet-based intervention augmented with diet and physical activity consultations.
MEASUREMENTS: The principal outcomes were prices per unit of interventions at which interventions were cost-effective or cost-saving.
RESULTS: Taking a societal perspective, a notional intervention reducing a range of dementia risk-factors by 5% was cost-effective at $A460 per person with higher risk groups at $2,148 per person. The on-line program costing $825 per person was cost-effective at $1,850 per person even if program effect diminished by 75% over time.
CONCLUSIONS: Interventions to address risk factors for dementia are likely to be cost-effective if appropriately designed, but confirmation of this conclusion requires longer term follow-up of trials to measure the impact and sustainability of short-term gains.

Key words: Dementia, risk factors, cost-effectiveness, interventions, sustainability.



While many studies have addressed the association of lifestyle and vascular factors with dementia, few have addressed whether interventions designed to reduce risk factors are cost-effective (1). This is in part because dementia risk reduction programs are implemented well before the usual age of dementia onset. This means that economic evaluation using simulation modelling requires parameters relevant to a long-term time frame. As most intervention studies to date have 5 years or less of follow-up (1) (exceptions include the planned trials of multi-domain interventions (2)), cost-effectiveness studies require model parameters to be extrapolated well beyond the data observed. Reviews of model-based economic evaluations of dementia interventions (1, 3) have identified very few methods which assess prevention strategies. The types of non-pharmaceutical interventions identified in these reviews mainly focused on early assessment of dementia, screening or diagnosis rather than reduction in risk factors (3).
Short-term cost-effectiveness studies (4) and methodologies have been published which address cost-effectiveness of transitions from mild cognitive impairment (MCI) to dementia (5). However, assessing cost-effectiveness of programs which reduce or treat risk factors (many of which occur in mid-life) requires modelling the impact of interventions over longer time frames (1, 3, 5-9) and requires assumptions on how trial results are sustained over the longer term. In the absence of robust estimates of many of the parameters needed for full Markov or other simulation models, we suggest an alternate approach to estimating the price at which programs are cost-effective. This approach provides transparency in estimating the sensitivity of these prices with highly uncertain parameter estimates.
A 2019 review of health economic evaluations of primary prevention programs for dementia (1) identified three analyses of prevention strategies (6, 8, 9) which modelled dementia progress and costs over the long-term. Noting the range of uncertainties, the review recommended that “extensive sensitivity analysis to examine the impact of assumptions” be implemented. This included assumptions regarding long-term vs short-term outcomes of interventions, the impact of optimal program targeting, and discounting (1). Two of the analyses were partial evaluations which addressed potential cost savings from reductions in dementia levels, but did not address health benefits (usually measured by Quality Adjusted Life Years (QALYs)), so cost-effectiveness was not testable(6, 8). While including an extensive sensitivity analysis, the study which addressed cost-effectiveness (9) required a range of assumptions to estimate parameters including annual risk rates, mortality rates for those with and without dementia and QALY levels by age for people with dementia (9).
Estimated age/gender specific incidence rates(10) for dementia are available for Australia, but the impact of interventions on incidence of dementia at each age is not known, nor are age-specific costs or QALY estimates. Hence, there is value in exploring non-simulation approaches to estimate the cost-effectiveness of interventions which address dementia risk factors using aggregate data. We use an approach based on average lifetime costs of dementia and losses in quality of life per individual who develops dementia. Until long-term parameters can be obtained with confidence, this approach avoids the need for transition probabilities and cost and QALY measures by age. It also gives a direct means of linking costs and benefits and provides a transparent means of undertaking sensitivity analyses of all factors, including parameters reflecting the sustainability of improvements in risk factors, program targeting and discounting.
To demonstrate the proposed approach we draw on two examples (11, 12): (1) a study that estimated the effects of risk reduction through population attributable risk(PAR) and (2) a recent randomized control trial (RCT) which assessed the impact of an on-line dementia prevention program. The RCT has a relatively short follow-up (15 months), so to estimate the long term cost-effective and cost-saving prices we provide a range of different assumptions including the degree to which gains in risk reduction are sustained and how well the program is targeted to people with high likelihoods of progressing to dementia.


Methods and Data


We used available estimates of the proportion of adults aged 65 and over who are expected to develop dementia and then estimated the reduction in prevalence of dementia for a target population from the two example interventions. Savings in costs and QALYs per person generated by the interventions were estimated using the average per person life-time costs of dementia and loss of QALY due to dementia. This enabled us to estimate the maximum price per person for an intervention to be cost saving or cost effective.
The standard measure of cost-effectiveness (technically cost-utility) is the incremental cost per QALY gained (i.e. the Incremental Cost-Effectiveness Ratio or ICER). For the purposes of this study, an intervention with an ICER below $50,000 is considered cost-effective. While Australia has no formal ICER thresholds this is the level most commonly quoted and is consistent with UK, Australian (13) and American(14) literature.
Apart from sensitivity analysis for uncertain parameters, we examined: (1) the impact of program targeting, as an intervention targeted at the highest risk groups has a greater opportunity to reduce dementia prevalence, (2) the impact of “decay” which reflects reduction in the gains from an intervention over time, and (3) the impact of different levels of discounting. Discounting is a means of “valuing down” (1) future financial and health costs as people may prefer to save money (or gain health benefits) now rather than in the future.

Lifetime Costs of Developing Dementia

Lifetime costs for people with dementia are the product of average annual costs of treatment/care and the duration of care. While estimates of duration of dementia vary widely depending mainly on age at diagnosis, international evidence and reviews suggest that a mean of 5 years is appropriate for the duration of care for dementia (15, 16) (noting this may not be the same as the actual duration of dementia) (17)).
The available estimates of costs of dementia take several perspectives. An American study (18) including direct healthcare costs and costs of informal care estimated $260,000 per person in 2015 (all costs in Australian Dollars); while a 2016 Australian analysis found average annual costs of $35,550 per person including indirect costs such as loss of productivity of both patients and carers (10). With 5 years life with dementia, this becomes $177,750 lifetime cost per person. A later Australian study (19) of people with dementia in residential care with a markedly different methodology estimated higher annual costs of $88,000 per year for residential care compared to $55,000 from the earlier study(10). Given the varying results from these studies, we used a figure of $200,000 as baseline, with a range from $150,000 to $300,000 used for sensitivity analyses.

Loss of Quality Adjusted Life Years by People Developing Dementia

The lifetime loss of QALYs for people with dementia includes the loss due to poorer quality of life and the loss due to premature mortality. A conservative median estimated years of life lost to dementia used here as a baseline is 5 years. This is consistent with previous studies (20) and an Australian systematic review (15). Some estimates as high as 9 years of life have been found (7, 21); we use this as the upper limit for sensitivity analysis purposes.
Few generally applicable estimates of QALY values for people with dementia are available (22). Most studies deriving QALYs in a dementia context relate to specific RCTs with specific populations rather than comparing average people with and without dementia. We draw on estimates of average QALYs for people with dementia and the wider aged population (7, 23). With 5 years of life with dementia, and 5 years loss of life due to dementia there is an average loss 1.5 QALYs while alive and 4.2 QALYS due to premature mortality giving a lifetime loss of 5.7 QALYs from the dementia. A previous estimate (7) based on 6 years with dementia and 9 years loss of life led to an estimated 9.4 QALYs lost which we use as an upper level for sensitivity testing.

Prevalence of dementia

Population prevalence data is not required for Example 1 as the predicted outcomes are explicitly in prevalence terms, although it is required for Example 2. While “Australian data on dementia prevalence are lacking” (AIHW 2018 p138 (24)), we use an estimate of 10% for people aged 65 or over from a study using Australian data (10), which is marginally above estimates combining Australian and international data (10, 24). For Example 2, we assume that any reduction in risk will lead to an equivalent reduction in prevalence when the cohort reaches age 65 or over, and that this reduction will apply to the estimated 10% prevalence of dementia in this age group.


People generally value future costs and effects less than current costs and effects and the value diminishes the further into the future they are expected to occur (25). Hence, economic evaluations adjust the value of costs and benefits for the time at which they occur, using discounting (25). Discounting over long periods has major impacts on results of cost-effectiveness studies (1), particularly when comparing program costs at midlife to medical and other savings in later life (26). A range of discount levels are used by different organisations including: a) the use of 3% for both costs and QALYs (9), b) discount rates of 4% for costs and 1.5% for QALYs( 1), c) the use of 5% for both costs and QALYs in Australia by the Medicare Services Advisory Committee (25), and d) a UK recommendation that 3.5% be applied to both costs and QALYs (25).
In the light of extremely low interest rates in Australia and many other countries at present, and the long durations of discounting in this study, we use baseline discount rates of 3% for both costs and QALYs. For sensitivity analysis we include the Australian standard of 5% for both costs and QALYs, and the 4%/1.5% applied in Holland (1).
Simulation approaches apply discounting each year. However, assuming on average no differences between treated and untreated groups before onset of dementia, the discounting will have no material impact on the differences between treatment groups prior to diagnosis (note that while in principle costs change at onset, they are only measured from diagnosis). We, therefore, discount from the average age of commencement of the intervention to approximately the mid-point of the dementia period. To establish the period of discounting we take an average age of diagnosis as being in the early 80s (27-29). Most studies addressing average age at diagnosis show averages from the high 70s to mid 80s, but most commence with aged populations which may lead to some upward bias. We, therefore, include some alternate discounting periods for sensitivity analysis.

Example 1 – Estimates of Dementia Prevention using Population Attributable Risk

Ashby-Mitchell et al.(2017) (11) explored the aggregate Population Attributable Risk (PAR) from a set of known correlates of dementia (midlife obesity, physical inactivity, smoking, low educational attainment, diabetes mellitus, midlife hypertension, depression). They used PAR values to estimate the impact of uniform reductions in these correlates on dementia prevalence. They concluded that a uniform 5% improvement across all risks would, over 20 years, lead to a reduction in the prevalence of dementia of 3.2% or 17,454 people in Australia.
Any intervention which aimed to reduce the risk factors addressed in Example 1 would need to improve obesity levels and hypertension in mid-life so we assume an intervention targeted at the population aged 45 years and over with an average age of around 65 years. Consistent with the modelling in Example 1 this gives a 20-year period from average age at intervention to average age of dementia diagnosis (early 80s) which we use for discounting (15 years used for sensitivity testing).

Example 2 – BBL-GP Intervention

The Body-Brain-Life in General Practice program (BBL-GP) aims to reduce known dementia risk factors using a mixture of on-line training and face-to-face consultations with dietitians and exercise physiologists (12). Results are assessed using an aggregate measure combining a range of known risk factors (the ANU-ADRI (30)) with program participants compared to an active control group. After 62 weeks the BBL-GP participants showed a decline in ANU-ADRI scores of 4.62 units more than the active controls (12). For a population of Australians aged 60-64 years at baseline, a difference in baseline values of 1 point of ANU-ADRI is associated with a difference of 8% in people developing mild cognitive decline (MCI) or dementia after 12 years (31). This suggests a BBL-GP effect of 37% if the 4.62 units improvement is sustained.
This is an upper limit. Firstly, it is unlikely all the gains in risk factors will be sustained (e.g. maintaining weight loss). Secondly, the evidence of the impact of one point of ANU-ADRI on MCI and dementia may be the same as the long-term impact on dementia, but need not be, as there is likely to be a bias towards reducing MCI in those who are least likely to go forward to dementia. In this case the 8% impact of one ADRI point would be an overstatement. Finally, it is not clear if differences in the index obtained from an intervention have the same effect as differences brought about by lifetime experiences. The size of “decay” for any particular intervention is, therefore, driven by a range of factors including the time period between the intervention and the age at which dementia diagnosis is likely. For sensitivity analysis we test a range of different levels of reduction in impact of the BBL-GP program on actual dementia risk, beginning with a 50% reduction and increasing to a 95% reduction. We term this “decay” to reflect both the difficulty in sustaining the intervention’s short-term gains and the other issues described.
The trial population in Example 2 had an average age of 51 years (12), so for discounting purposes there is approximately 30 years to the average age of dementia diagnosis (20 years used for sensitivity testing). The average cost per participant in the BBL-GP trial relative to an active control was $2,700 including set-up costs. The number of participants in this trial was small, and while there are fixed costs of around $200 per person, other expenditures was almost independent of participant numbers. If more fully implemented the program would be expected to be at least quadrupled in size and costs would become $825 per person. We use this figure to assess cost-effectiveness. With a larger implementation, average costs would be further reduced.



Table 1 shows baseline estimates for Example 1 with a target population of all people aged 45 years or over. This suggests that, ignoring program costs and discounting, over the lifetimes of the people protected from dementia by the lifestyle changes there would be savings of $3.5b and 99,488 QALYs. While these savings are large, with a targeting across the whole population, the savings per targeted person are only $342. After allowing for discounting, the maximum cost per targeted person which could lead to a cost saving program is $189, while a cost less than $460 would achieve a cost-effective incremental cost per QALY gained (the ICER) of less than $50,000.

Table 1. Example 1 – PAR – Baseline costing

1. (10)= ((4) + (9)*(5))/(6)
Table 2 provides estimates of maximum costs per person for a program to be cost saving or cost-effective under different assumptions on target size, lifetime costs, QALY losses and discount rates. Tests 1-3 show relatively little sensitivity in cost-effective or cost-saving prices to changes in estimated lifetime costs and lifetime QALY losses to dementia, with greater effects of QALY increases than cost increases on the cost-effective price. Test 4 assumes the intervention targets only half the population aged 45 and over and assumes the targeting is so well focused on those at higher risk that the number of people avoiding dementia is unchanged. This generates a much greater change in the maximum acceptable costs than shown in Tests 1-3. Test 5 assumes an intervention targeted at a population of only 10,000 who are at very high risk of developing dementia (25% prevalence rate), and again with 3.2% of the anticipated cases “saved” from dementia (11). The cost-effective price increases to $2,069 (after discounting), more than 4 times the baseline estimate. With such precise targeting the percentage saved would probably be greater than 3.2%, and any increase in this parameter would increase the cost-effective prices proportionately. Table 2 also shows the impact of different discounting rates, with the 4%/1.5% levels having broadly similar results to baseline, but the 5%/5% showing acceptable prices around 60% of 3%/3% meaning interventions are considerably less likely to be cost-effective. Should the duration of discounting (the period from the intervention to average age of diagnosis) be reduced, for the 3%/3% calculation the maximum cost-effective price would increase by 15% meaning more expensive interventions would be cost-effective.

Table 2. Example 1 – PAR Costings – Sensitivity analyses

NOTE: * shows variation from baseline
Table 3 provides baseline estimates for Example 2. For presentation purposes the assumed population is 10,000 but results are independent of this number. The discounted program prices at baseline of $3,052 per person to be cost saving and $7,401 per person for the program to be cost-effective are well above the average price per participant of $825 relative to the active control.

Table 3. Example 2- BBL-GP – Baseline Costing

Table 4 provides sensitivity testing which in addition to the factors tested for Example 1 tests levels of “decay”, and shows that the targeting, decay and discounting assumptions have the greatest impact on the overall outcomes. The targeting level of 60% was chosen as the trial participants were mainly people with obesity, with the relative risk of developing dementia of 1.6 (11). With an average price of $825, results discounted at 3% and all other factors at baseline level, a decay of up to 88% would be cost-effective, although not 95% (Test 3). With a 60% loading for targeting and the maximum levels of cost savings from preventing dementia and QALY lost to dementia, the intervention would be cost-effective at 95% decay (Test 7). Test 8 shows that with the 60% loading for targeting and other factors at baseline, even at 93% decay from the short term results the program would be cost-effective.

Table 4. Example 2 – BBL-GP Costing – Sensitivity Analyses

NOTE: * shows variation from baseline
The patterns in these tables show that results are linear with respect to both targeting and “decay”, and less than linear with respect to estimated lifetime costs and QALYs lost to dementia. As for Example 1, discounting has a major impact on the results, although even with relatively high levels of “decay” (80% with all other factors at baseline) the intervention is likely to remain cost-effective with 5%/5% discounting. Should the duration of discounting be reduced, the maximum cost-effective price would increase by 34% for the 3%/3% discounting, although this does not lift any of these prices above $825 for the examples in Table 4.



Our results suggest that multi-domain programs such as the BBL-GP in Example 2 are likely to be cost-effective (unless program impacts decay almost completely over time), while the more generic approach of Example 1 requires tight targeting to at-risk populations to be cost-effective. These results are consistent with prior studies (1, 32) in showing the importance of targeting and sustainability of observed results beyond the period of study follow-up.
The estimated cost of $825 per person in Example 2 would be reduced with wider implementation. Previous studies have estimated cost of dementia risk reduction programs of $200 to $500 per person (9, 33). If Example 2 could be conducted at these lower costs it is more likely to be cost-effective even at high levels of “decay”. Should the duration from intervention to diagnosis of dementia be less than the assumed levels, the effect of discounting would be reduced, and maximum cost-effective program prices increased.
Recalling that “decay” includes other factors as well as the need for participants to maintain lifestyle changes over many years, high levels of decay are possible. Studies with long follow-up are needed to assess actual program effects. Programs which continue to interact with the participants continuously over time are likely to improve effects but increase costs. We also note that improving dementia risk factors would improve a range of other health outcomes (e.g. cardiovascular health, diabetes, mild cognitive impairment), in addition to dementia related outcomes. If the total benefits of risk reduction programs were included, they would be even more likely to be cost-effective.


The main limitation in this and any other analysis of cost-effectiveness of dementia prevention interventions is the uncertainty in many parameters, which has required extensive sensitivity analysis to assess a reasonable range of outcomes. However, the approach taken here integrates sensitivity analysis and facilitates estimation of outcomes under varied assumptions.
The study assumed binary outcomes of dementia against no dementia and did not address the benefit of delay in onset of dementia, which also reduced the likelihood of finding cost-effective outcomes. Dementia related QALY losses prior to diagnosis were not included in the study, leading to a further conservative bias in estimates.
Like all approaches to cost-effectiveness modelling for dementia prevention interventions this study is limited by having only short-term program outcomes (1). The baseline calculations assume (1) in the case of Example 1, that well-established associations between risk factors and dementia are causative; (2) for both examples, changes in risk factors driven by interventions have the same effect as if the level of the risk factor was achieved ”naturally” (e.g. reversing midlife obesity with an intervention has the same effect as achieving a normal weight at midlife without intervention) and; (3) changes in risk from a short-term intervention are sustained over time(e.g. weight does not revert to previous levels). The approach used here however provides a simple and transparent way to test the impact of these ongoing concerns.



To explore the cost-effectiveness of interventions aimed at dementia risk reduction requires a means of extrapolating outcomes from what, to date, have been relatively short-term trials. We examined lifetime costs (in both dollar and QALY terms) of dementia and applied these to projected changes in risks of dementia from two example studies. The results suggest that the multi-domain approach of BBL-GP is highly likely to be cost-effective.
The approach shows further the importance of targeting programs to “at risk” portions of the population and the sensitivity to the sustainability or otherwise of trial results. While these factors are well-known, the approach provides a means of estimating the orders of magnitude of program impacts and reinforces the need for longer-term studies to measure all relevant factors to enable assessment of cost-effectiveness with greater confidence.


Funding Sources: This research was undertaken as part of the Centre for Research Excellence in Cognitive Health, which was funded by the National Health and Medical Research Council grant #1100579. Anstey is funded by NHMRC Fellowship #1102694, Zheng is part supported by the NHMRC Dementia Centre for Research Collaboration. The funders had no role in the design and conduct of this study; in the analysis and interpretation of the data; in the preparation of the manuscript; or in the review or approval of the manuscript.
Acknowledgements: We acknowledge the ARC Centre of Excellence in Population Ageing Research.

Conflict of Interest: Dr McRae, Dr Zheng, Dr Bourke, and Professor Cherbuin declare that they have no conflict of interest. Professor Anstey reports personal fees from StaySharp, outside the submitted work.

Ethical standards: The authors followed the ethical guidelines of the Journal for this manuscript.



1. Handels R, Wimo A. Challenges and recommendations for the health-economic evaluation of primary prevention programmes for dementia. Aging & mental health. 2019 Jan;23(1):53-9.
2. Kivipelto M, Mangialasche F, Ngandu T. World Wide Fingers will advance dementia prevention. Lancet Neurol. 2018 Jan;17(1):27.
3. Nguyen KH, Comans TA, Green C. Where are we at with model-based economic evaluations of interventions for dementia? a systematic review and quality assessment. International psychogeriatrics. 2018 Nov;30(11):1593-605.
4. Meeuwsen E, Melis R, van der Aa G, Golüke-Willemse G, de Leest B, van Raak F, et al. Cost-effectiveness of one year dementia follow-up care by memory clinics or general practitioners: economic evaluation of a randomised controlled trial. PloS one. 2013;8(11):e79797-e.
5. Green C, Handels R, Gustavsson A, Wimo A, Winblad B, Skoldunger A, et al. Assessing cost-effectiveness of early intervention in Alzheimer’s disease: An open-source modeling framework. Alzheimer’s & dementia : the journal of the Alzheimer’s Association. 2019 Oct;15(10):1309-21.
. Lin PJ, Yang Z, Fillit HM, Cohen JT, Neumann PJ. Unintended benefits: the potential economic impact of addressing risk factors to prevent Alzheimer’s disease. Health affairs (Project Hope). 2014 Apr;33(4):547-54.
7. Tsiachristas A, Smith AD. B-vitamins are potentially a cost-effective population health strategy to tackle dementia: Too good to be true? Alzheimer’s & Dementia: Translational Research & Clinical Interventions. 2016;2(3):156-61.
8. van Baal PH, Hoogendoorn M, Fischer A. Preventing dementia by promoting physical activity and the long-term impact on health and social care expenditures. Preventive medicine. 2016 Apr;85:78-83.
9. Zhang Y, Kivipelto M, Solomon A, Wimo A. Cost-effectiveness of a health intervention program with risk reductions for getting demented: results of a Markov model in a Swedish/Finnish setting. Journal of Alzheimer’s disease : JAD. 2011;26(4):735-44.
10. Brown L, Hansnata E, La HA. Economic Cost of Dementia in Australia 2016-2056, Report Prepared for Alzheimer’s Australia. Canberra: NATSEM at the Institute for Governance and Policy Analysis, University of Canberra2017.
11. Ashby-Mitchell K, Burns R, Shaw J, Anstey KJ. Proportion of dementia in Australia explained by common modifiable risk factors. Alzheimer’s Research & Therapy. [journal article]. 2017 February 17;9(1):11.
12. Anstey KJ, Kim S, Pond CD, Cherbuin N, McMaster M, Lautenschlager N, et al. Internet-based Intervention Augmented with Diet and Physical Activity Consultation to Decrease Risk of Dementia in At-risk Adults in a Primary Care Setting: Pragmatic Randomized Controlled Trial. Journal of Medical Internet Research. 2020;Accepted for Publication.
13. Wang S, Gum D, Merlin T. Comparing the ICERs in Medicine Reimbursement Submissions to NICE and PBAC—Does the Presence of an Explicit Threshold Affect the ICER Proposed? Value in Health. 2018 2018/08/01/;21(8):938-43.
14. Neumann PJ, Cohen JT, Weinstein MC. Updating Cost-Effectiveness — The Curious Resilience of the $50,000-per-QALY Threshold. The New England Journal of Medicine. 2014 2014 Aug 28;371(9):796-7.
15. Brodaty H, Seeher K, Gibson L. Dementia time to death: a systematic literature review on survival time and years of life lost in people with dementia. International psychogeriatrics. 2012 Jul;24(7):1034-45.
16. Sachs GA, Carter R, Holtz LR, Smith F, Stump TE, Tu W, et al. Cognitive impairment: an independent predictor of excess mortality: a cohort study. Annals of internal medicine. 2011 Sep 6;155(5):300-8.
17. Savva GM, Arthur A. Who has undiagnosed dementia? A cross-sectional analysis of participants of the Aging, Demographics and Memory Study. Age and Ageing. 2015;44(4):642-7.
18. Jutkowitz E, Kane RL, Gaugler JE, MacLehose RF, Dowd B, Kuntz KM. Societal and Family Lifetime Cost of Dementia: Implications for Policy. J Am Geriatr Soc. 2017 Oct;65(10):2169-75.
19. Gnanamanickam ES, Dyer SM, Milte R, Harrison SL, Liu E, Easton T, et al. Direct health and residential care costs of people living with dementia in Australian residential aged care. International journal of geriatric psychiatry. 2018;33(7):859-66.
20. Haaksma ML, Eriksdotter M, Rizzuto D, Leoutsakos J-MS, Olde Rikkert MGM, Melis RJF, et al. Survival time tool to guide care planning in people with dementia. Neurology. 2020;94(5):e538-e48.
21. Strand BH, Knapskog AB, Persson K, Edwin TH, Amland R, Mjorud M, et al. Survival and years of life lost in various aetiologies of dementia, mild cognitive impairment (MCI) and subjective cognitive decline (SCD) in Norway. PloS one. 2018;13(9):e0204436.
22. Prince MJ, Wimo A, Guerchet MM, et al. World Alzheimer Report 2015- The Global Impact of Dementia London: Alzheimer’s Disease International2015.
23. Orgeta V, Edwards RT, Hounsome B, Orrell M, Woods B. The use of the EQ-5D as a measure of health-related quality of life in people with dementia and their carers. Qual Life Res. 2015;24(2):315-24.
24. Australian Institute of Health and Welfare. Australia’s health 2018. Canberra: Australian Institute of Health and Welfare2018.
25. Attema AE, Brouwer WBF, Claxton K. Discounting in Economic Evaluations. Pharmacoeconomics. 2018;36(7):745-58.
26. Devlin N, Scuffham P. Health today versus health tomorrow: does Australia really care less about its future health than other countries do? Aust Health Rev. 2020 Jun;44(3):337-9.
27. Plassman BL, Langa KM, McCammon RJ, Fisher GG, Potter GG, Burke JR, et al. Incidence of dementia and cognitive impairment, not dementia in the United States. Ann Neurol. 2011;70(3):418-26.
28. Brinks R, Landwehr S, Waldeyer R. Age of onset in chronic diseases: new method and application to dementia in Germany. Population Health Metrics. 2013 2013/05/02;11(1):6.
29. Wolters FJ, Tinga LM, Dhana K, Koudstaal PJ, Hofman A, Bos D, et al. Life Expectancy With and Without Dementia: A Population-Based Study of Dementia Burden and Preventive Potential. Am J Epidemiol. 2019 Feb 1;188(2):372-81.
30. Anstey KJ, Cherbuin N, Herath PM, Qiu C, Kuller LH, Lopez OL, et al. A self-report risk index to predict occurrence of dementia in three independent cohorts of older adults: the ANU-ADRI. PloS one. 2014;9(1):e86141.
31. Cherbuin N, Shaw ME, Walsh E, Sachdev P, Anstey KJ. Validated Alzheimer’s Disease Risk Index (ANU-ADRI) is associated with smaller volumes in the default mode network in the early 60s. Brain imaging and behavior. 2019 Feb;13(1):65-74.
32. Richard E, Andrieu S, Solomon A, Mangialasche F, Ahtiluoto S, Moll van Charante EP, et al. Methodological challenges in designing dementia prevention trials – the European Dementia Prevention Initiative (EDPI). Journal of the neurological sciences. 2012 Nov 15;322(1-2):64-70.
33. Clare L, Nelis SM, Jones IR, Hindle JV, Thom JM, Nixon JA, et al. The Agewell trial: a pilot randomised controlled trial of a behaviour change intervention to promote healthy ageing and reduce risk of dementia in later life. BMC Psychiatry. 2015 2015/02/19;15(1):25.