Reply

We thank Verma et al for their careful consideration of our recent publication titled ‘The usability and reliability of a smartphone application for monitoring future dementia risk in ageing UK adults’. ^{Reference Sithara, Mathur and Verma1} In continuation, we address each of the comments that they provide.

Using a proprietary scale rather than a standardised one (e.g. the system usability scale (SUS))

We acknowledge the potential bias of using a non-validated subjective measure of usability; however, we also used several objective measures to complement the description of usability/acceptability:

• • assessment completion rates

• • time taken to complete each assessment

• • test–retest reliability

By using the sum of both subjective and objective measures, and not one single measure in isolation, we were able to evaluate more facets of the acceptability and usability of this system.

We thank Verma et al for directing our attention to the SUS. Indeed, this scale has undergone extensive validation testing and could be an improvement on our approach to assess the subjective usability of the Five Lives smartphone application. We plan to incorporate this scale in future studies.

Subjective usability scale measurement and reporting

Verma et al claim that there is a discrepancy between the described method and the reported results concerning the subjective measures of usability. On this point, we respectfully disagree. The subjective usability of the app was assessed in several ways. The primary subjective assessment included three questions with responses measured on a 10-point Likert scale:

• • How would you rate the app in terms of its ease of use?

• • How did you find the length of the assessment?

• • How did you find the difficulty of the assessment?

The results of this assessment are correctly reported as averages of the 10-point scale in the results section of the article, p. 248, first paragraph. There were also secondary subjective measures of usability which involved 5-point Likert scale questions and a word frequency analysis of open-ended questions. These are also correctly reported in Supplementary Material 6 (for the 5-point Likert-style questions) and the main body of the article, p. 248, first paragraph (for the word frequency analysis).

Linear regression sample size – analysis of cognitive tests

Verma et al correctly point out a difference between the sample sizes reported for each linear regression in Table 2. The Breeze test linear regression has a lower sample size for the same reasons as the Cast test – the test was re-developed during the study and we analysed only the scores coming from the latest version. While we reported the reasons for the reduced sample size for Cast in the supplementary material, we unintentionally omitted this information for Breeze. The changes in the Breeze test involved updating the logic of which word categories can be presented together during the learning phase, adding the option to skip the free-recall phase and jump straight to cued recall, and some visual design updates.

As to the difference between the total number of recruited participants and the sample sizes of the cognitive test analysis – this is likely the result of a copy-paste error that has slipped through our review process. The total number of enrolled participants (i.e. those who completed their first assessment) is 768, 1 of whom withdrew from the study. This corresponds to the 767 reported in the cognitive test regression analysis in Table 2. The 756 reported participants are the participants included in the demographics and lifestyle risk factor analysis (excluding 11 participants for whom data were not collected due to a technical issue in the data export). Thus, the total number of recruited participants should be 768, not 756. We are happy to have this unintentional error amended.

Lifestyle score calculation

The authors point out that our approach to measuring lifestyle has not taken into account the individual differences in risk factor contribution to brain health. This is true, but it was also not the intention in the first place. The current study aimed to assess the feasibility of a smartphone assessment of lifestyle and cognitive factors relevant to dementia risk in older adults. We did not aim to conduct a validation study or to optimise the measurement of individual risk factors and quantify their precise contribution to dementia risk. While this is unarguably a pertinent question, it remains to be addressed in future studies. For this reason, we opted for a relatively straightforward quantification of lifestyle which we disclosed fully in our supplementary materials to facilitate replication and interpretability.

Practice effects, app availability between testing sessions and app availability as a desktop app

Verma et al suggest that the practice effects on cognitive testing reported in the study may have been biased because the participants had access to the cognitive tests between testing sessions. To clarify, the research version of the Five Lives app was specifically developed to accommodate the design of the study and the tests were only available to the participants once at sign-up (baseline), once 2 weeks later, and once 3 months later. The research version of the app used in this study may have been mistaken for the commercial version of Five Lives. Participants were not made aware of the existence of the commercial version and, at the time of conducting the study, the commercial version was behind a paywall in the UK app stores.

Finally, the authors suggested that the lack of a desktop version of the Five Lives assessment may have limited the usability of the system in older adults. While older adults preferring desktop systems is a valid concern, it is out of scope with the current study, which was specifically designed to validate a mobile application. The rationale behind this choice was that mobile devices offer the benefit of wider accessibility than desktop systems thanks to lower costs, wider availability and easier distribution to users. ^{Reference Reid, Vassilev, Irving, Ojakäär, Jacobson and Lawrence2}