Introduction
Self-reported difficulties in cognitively demanding tasks, usually referred to as “subjective cognitive complaints,” are fairly common among the older adults and can be a source of significant concern (Sachdev et al., Reference Sachdev2010). It has been proposed that cognitive complaints may identify older persons who could benefit from further cognitive assessment, representing an opportunity to detect dementing illnesses in early or even prodromal stages (Jonker et al., Reference Jonker, Geerlings and Schmand2000; Treves et al., Reference Treves, Verchovsky, Klimovitzky and Korczyn2005).
However, the usefulness of cognitive complaints for identifying cognitive impairment has often been questioned because of the high prevalence of these complaints and their strong relationship with some psychological factors, such as depression, anxiety, and neurotic personality traits (Slaving et al., Reference Slavin2010). In addition, the perception of cognitive decline involves self-awareness and it has been recognized that lack of insight may affect a substantial proportion of patients with mild cognitive impairment (MCI) and dementia (Vogel et al., Reference Vogel, Stokholm, Gade, Andersen, Hejl and Waldemar2004; Palmer et al., Reference Palmer, Bäckman, Winblad and Fratiglioni2008).
Attempts to use subjective complaints to predict objective deficits have yielded mixed results, reflecting the complexity of the issues involved. Some studies have described a modest association between cognitive complaints and cognitive impairment, whereas others have found no significant association (Reid and Maclullich, Reference Reid and Maclullich2006). These inconsistencies can be attributed to the variety of methods used to elicit complaints. Some studies utilize simple questions with yes/no responses, some have questions with graded responses, and a number of them use competing questionnaires (Abdulrab and Heun, Reference Abdulrab and Heun2008).
A few studies that have investigated the characteristics of cognitive complaints and their association with cognitive impairment have reported that some specific types of complaints are more associated with objective impairment than others. In a study conducted with older volunteers, cognitively impaired individuals were more prone to give positive answers when asked if they were unable to “recall the names of good friends” or “follow and recall conversations,” but not when asked if they used to “forget where things are placed” (Lam et al., Reference Lam, Lui, Tam and Chiu2005). A primary care-based study asked patients to rate their abilities in 14 memory-related activities compared with last year. The item “remembering things that happened or were said a few days ago” was the only item with acceptable capacity to discriminate between upper and lower memory ability groups (Snitz et al., Reference Snitz, Morrow, Rodriguez, Huber and Saxton2008). In another cross-sectional study conducted with older individuals, participants were asked seven questions about difficulties with cognitively demanding activities. Positive answers to some questions, such as “trouble following a group conversation” or “finding one's way around familiar streets,” were more associated than others with higher odds of cognitive impairment (Amariglio et al., Reference Amariglio, Townsend, Grodstein, Sperling and Rentz2011).
In prior studies such as those described above, researchers have asked participants to rate their difficulties in a limited number of situations described by standardized questionnaires, rather than allowing the individuals to express such complaints spontaneously. In truth, a questionnaire-based approach provides self-rated perception of performance but not real complaints. From the clinical perspective, there is a significant difference between individuals who admit to having difficulty on direct questioning and those who actively seek medical attention to report their difficulties.
To the best of our knowledge, no studies have been conducted to characterize spontaneously reported cognitive complaints (SRCCs) and to investigate whether specific types of SRCCs are associated with the early stages of neurodegenerative processes. This deficiency in the literature may be attributed to the absence of a reliable instrument that is able to classify these complaints. The purpose of this study was to develop a classification system for SRCCs and test its reliability in a sample of older individuals seeking medical care for cognitive complaints.
Methods
This study was conducted in two phases. In phase 1, we employed the modified Delphi technique to develop an instrument that could be used to classify SRCCs into descriptive categories and cognitive domains. In phase 2, a sample of elderly patients seeking medical attention for cognitive complaints was recruited to test the inter-observer reliability of the proposed instrument.
Delphi consensus procedure
For the purpose of this study, nine experts in dementia care were invited to serve on the panel, based on their participation in memory clinics and authorship on peer-reviewed publications in the field. In addition, all of the invited experts are board-certified geriatricians who provide care in outpatient settings; therefore, these professionals are exposed to “cognitive complainers” in everyday practice. The panel members were questioned about specific items in four sequential rounds that were carried out by email. The responses were de-identified to allow for expression of diverse opinions and diverging standpoints.
Before initiating the Delphi process, an initial pool of 44 descriptive categories was developed by the researchers in brainstorming sessions. The construction of the first draft was based mainly on the cognitive complaints registered in the medical charts of patients who had been previously evaluated for suspected cognitive impairment. At this preliminary step, the main goal was to create categories that could be defined precisely in one sentence and could easily accommodate naturally occurring complaints. In addition, seven domains were proposed that correspond to well-accepted cognitive functions: temporal orientation, attention, prospective memory, retrospective memory, language, visuospatial skills, and executive functions. Initially, the descriptive categories and domains were not associated but instead were arranged in two discrete lists.
In the first round, the initial pool of descriptive categories and domains was presented to the panel of experts for review. Panelists were encouraged to submit suggestions for categories or domains that they felt should be included, excluded, separated, combined, or rephrased. In the second round, suggestions were compiled and presented for the whole group to assess their appropriateness. For that purpose experts were required to rate each suggestion on a five-point Likert scale indicating the degree to which a particular suggestion could improve the classification system. Suggestions that obtained a mean rate ≥4 were incorporated. In the third round, the panelists were asked to place each descriptive category into one of the seven domains. A consensus was reached for a given category when it was classified into the same domain by at least two-thirds of the experts. In the fourth round, the descriptive categories that did not reach consensus were circulated again, and the experts were given the opportunity to review their position in light of the group response. Experts with a minority opinion could adhere to the prevailing opinion or write a comment justifying their initial position. The instrument resulting from these procedures was employed in further steps.
Patient recruitment and assessment
Older patients seeking medical attention for cognitive complaints were recruited prospectively from June 2009 to February 2011 in a government-financed outpatient geriatric clinic in the city of São Paulo, Brazil. During this period 1,306 new patients were screened systematically to determine eligibility. To be eligible, subjects had to meet the following criteria: (1) the patient was ≥60 years of age; (2) a cognitive complaint was the main reason for seeking care; and (3) the patient was able to give at least one example of his/her everyday cognitive difficulties. We excluded patients who, in a subsequent standardized evaluation, were diagnosed with moderate to severe dementia, as indicated by a Clinical Dementia Rating (CDR) of 2 or 3, because these patients have limited capacity to provide spontaneous reports or benefit from the diagnostic implications related to the cognitive complaints. Subjects were informed of the procedures involved in the study, and they provided written consent before the interview. The research protocol was approved by the local ethics committee.
At the beginning of the interview, patients were prompted to provide free reports of their cognitive difficulties with the following instructions: “Have you noticed any problems with your memory or thinking? Please, describe the situations in which you have been presenting difficulties.” Each discrete complaint was transcribed verbatim by the interviewer. When the participants seemed to have completed their narratives, the interviewer tried to stimulate additional reports by asking: “Can you give me any other examples?” The procedure was repeated until the participant had no additional complaints.
Perceived cognitive function was further characterized using the Memory Complaint Questionnaire (MAC-Q; Crook et al., Reference Crook, Feher and Larrabee1992), an instrument that requires the subjects to rate current performance against past performance for a given situation. The MAC-Q is composed of six items that are rated on a 5-point Likert scale. Possible scores range from 7 to 35, with higher scores indicating a greater degree of perceived decline in memory function.
A structured interview was conducted to collect demographic and clinical data. Participants underwent a comprehensive protocol that included a 45-minute neuropsychological assessment composed by the Free and Cued Selective Reminding Test (Grober and Buschke, Reference Grober and Buschke1987) to evaluate episodic memory and the NEUROPSI (Ostrosky-Solís et al., Reference Ostrosky-Solís, Ardila and Rosselli1999), a standardized battery comprised of 26 subtests that assess a wide spectrum of cognitive functions. The CDR interview was administered to a knowledgeable informant and the score provided by the sum of boxes (CDR-SOB) was used as a global measurement of cognitive impairment, with higher scores indicating worse cognitive functioning (O'Bryant et al., Reference O'Bryant2008). Depressive symptoms were measured using the 15-item Geriatric Depression Scale (GDS-15), with higher scores indicating increased symptoms (Sheikh and Yesavage, Reference Sheikh and Yesavage1986). Dementia and depression were diagnosed according to the Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (American Psychiatric Association, 1994). A diagnosis of MCI was made when the subject fulfilled the criteria described by the International Working Group on Mild Cognitive Impairment (Winblad et al., Reference Winblad2004).
Procedures to test reliability
To test inter-observer agreement, we invited three board-certified geriatricians who are active in clinical practice. They did not participate in the development of the classification system and did not have access to any demographic or clinical information on the patients being considered. The three professionals were blind to the classifications made by the other geriatricians and were deemed equally competent to make judgments, with no a priori criterion for correctness.
Transcribed SRCCs obtained from the patients were given to these observers along with the proposed categories for classification. The geriatricians were instructed to classify each complaint in only one descriptive category, and for complaints that did not fit in any category, they were required to choose a domain. No training was provided, as the written instructions for classification were assumed to be sufficient.
Data analysis
We performed descriptive statistics to characterize the sample and the studied variables. Sociodemographic and clinical characteristics of the patients were presented for each diagnostic category (normal, MCI, and mild dementia). An SRCC was classified in a descriptive category or domain when at least two of the three observers were in agreement. SRCCs for which there was no sufficient agreement were excluded from the analysis. To test the usefulness of the total number of SRCCs as a global measure of self-perceived cognitive dysfunction, we studied the relationship between this variable and the MAC-Q total score using the Spearman's correlation coefficient. To assess whether SRCCs would differ from the MAC-Q, we described the proportion of individuals with complaints in equivalent categories for each classificatory system and evaluated the associations using the Cochran–Armitage test for trend.
To examine reliability, a dichotomous variable was created to indicate the presence or absence of an SRCC in each descriptive category for each patient. To examine agreement on domains, an ordinal variable was created representing the number of SRCCs in each domain for each patient. The unweighted κ for multiple coders was calculated to measure inter-observer reliability. The κ statistic is used to determine the chance-corrected agreement between two or more sets of categorized data. The coefficient is scaled to be 0 when the amount of agreement is what would be expected by chance and 1 when there is perfect agreement. We interpreted κ coefficients with established conventions, in which values ranging from 0.41 to 0.60 may be considered moderate agreement, 0.61 to 0.80 as substantial, and >0.80 as almost perfect (Landis and Koch, Reference Landis and Koch1977). In categories with a limited number of positive cases, inter-observer discrepancies even for a single case will provoke a significant random error on the computation of κ. To minimize the effect of this problem, we calculated κ only in categories with at least 10 positive cases (Nelson and Cicchetti, Reference Nelson and Cicchetti1995).
A multivariate regression was carried out to identify sociodemographic and clinical factors associated with the total number of complaints. In this model the dependent variable was defined as the total number of SRCCs. Independent variables entered in the model included age, gender, educational attainment, severity of the cognitive impairment as indicated by the CDR-SOB, and depressive symptoms as measured by the GDS-15. Logistic regression models were performed to study the relationship of SRCCs in each domain with sociodemographic and clinical variables. In these models, constructed for each of the seven domains, the dependent variable was defined as the presence or absence of at least one SRCC in a given domain. Independent variables were the same as described for the multivariate regression model. Analyses were conducted with Stata version 12.0 (College Station, TX), and significance was set at α < 0.05.
Results
Development of the classification instrument
Of the nine experts who were invited to join the Delphi survey, eight agreed to contribute. Of these, two failed to return answers for the first round. Therefore, the panel was composed of six experts who completed the whole process. In round 1, experts suggested a total of 19 changes in the initial draft. In round 2, five suggestions were approved by the panel according to the adopted criteria. As a result of the first two rounds, we have developed two distinct sets: one with 42 descriptive categories and another with seven domains. In the third round, 38 of the 42 proposed categories were placed on the same domain by at least two-thirds of the experts. In round 4, agreement was finally attained on the placement of four controversial categories, as some panelists changed their positions in light of the group response. At the end of these procedures, the resulting instrument was considered to be worthy of testing in a clinical field study.
After collecting data from patients, we made further adjustments in view of the empirical evidence. Three categories were excluded because they were not mentioned by any participant. Four categories were combined because they were sufficiently related and had been mentioned by only a small proportion of patients. The final instrument consists of 37 descriptive categories that are organized in seven domains. Figure 1 depicts a flowchart with the steps involved and presents information about the categories that were excluded or combined. Table 2 shows the classification instrument in its final form.
Flowchart depicting the steps involved in the modified Delphi process and adjustments made in light of the empirical evidence.
Sample characteristics and occurrence of cognitive complaints
Two hundred and thirty consecutive patients referred for suspected cognitive impairment were considered for participation. Of these, 42 individuals were excluded because they denied cognitive difficulties and had been brought to the clinic by relatives. Three individuals were ineligible because, although confirming cognitive difficulties in a general sense, they were unable to provide examples of everyday situations in which they had those difficulties. Finally, four patients were excluded because they had a diagnosis of moderate dementia and another was excluded because of missing data. Our final sample consisted of 180 older adults with a mean age of 74.0 (±7.2) years (76.1% women). After a standardized evaluation, 93 (51.7%) of the patients had normal cognitive aging, 43 (23.9%) had MCI, and 44 (24.4%) were diagnosed with mild dementia. Sociodemographic and clinical data for the participants are presented in Table 1.
Demographic and clinical characteristics of the study participants
Data are shown as the means (standard deviation) or numbers (percentage).
MCI = Mild Cognitive Impairment; CDR-SOB = Clinical Dementia Rating–Sum of Boxes.
The 180 patients reported a total of 493 SRCCs, with a range of 1 to 6 complaints per patient and a mean of 2.7 (±1.3). The proportions of SRCCs that the three observers were able to classify in descriptive categories were 91.9%, 93.5%, and 95.7%. Some categories occurred frequently: “Forgetting where one has placed objects” (35.0%); “difficulty with recalling people's names” (22.2%); “getting distracted and forgetting what one was going to do next” (20.6%); “burning food because of forgetting to turn off the stove or oven” (18.9%); “forgetting appointments or meetings” (17.2%); and “forgetting about recent conversations” (15.0%). The proportion of patients presenting SRCCs in each category and domain is presented in Table 2.
Final version of the classification instrument, with proportion of patients reporting complaints in each category and corresponding inter-rater reliability
The mean MAC-Q total score was 29.1 (±1.7). The Spearman's coefficient between the MAC-Q total score and the number of SRCCs was 0.47, revealing a moderate correlation. Analyzing the relationship between each item on the MAC-Q and its corresponding category on the newly developed classification system, we found significant association in all categories except for “remembering facts from a magazine or newspaper.” The questionnaire determined an important cueing effect, as most of the situations that were rated as “much worse now” on the MAC-Q were not mentioned as an SRCC. Of the 83 patients who rated the item “recalling where objects were placed” as “much worse now,” only 40 (48.2%) had mentioned the same situation on their spontaneous reports. For the remaining items of the MAC-Q, this proportion was even lower, as seen in Table 3.
Proportion of patients with SRCCs in categories that overlap with MAC-Q items (n = 180)
Data are shown as the numbers (percentage).
MAC-Q = Memory Complaint Questionnaire.
*Cochran–Armitage test for trend.
Inter-observer reliability
Regarding the classification within descriptive categories, there was total agreement between the three observers for 409 (83.0%) complaints and complete disagreement for 20 (4.1%) complaints. In respect of domains, there was total agreement between the three observers for 448 (90.6%) complaints and complete disagreement for only 5 (1.0%) complaints. The unweighted κ varied from 0.79 to 1 for descriptive categories, with coefficients indicating “almost perfect” agreement for 13 of the 15 categories for which reliability was calculated. For domains, the unweighted κ varied from 0.83 (retrospective memory) to 0.97 (temporal orientation). Therefore, classification of SRCCs attained “almost perfect” agreement in all domains. Reliability data for categories and domains are presented in Table 2.
Relationship between spontaneously reported cognitive complaints and participants’ characteristics
As indicated by multivariate analysis, a greater number of SRCCs were associated with higher scores on the GDS-15 (p = 0.002), younger age (p = 0.006), and lower scores on the CDR-SOB (p = 0.023). The occurrence of SRCCs in some domains was associated with sociodemographic and clinical variables. In fully corrected logistic regression models, attention complaints were more common among women (p = 0.007) and associated with lower scores on the CDR-SOB (p = 0.016). Prospective memory complaints were associated with younger age (p = 0.017). Language complaints were associated with higher educational attainment (p < 0.001) and lower CDR-SOB scores (p = 0.035). Executive function complaints were significantly associated with higher educational attainment (p = 0.019). No variables were associated with the presence of SRCCs in other domains (orientation, retrospective memory, and visuospatial).
Discussion
We developed a well-structured system for classifying SRCCs in descriptive categories and cognitive domains. The proposed conceptual framework achieved a high level of acceptance among experts who participated in a Delphi process as well as clinicians who classified the complaints. Each of the three observers was able to classify the majority of SRCCs in descriptive categories, and inter-observer reliability of this newly developed instrument was almost perfect for all domains and most categories.
The characteristics of the studied sample do not significantly deviate from population census data for Brazilian elderly, but the low education level of the participants may raise concerns about the generalizability of this study to other populations. However, a comparison of our findings with those obtained by a previous study conducted in England seems to lessen these concerns. Kapur and Pearson (Reference Kapur and Pearson1983) have described SRCCs by clustering similar items but without structured criteria. The study was conducted with a sample of 100 brain-damaged patients who were suffering from a variety of conditions. Most of the patients were men younger than 60 years. In comparing the 10 most commonly reported types of SRCCs in the two studies, we have noticed that seven categories were roughly the same in both studies. A more objective comparison was not possible because Kapur and Pearson provided only a vague description for some complaints, such as “remembering to do something.” Considering that the two samples had markedly different sociodemographic profiles, it is surprising to find that the most prevalent complaints were similar in both studies. These findings seem to indicate that, although sociocultural factors will certainly have some influence in shaping the characteristics of naturally occurring cognitive complaints, there appears to be a strong universal component underlying these manifestations. For this reason, we believe that this classification system may have good cross-cultural properties.
Comparisons between cognitive complaints that were spontaneously reported and those that resulted from answers to a structured questionnaire provide interesting insights. The correlation coefficient between the number of spontaneous SRCCs and the total score of the MAC-Q was 0.47, indicating that although both variables are probably useful as global measures of self-perceived cognitive dysfunction, they provide measures that are related but not the same. We have also shown that most categories that were rated “much worse now” on the MAC-Q were not mentioned as SRCCs. When responding to a questionnaire, subjects indicate answers that they consider appropriate but which they may not have remembered or considered important enough to be addressed. It is clear that the strategy of eliciting free reports can bypass the cueing effect provoked by questionnaires. The advantages and drawbacks of avoiding the cueing effect associated with questionnaires need to be addressed in future studies.
When examining the factors related to the total number of complaints, we have observed an expected positive association between the number of SRCCs and the GDS-15 score. In fact, strong associations between depressive symptoms and global measures of cognitive complaints have been widely reported (Reid and Maclullich, Reference Reid and Maclullich2006). Somewhat counterintuitively, our results suggest that the total number of SRCCs presents a negative association with the CDR-SOB. This finding highlights the fact that SRCCs need to be interpreted in a different way than questionnaire-elicited cognitive complaints. When prompted to provide free reports of their cognitive difficulties, patients are challenged to use memory and language skills that may be somewhat affected even in the mildest forms of cognitive impairment.
The relationship between sociodemographic factors and SRCCs in some specific domains allows us to make logical inferences involving exposures and demands. Language and executive function complaints were associated with higher educational attainment. Indeed, more educated individuals tend to be engaged in activities that place higher demands on communication skills, capacity to develop strategies, solve problems, organize information, and take decisions. The occurrence of prospective memory complaints was associated with younger age. We speculate that younger subjects may have a greater social network and, therefore, attend to meetings and appointments with greater frequency. Attention and language SRCCs were negatively associated with the CDR-SOB. If these findings are replicable, clinicians will benefit from prompting patients to describe their difficulties and using this classification system as practical instrument for risk stratification. Prospective studies are needed to investigate whether SRCCs in specific domains are associated to later decline.
Some limitations of this study should be noted. First, even if given the same information, different expert panels are unlikely to produce the same results. To attenuate this shortcoming, a panel of adequate size is necessary (Hasson et al., Reference Hasson, Keeney and McKenna2000). Although what should be considered an adequate number of panelists has not been defined, we must recognize that the participation of only six experts may have restricted the diversity of influences on the construction process. Second, the relatively limited patient sample size has undermined the statistical analysis of this study. Because some descriptive categories presented a very low rate of positive results, we could not calculate the reliability of a non-negligible proportion of categories. The difficulties of analyzing data from low-occurrence categories are likely to recur in studies with small samples. Under these circumstances, conducting analyses based on domains instead of categories appears to be a better solution. Third, although we anticipate that the total number of complaints may be one of the important measures of the instrument, it was not possible to calculate inter-observer reliability for this measure. Complaints were presented to the observers in the form of a list in such a way that each complaint was represented by a sentence. Therefore, the number of complaints mentioned by each patient was automatically provided as the number of sentences in the list.
In conclusion, our newly developed instrument proved to be a practical tool for classifying the majority of SRCCs with high reliability. We believe that this classification system fulfills an important gap in this research area and that it will stimulate further studies to investigate the clinical meaning of SRCCs.
Conflict of interest
None.
Description of authors’ roles
Daniel Apolinario conceived the study, collected data from patients, carried out data analysis, interpreted the results, and prepared the final draft of the manuscript. Rafaela Branco Miranda participated in designing the study, collected data from professionals, and prepared the first draft of the manuscript. Regina Miksian Magaldi and Alexandre Leopold Busse took part in designing the study and participated in the Delphi consensus. Aline Thomaz Soares recruited patients and participated in the Delphi consensus. Leonardo da Costa Lopes, Juliana Yumi Tizon Kasai, and Erika Satomi classified complaints and reviewed the manuscript. Claudia Kimie Suemoto and Elina Lika Kikuchi participated in the Delphi consensus and reviewed the manuscript. Wilson Jacob-Filho took part in designing the study and reviewed the manuscript.
