Introduction
An increasing number of everyday tasks and routines in modern life require their users to deal with (information) technology. Examples of everyday technological devices for which non-technological alternatives are now difficult to find are automatic teller or cash machines (ATM), train-ticket vending machines and central-heating thermostats. Many older adults experience difficulties when using modern technologies. For example, it has been shown in several studies that when learning to use a computer, older adults take longer to master the system, make more errors and require more help than younger people (Charness et al. Reference Charness, Bosman, Kelley, Mottram, Rogers, Fisk and Walker1996; Czaja and Sharit Reference Czaja and Sharit1993; Kelley and Charness Reference Kelley and Charness1995). Furthermore, it has been reported that older adults experience problems when using common everyday systems such as ATMs (Rogers, Gilbert and Fraser Cabrera Reference Rogers, Gilbert and Fraser Cabrera1997). The difficulties that older adults encounter when using such technologies compromise their independence, which is a primary goal of many older people (Rogers and Fisk Reference Rogers, Fisk, Craik and Salthouse2000; Willis Reference Willis, Birren and Schaie1996). Everyday tasks that are essential to independent functioning, but which have become more and more technology driven, may become too difficult for older adults to perform autonomously. It is therefore important to evaluate and improve the usability of everyday technological systems to accommodate the needs of older users.
An essential step in the process of enhancing usability is to study the types and causes of the problems that older adults face when they use technology. One potential cause is age-related cognitive change, for cognitive abilities tend to decline with age (Craik and Salthouse Reference Craik and Salthouse2000; Freudenthal Reference Freudenthal2001; Houx and Jolles Reference Houx and Jolles1993; Kelley and Charness Reference Kelley and Charness1995; Rogers and Fisk Reference Rogers, Fisk, Craik and Salthouse2000; van der Elst et al. Reference van der Elst, van Boxtel, van Breukelen and Jolles2005; van Hooren et al. Reference van Hooren, Valentijn, Bosma, Ponds, van Boxtel and Jolles2005). Many cognitive skills seem to be critical when using complex technological systems. For instance, speed of information processing may be important when using many public electronic devices such as ticket machines, as they tend to ‘time out’ without sufficient input. Psychomotor processes, such as response selection, are probably also important when using technological systems, because often multiple options are simultaneously available and the user has to make choices with appropriate responses. Short-term or working memory abilities may be important in task performance, e.g. in remembering options or a system's output at a later stage, or to retrieve factual or procedural information from long-term memory. Finally, the ability to switch between several concepts or cognitive processes is vital in many technological tasks, for example switching between remembering a PIN code and selecting the appropriate buttons on an ATM. By increasing knowledge of exactly which cognitive abilities lead to problems with technology, it may be possible to modify devices to accommodate older users' capacities and thereby to improve the efficiency of their use.
The experiment
At present, an ecologically valid test of technological efficiency is not known. Many studies of age-related differences in coping with technology use very specific, mostly computer-based technological applications, e.g. several computer tasks (Czaja and Sharit Reference Czaja and Sharit1993; Mead et al. Reference Mead, Jamieson, Rousseau, Sit and Rogers1996), automatic teller machines (Mead and Fisk Reference Mead and Fisk1998) or simulated unfamiliar complex devices (Freudenthal Reference Freudenthal2001). These applications were regarded as too specific for the study of the technological abilities that are important for everyday independent functioning. A new test was therefore designed that incorporates several technological tasks that are common in daily life. The test was used in a randomised trial of the impact of computer training and Internet use on the autonomy of older adults. The participants assigned to the intervention group were given a home computer with a high-speed Internet connection to use for one year. The technological ability test was intended to measure whether learning to use a personal computer and the Internet had a ‘spin off’ effect on the use of other (everyday) technologies. This test was named the ‘Technological Transfer Test’ (TTT).
By simultaneously measuring cognitive abilities together with the TTT in the present study, it was possible to examine the relationship between actual technological skills and cognitive abilities in a large sample of older adults. To our knowledge, this has not been done before in relation to real-life technological tasks. It was hypothesised that cognitive abilities known to decline with age (specifically verbal memory, information processing speed, cognitive flexibility and psychomotor processes) determine older adults' efficiency in performing common technological tasks. The cognitive tests that were used in this study tap into several cognitive abilities and formed the core battery of output measures for a longitudinal study of the determinants of cognitive ageing (Jolles et al. Reference Jolles, Houx, van Boxtel and Ponds1995). Other factors that might influence technological efficiency were controlled, namely visual acuity (as some of the devices had rather small buttons and visual information), basic motor speed (as participants who have very high motor speed probably operate the devices faster), and the past experience of participants with similar devices. Finally, since there were participants with and without an interest in information technology (IT), and those with and without previous exposure to the intervention (i.e. they had used a computer and the Internet for 12 months), we also controlled for interest in IT and the relevant exposure.
The design and participants
The study sample was 236 healthy people aged 64–75 years who were randomly recruited from the City of Maastricht population register. Individuals who were interested in learning to use computers and the Internet and those without this interest could participate. Other inclusion criteria were that they had no prior active computer experience and were living independently. A strictly controlled design required three control groups in addition to the intervention group, to account for the simultaneous effects of using a computer and the Internet for 12 months, of computer training, and of participants' interest in learning to use computers. The 191 participants interested in taking part in the intervention programme were randomly assigned to one of three groups: ‘Training/Intervention’, ‘Training/No intervention’ and ‘No training/No intervention’. The 45 participants who were not interested were assigned to the third ‘No Intervention Control Group’ to allow for the possible effect of an interest in computers on technological efficiency. The training involved three meetings at which the participants were acquainted with a personal computer, the operating system and several standard software applications (e.g. word processor, e-mail client, and a web browser). Participants in the Intervention Group received a personal computer with a fast Internet cable connection at their homes for 12 months. Participants in the other three groups agreed to refrain from any (further) computer use during the study period. All participants were screened with respect to cognitive functions and technological ability at baseline and after four and 12 months.
Procedures
The Technological Transfer Test (TTT) was administered twice during the study, once at baseline and once after 12 months. On both occasions, participants had to operate four devices that are common in modern daily life (see Table 1). The technological devices in the TTT were chosen to represent a wide range of the technological tasks with which older adults are regularly confronted in everyday life. The eight test levels of familiarity and difficulty with the particular devices were balanced over the two test occasions to avoid assigning similar tasks on both occasions. By doing so, we were able to design two parallel versions of the TTT. Since many of the specific devices that were used in this test were unfamiliar to the participants, learning effects were expected from repetition of the task. We therefore chose to use different tasks in each test. At each test administration, two of the tasks were real-life consumer devices, and two were computer simulations of common public technology devices operated with a touch-screen interface. These devices were simulated because they are for public use only and are not commercially available.
The TTT devices and assignments
The participants received a brief instruction sheet with a clear assignment for each device (see Table 1). The instruction sheet for the real-life devices also contained directions for use that had been adapted from the original instructions. The participants were instructed that they could, but did not have to, use these instructions. Overall, participants were instructed to complete the assignments as fast and accurately as possible. The experimenter was not allowed to assist the participants in any way. If a participant was convinced that the task had been completed when this was not the case, the participant was informed that the task had not been completed and was encouraged to try again.
Measures
As measures of technological ability, we used performance time to measure the efficiency of completing the tasks. The number of errors that participants made with the simulated devices was available from electronic logs. As using the instructions provided with the real-life devices was not obligatory, the time spent reading the instruction sheet (including the directions for use) was included in the general performance time. This was to take into account whether or not the participants needed the directions. As regards the simulated devices, no instructions were provided, so in order to avoid measuring differences in reading speed, participants were allowed to read the instructions before the task started. Performance time was used as the primary outcome measure because it is a simple measure of efficiency. It not only gives information about the time that the participants needed to understand and operate the devices, but also about the time spent on recovery from mistakes. Finally, a total or overall score for the four tasks was computed on both test occasions. As the tasks required quite different times to complete, the scores for each individual task were transformed to z scores and summed to give the overall score. This has been used to compare the four tasks in each test administration.
Level of education was measured on an eight-point scale as used by Statistics Netherlands (de Bie Reference de Bie1987). The scale ranges from primary education to higher vocational training and having a university degree. Visual acuity was measured with a Landolt-C optotype chart at a distance of five metres under standard luminescence and with corrected vision (Hollwhich Reference Hollwhich1989). A measure of basic motor speed was obtained from a continuous tapping test: the participants were required to press the button of an electronic counting device with the index finger of their preferred hand at maximum frequency for 30 seconds (Brand and Jolles Reference Brand, Jolles, Maarse, Mulder, Sjouw and Akkerman1987). A variable for the frequency of using TTT devices was obtained by asking the participants to rate, on a five-point scale, how often they used each of the eight technological devices in the TTT. The scale ranged from ‘never’ to ‘at least once a week’.
There were several measures of cognition. The Letter-Digit Substitution Test (LDST) (van der Elst et al. Reference van der Elst, van Boxtel, van Breukelen and Jolles2006) is a modification of the Symbol-Digits Modalities Test (Lezak Reference Lezak1995) and is used to measure the general speed of information processing. A code is provided at the top of a sheet of paper that couples the numbers 1 to 9 with random letters. Participants were asked to fill in as many corresponding numbers as possible in 90 seconds in boxes on the rest of the sheet that contained only letters. The Visual Verbal Learning Test (VVLT) was used to measure verbal memory and verbal learning (Brand and Jolles Reference Brand and Jolles1985; van der Elst et al. Reference van der Elst, van Boxtel, van Breukelen and Jolles2005). In this test, 15 monosyllabic, low-associative words were presented one after another on a computer screen, and the participants were asked to recall as many words as possible without any time or order constraint (immediate recall). This procedure was repeated five times with the same list of words. Twenty minutes after the recall of the fifth trial, the participants were once more asked to recall as many words as possible (delayed recall). The score in the first trial of the immediate recall and the delayed recall score were used as indications respectively of short-term and long-term memory.
The Concept Shifting Test (CST) was modified from the Trail Making Test and used to measure cognitive flexibility (Reitan Reference Reitan1958; Vink and Jolles Reference Vink and Jolles1985). This test consists of three sheets of paper with 16 small circles that are grouped in a larger circle. On the first sheet, numbers appear in the small circles in a fixed random order. Participants were asked to cross out these numbers in the right order as fast as possible. Instructions for the second sheet were identical to those for the first sheet, except that on this sheet letters appear in the circles. On the third sheet, participants had to alternate between numbers and letters. The time needed to complete each of the sheets was recorded. The difference between the score for the third sheet and the mean of the first and second scores was used as an estimate of the slowing-down due to the shifting between two concepts (numbers and letters), i.e. cognitive flexibility.
The Motor Choice Reaction Time test (MCRT) was included to measure psychomotor speed (Houx and Jolles Reference Houx and Jolles1993). This test is conducted with a six-button panel with one red and five white buttons, laid out in a semicircle around the red button. The participants were asked to hold down the red button with the index finger of the preferred hand as long as no white button was lit. As soon as a white button lit, the participants were to release the red button and then press the lit button (or a button adjacent to it) as quickly as possible. After this, the red button had to be held down again. The MCRT had three sub-tasks that yielded three performance measures. For the first (simple reaction time), only the upper white button was lit. For the second (choice reaction time), one of the three upper buttons was lit. For the third (incompatible choice reaction time), one of the three upper buttons lit, but the button immediately to the right of the lit button had to be pressed. Two variables were of interest for this study. First, the difference between the median response times of the second and the first sub-tasks was used as an indication of response selection. Secondly, a measure of inhibition of a prepotent response (Kornblum, Hasbroucq and Osman Reference Kornblum, Hasbroucq and Osman1990) was provided by the difference between the third and the second sub-tasks.
Statistical analyses
To study the reliability of the newly developed TTT, reliability analyses and factor analyses were conducted on the performance times. To test the consistency between the four tasks at the two administrations, Cronbach's alpha and the correlations between each of the four tasks were calculated. Factor analyses were performed to test whether the two sets of four tasks loaded on the same factors. The performance time for each of the TTT tasks and the general scores of both tests were predicted using multiple hierarchical regression analysis. Since the performance time data of the train-ticket vending machine task were not normally distributed, a log-transformation was used. The error scores of the tasks with simulated devices were very skewed to the right. These scores were dichotomised (‘no errors’ and ‘one or more errors’) and analysed using logistic regression.
Successively more elaborate logistic regressions were run. At Step 1, age, level of education and gender were entered. At the second step, the frequency with which participants performed the particular TTT task in daily life was entered as a measure of experience with the device. As regards the general scores of the TTT, the average use of the four devices was entered in this step. At Step 3, the factors ‘interest in computers and the Internet’ and ‘participation in intervention’ were entered (both dichotomous variables). Next, visual acuity was included together with basic motor speed. At the final step, all cognitive ability measures were entered to detect an association with performance in the technological tasks.
Results
Reliability analyses
Cronbach's alpha calculated for the technological tasks was 0.55 at baseline and 0.63 at the 12-month follow-up tests. The test was repeated with deletions of the individual technological tasks but in no case did alpha increase, indicating that internal consistency was not disrupted by any of the tasks. The zero-order correlations among the four technological tasks of each test administration are summarised in Table 2. The tasks were significantly but only moderately inter-related, with the zero-order correlations ranging from +0.24 to +0.41. The factor analysis showed that a single-factor solution reproduced the correlation matrix adequately. No residual correlation deviated more than 0.05 from zero in either test. The factor extracted for the baseline tasks explained 30 per cent of the variance, whereas for the 12-month follow-up tasks, the explanation was 33 per cent. The factor loadings of each task for the factors ranged from 0.45 to 0.64 at baseline and from 0.44 to 0.68 at the 12-month follow-up, indicating that the factors explained reasonably high proportions of the variance.
Zero-order correlation coefficients between performance in the TTT tasks at baseline and follow-up
Notes: All correlations were significant at p<0.01. 1. Train-ticket vending machine.
Regression analysis
Table 3 shows the proportions of explained variance (R 2) and the change in the proportion of explained variance (ΔR 2) in performance times for each of the four steps in the regression model. The unstandardised regression coefficients for each of the predictors of performance time in the final steps are listed in Table 4. At Step 1, age, level of education and sex explained significant proportions of the variance of all the performance time measures, except for the microwave task. The explained variance ranged from 6.4 per cent for the alarm-clock task to 20.1 per cent for the general baseline score. At the second step, the frequency of use of the separate TTT devices was entered; it added significantly to the explained variance of: the performance times for the train-ticket task, the alarm-clock task, the microwave task and the voice-menu task; of making errors during the train-ticket task, and of the general scores on both test occasions (the additional explained variance ranged from 1.7 to 5.5 per cent).
Explained variance (R2) and change in explained variance (ΔR2) of the performance times for each technology task at the five steps of the regression model
Notes: ΔR 2 change in R 2. Included variables at Step 1: age, level of education, sex; at Step 2: experience with the particular task; at Step 3: interest in IT; at Step 4: vision and basic motor speed; at Step 5: cognitive measures.
Significance levels: * p⩽0.05; ** p⩽0.01 of R 2 change after each step.
Unstandardised regression coefficients (B) and standard errors (SE) in the final model for all technology tasks
Note: For explanation of the measures and the acronyms, see text.
Significance levels: *p⩽0.05; ** p⩽0.01.
Step 3, which introduced ‘interest in computers and the Internet’ and ‘expereience with the intervention’, only provided significant additional explanation at the 12-month follow up for the alarm-clock and voice-menu task performance times, for making errors during the voice-menu task, and for the general score (the additional explained variance ranged from 3.7 to 6.7 per cent). Step 4, which introduced visual acuity and basic motor speed, produced significant additional explanation (3.0 to 5.2 per cent) of performance times for the ATM task, for making errors during the train-ticket vending machine task, and for the general score at baseline. The cognitive measures introduced at the final step explained significant proportions of variance in all baseline performance time measures, the smartcard task and both general scores. The additional explained variance ranged from 5.4 to 12.2 per cent. None of the error measures was sensitive to the cognitive variables.
Significant cognitive predictors of the separate TTT tasks and the general scores differed for each of these measures (see Table 4 for the unstandardised regression coefficients in the final models). At baseline, the time needed to complete the CD-player task was significantly predicted by cognitive flexibility. Participants who needed more time to switch between the two Concept Shifting Test (CST) concepts also needed more time to complete the task. Performance in the telephone task was predicted by general cognitive speed, and a larger number of boxes filled in on the Letter-Digit Substitution Test associated with a shorter performance time. Long-term memory and cognitive flexibility were significant predictors of the automatic teller machine (ATM) task. Participants who needed less time to switch between the two CST concepts needed less time and made fewer errors.
Surprisingly, participants who were able to recall more words in the delayed recall task of the Visual Verbal Learning Test (VVLT) needed more time to complete the ATM task. The time needed to buy a train ticket was not significantly predicted by any of the separate cognitive measures. Finally, the general score for the four baseline TTT tasks was significantly predicted by general cognitive speed, working memory, long-term memory, and cognitive flexibility. Participants with better scores on the LDST, the first trial of the VVLT and the CST flexibility measure had better general TTT scores. Again, surprisingly, participants who recalled more words on the delayed recall task of the VVLT had worse scores.
At the 12-month follow-up, as with the baseline tasks, the cognitive variables that significantly predicted the performance times differed for each of the TTT tasks and for the overall score. Performance times for neither the alarm-clock task nor the microwave task were significantly predicted by any of the cognitive measures. The significant predictors of the time needed to charge a smartcard were general cognitive speed and inhibition of a prepotent response. The participants with faster performance on the cognitive measures needed less time to charge the smartcard. The number of errors was not significantly predicted by any of the cognitive measures. The fourth task, arranging travel insurance using a voice menu system, was significantly predicted by cognitive flexibility. Participants who needed less time to switch between the two concepts of the CST performed better in the voice-menu task. Again, not one of the cognitive measures significantly predicted the umber of errors during this task. Finally, the overall score on the four 12-month follow-up TTT tasks was significantly predicted by general cognitive speed. In other words, participants who performed better in the LDST were more likely to have a better general score.
Discussion
The aim of this study was to determine the role of cognitive abilities in the efficiency with which older individuals use everyday technological devices. Before answering the main question of this paper, we studied some methodological aspects of the TTT as a measure of general technological ability. Factor analyses indicated that one factor underlied the four tasks in each test, which may be called ‘general technological ability’. Because the reliability of the TTT tasks did not increase after deletion of one of the technological tasks at either baseline or follow-up, all tasks are considered to measure the same construct. The zero-order correlations between the separate TTT tasks in both tests showed moderate statistically significant inter-relations, which suggested a shared underlying common factor on the one hand, and characteristics unique to each task, on the other.
The final step in refining the regression models was to include the cognitive variables. For most of the task performance times, these added significantly to explained variance, which indicates that cognitive functions do play a role in the time needed to perform many everyday technological tasks. In spite of the high percentages of the explained variances at the final step, the predictive values of the separate cognitive variables were inconsistent. Overall, cognitive flexibility and general cognitive speed were most predictive of the outcome measures, followed by long-term and short-term verbal memory. Psychomotor abilities were less strongly related to performance of the tasks.
The fact that cognitive flexibility was an important predictor of performance in many of the tasks underscores the intuitive notion that this function is indeed involved in using diverse technologies. For instance, in the voice-menu task, the participants had to switch between choosing insurance for travel to countries worldwide to that for only European countries, by pushing the appropriate buttons. In other words, they first had to undertake a decision task, and then a motor task while remembering the decision that they had just made.
The second cognitive ability that was found to predict technological performance was general cognitive speed, or speed of information processing. A ‘simultaneity mechanism’ has been described as central to the processing speed theory of differences among adults in cognition (Salthouse Reference Salthouse1996). It is based on the idea that the products of early information processing steps may be lost by the time that the later steps are reached. For example, in the telephone-task, the procedure for programming a telephone number is quite complicated and the user has to push several buttons in a particular order. Because the telephone does not provide clear feedback about the actions that have been implemented, it is possible that older adults forget which button they pushed last when trying to decide on the next action. Unexpectedly, the long-term memory measure was positively related to technological performance. Participants with better long-term verbal memory needed more time to perform the technological tasks – we can only speculate about this finding. Perhaps these participants not only put more effort into remembering words from the VVLT, but also tried harder to remember aspects of either the assignment (after the task had started) or the device, and so needed more time.
The two psychomotor measures included in the analyses did not play an important role in the participants' performance, which was contrary to our expectation, as both response selection and the inhibition of prepotent responses (MRKT) appear to be abilities relevant to technological performance. One explanation for not finding a significant predictive effect of the ability to inhibit prepotent responses on all technological measures (except performance time for the smartcard task), may be that older adults have less experience with modern technological devices. They may, as a result, have less strongly ingrained stimulus-response combinations, and under these circumstances the inhibition of prepotent responses may be of little importance. The cognitive predictors of technological abilities only predicted performance time measures. Whether participants made errors or not did not seem to be related to cognitive abilities (with the exception of cognitive flexibility as a predictor of the number of errors during the ATM task). The lack of a relationship between cognitive measures and making errors may, however, also have methodological causes. For instance, error scores are extremely difficult to reproduce (i.e. reliability is low) and their distributions tend to be extremely skewed.
That cognitive measures are important in everyday technological problem solving and may have important consequences for older adults' independent functioning. Because many cognitive abilities decline as a result of normal ageing processes, older adults are more likely to experience problems in the use of technological devices that are essential to daily tasks or that could enhance their autonomy. It is therefore important that the designers of everyday technological devices are aware of the changing cognitive capacities of older adults when developing new products (Holt and Morrell Reference Holt, Morrell and Morrell2002; Rogers and Fisk Reference Rogers, Fisk, Craik and Salthouse2000).
This study's findings suggest, for instance, that products should not place heavy burdens on the user's ability to process information, as when information is presented only at the moment it is required. The need to switch between several cognitive domains should also be limited, for instance by providing a logical sequence of a comprehensive series of necessary actions to complete a goal. In developing a new product, designers ideally should identify the exact cognitive processes that its use requires. By doing this, they will become more readily aware of design flaws and more able to make modifications that minimise the load on the user's cognitive abilities.
Entering the socio-demographic independent variables into the regression models generally produced significant additional explained variance. This is consistent with the findings of research into ageing and cognition, which have amply documented the effects of age, level of education and gender on cognitive performance (Elias et al. Reference Elias, Elias, D'Agostino, Silbershatz and Wolf1997; Gallacher et al. Reference Gallacher, Elwood, Hopkinson, Rabbitt, Stollery, Sweetnam, Brayne and Huppert1999). In the present study, the predictive value of age was not high, probably because of the limited age range of our participants (64–75 years). Level of education and gender predicted performance as expected. Participants with higher levels of education and males were generally faster and made fewer errors.
Entering measures of interest in computers and the Internet and of participation in an Internet-related intervention only explained significant proportions of variance in TTT tasks at the 12-month follow-up. It is remarkable that the participants who were more interested in the Internet needed more time overall and made more errors. As this finding is difficult to explain, and as the measure did not significantly predict any of the other outcomes, we suppose that it was a chance result. At the 12-month follow-up, individuals who had participated in the intervention performed both the alarm-clock task and the voice-menu task faster, and had better general scores. This suggests that the generalised computer and Internet skills that were obtained through participation in the study promoted the efficient use of technological devices in daily life. This finding will be discussed in more detail in a forthcoming paper about the intervention study.
The major appeal of the Technological Transfer Test was its ecological validity. Although the instructions for the assignments were not completely identical to those for technological devices in daily life, we feel that this is a feasible and reliable way to quantify the efficiency of everyday uses of technological devices. It was not possible to assign exactly the same test twice to the same participants, because learning effects were expected to be quite large. Because no validated instruments were available from earlier research, we tried to circumvent the problem by designing eight different technological tasks that were balanced over the two measurements with respect to expected experience and levels of difficulty. Altogether the eight tasks represented the broad domain of everyday technological tasks and provided parallel tasks for the repeat test. This is a first approach to generating hands-on quantitative data about the efficiency of dealing with actual existing everyday technological devices. The large sample of participants and the two parallel versions of four tasks in each of these tests enabled us to study thoroughly the cognitive predictors of technological efficiency that are essential to everyday functioning.
Overall, the findings from this study confirm that cognitive abilities play a role in the execution of daily technological tasks. Cognitive effects are of course task dependent. In general, of all the cognitive measures, cognitive flexibility and the general speed of information processing were the best predictors of performance for the majority of the included tasks. Since knowledge about the role of cognitive skills when dealing with everyday technology is still very scarce, especially among older adults, future research should focus on developing methodologies to measure technological skills. Also, for the design of technological products older adults are able to use without problems, one should focus on the exact cognitive abilities that are engaged when performing individual tasks.
Acknowledgements
We thank Nico Rozendaal for assistance in the computer training of the participants and in providing technological support and assistance in data management. This study was funded by a grant from the Dutch Research Council (NWO: 014-91-048).
