The Austin Maze is a neuropsychological assessment tool commonly used within both research and clinical domains to measure cognitive function (Walsh, Reference Walsh1991). Originally developed from the Spatial Maze Learning Test (Milner, Reference Milner1965), the task consists of a 10 × 10 grid whereby examinees are required to learn a hidden pathway from start to finish. The Austin Maze has been found to be sensitive to cognitive dysfunction across a range of clinical disorders, including traumatic brain injury (Walsh, Reference Walsh1994), alcohol abuse (Bowden, Reference Bowden1988; Bowden & McCarter, Reference Bowden and McCarter1993; Unkenstein & Bowden, Reference Unkenstein and Bowden1991) and even psychiatric disorders such as schizophrenia (Mattson, Berk, & Lucas, Reference Mattson, Berk and Lucas1997). However, significant controversy exists regarding which cognitive constructs the Austin Maze measures. Previous researchers have utilised two methodological approaches to address this research question. Some have investigated associations between the Austin Maze and discrete brain lesions, while others have explored associations with other established neuropsychological measures. Both these approaches are reviewed below.
Using a brain lesion approach, Milner (Reference Milner1965) reported rule-following difficulties on the Spatial Maze Learning Test in patients with frontal lobe lesions, while Hohne and Walsh (Reference Hohne and Walsh1970) reported reduced learning on the Austin Maze in a group of prefrontal leucotomy patients. Based on several clinical case studies, Walsh (Reference Walsh1985) suggested that people with frontal lobe damage exhibit difficulty with error utilisation and thus have difficulty obtaining a stable, error-free performance on the Austin Maze. Based on the above, many have conceptualised the Austin Maze as sensitive to frontal lobe damage. However, this supposition has been challenged. Bowden and Smith (Reference Bowden and Smith1994) found that early performance on the Austin Maze predicted later trial performance equally across healthy and clinical populations, which is not consistent with Walsh's theory of protracted Austin Maze learning in frontal lobe patients. In addition, Hohne and Walsh (Reference Hohne and Walsh1970) found no difference in rule-break errors between healthy and prefrontal leucotomy patients. Perhaps more consistent are reports of right posterior hemisphere contributions to Austin Maze performance. Milner (Reference Milner1965) reported reduced maze learning in patients with right posterior lesions, with relatively intact performance in a left posterior lesion group. Newcombe, Ratcliffe and Damasio (Reference Newcombe, Ratcliff and Damasio1987) also reported reduced Austin Maze performance in a patient group with selective right posterior lesions.
The above-mentioned brain lesion studies suggest that right posterior hemisphere and perhaps frontal lobe damage is associated with reduced Austin Maze performance. However, a number of methodological issues inherent in brain lesion studies limit the reliability and validity of these findings (Rorden & Karnath, Reference Rorden and Karnath2004). Brain lesion studies often rely on opportunistic sampling, limiting how well groups can be matched and controlled. In addition, lesion studies often do not account for other sources of co-existing brain damage, such as diffuse axonal injury in traumatic brain injury samples or small vessel disease in stroke samples. More recent studies have investigated associations with other established measures of neuropsychological function in healthy populations as a means of exploring the cognitive determinants of Austin Maze performance. Bowden et al. (Reference Bowden, Dumendzic, Clifford, Hopper, Tucker and Kinsella1992) reported specific visuospatial contributions to Austin Maze performance, based on a significant association with the Block Design subtest of the Wechsler Adult Intelligence Scale – Revised (WAIS-R) (Wechsler, Reference Wechsler1981) and a comparatively minor association with the Vocabulary subtest of the WAIS-R. Tucker, Kinsella, Gawith and Harrison (Reference Tucker, Kinsella, Gawith and Harrison1987) reported a significant association between the Austin Maze and a measure of non-verbal intellectual function. Crowe et al. (Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999) have conducted the most comprehensive investigation using this methodology. They explored the associations between a range of cognitive constructs and Austin Maze measures using hierarchical regression. Visuospatial ability and visuospatial memory correlated with all three Austin Maze measures. Associations with executive measures such as planning, error utilisation and working memory were comparatively low. The authors concluded that the Austin Maze was predominantly a measure of visuospatial ability and visuospatial memory. The methodological design of the Crowe et al. (Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999) study was of a high standard, although some limitations have been identified. For example, the planning time score from the Tower of London test was used as a measure of planning ability. However, incorrectly completed items were excluded from analysis, potentially biasing against detection of poor performance. Furthermore, no further research to date has been conducted to confirm the reliability of the findings.
Recently, an updated computer version of the Austin Maze has become available, published by the Australasian Society for the Study of Brain Impairment (Bray & McDonald, Reference Bray and McDonald2010). The ease with which this test can be accessed makes it attractive to clinicians and researchers alike. Although this computerised version is generally true to the conventional Austin Maze in its design, minor modifications were made to the instructions for the computer version to aid participants’ understanding. The most crucial difference is increased feedback on the computer version. Unlike the conventional version, the number of errors made on each trial is revealed to the examinee at the end of each trial. The last correct square also remains green until examinees move on to the next correct square of the hidden pathway. There are many benefits of computer-based neuropsychological testing, including ease of administration, data storage and the ability to measure important aspects of performance such as performance variability (Bugbee, Reference Bugbee1996). It is recognised, however, that equivalence between traditional and computer-based versions of tests cannot be assumed, even when test stimuli are similar (Bugbee, Reference Bugbee1996; Bugbee & Bernt, Reference Bugbee and Bernt1990). This highlights the need to investigate equivalency when a new computerised version of a test is developed. Recently, McKay and colleagues compared performance across the conventional and computerised versions of the Austin Maze (McKay, Lee, Stolwyk, & Ponsford, Reference McKay, Lee, Stolwyk and Ponsford2012). They found no differences in terms of mean scores and distributions of performance between versions, and also reported significant correlations in performance across versions. These findings supported the equivalency of the conventional and computer versions of the Austin Maze based on performance. However, as yet, no study has investigated whether the two versions are measuring equivalent cognitive constructs. It is possible that the computerised Austin Maze may possess different perceptual or attentional demands compared to the conventional version. Furthermore, the computerised Austin Maze (Bray & McDonald, Reference Bray and McDonald2010) provides higher levels of overt feedback to examinees and thus may possess different ‘executive’ demands compared to the conventional version.
To summarise the above, although the Austin Maze has been found sensitive to cognitive dysfunction across a range of clinical populations, only one study to date (Crowe et al., Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999) has comprehensively investigated the cognitive determinants of this measure. Furthermore, recent research has highlighted that a new computerised version of the Austin Maze is equivalent to the conventional version in terms of means and distributions of performance. However, equivalency in terms of the cognitive constructs measured by these different versions has yet to be established. The current study had two major aims. First, we aimed to follow a similar methodology to Crowe et al. (Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999) and investigate what cognitive constructs are associated with conventional Austin Maze performance in a healthy population. Based on Crowe et al.'s (Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999) findings we hypothesised that visuospatial ability and visuospatial memory would be significantly associated with all Austin Maze performances. The second aim of this study was to compare the cognitive constructs associated with conventional and computer Austin Maze performances in a healthy population. We hypothesised that the cognitive determinants of the Austin Maze would be consistent across conventional and computer modalities.
Method
Participants
Sixty-three healthy people (23 male, 40 female) aged between 18 and 27 years (mean (M) = 21.51, standard deviation (SD) = 2.47) participated in this study. Participants were recruited from the general community and from a pool of undergraduate university students. The formal education of participants ranged between 11 and 20 years (M = 14.63, SD = 2.09). Participants were excluded from the study if English was not their first language, if they had a history of major psychiatric disorder, neurological disorder, alcohol or drug dependency, or learning disorder, or if they were taking medication that could significantly impair cognitive function.
Materials
Both the conventional and computer versions of the Austin Maze were included in this study, in addition to five cognitive tests purporting to measure planning, error utilisation, working memory, visuospatial ability and visuospatial memory. The five cognitive constructs and the associated measures were selected for inclusion based on the original study by Crowe et al. (Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999) to enable comparison of findings across studies. However, some minor modifications were made. First, due to time constraints, only one measure of working memory was included. The Digit Span Backwards subtest of the Wechsler Adult Intelligence Test (4th ed.) (WAIS-IV; Wechsler, Reference Wechsler2008) as opposed to the Brown–Peterson task (Peterson & Peterson, Reference Peterson and Peterson1959) was selected, based on a stronger association with Austin Maze performances noted by Crowe et al. (Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999). Second, as noted above, Crowe and colleagues (Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999) used total planning time (which is the time taken to initiate a response following the presentation of a problem) for correctly completed items as their measure of planning. However, this exclusion of incorrectly completed items may have biased against detection of some poor performances. In light of this limitation, total additional moves (which is the number of moves beyond the minimal required to complete an item) was selected as the measure of planning in this study.
Austin Maze
The conventional version of the Austin Maze was administered in accordance with the instructions of Walsh (Reference Walsh1987, Reference Walsh1991). The computerised version was adminstered in accordance with the computerised Austin Maze manual (Bray & McDonald, Reference Bray and McDonald2010). To minimise practice effects, participants completed each version of the Austin Maze test using a different pathway – original and alternate. The Austin Maze test was concluded when participants either achieved the criterion of 3 consecutive error-free trials or they reached 20 trials, whichever came first. A limit of 20 trials was imposed as studies have suggested that not all healthy participants are able to achieve criterion (Bowden et al., Reference Bowden, Dumendzic, Clifford, Hopper, Tucker and Kinsella1992). Also, findings from studies of healthy participants have shown that the average number of trials required to reach criterion generally ranged from 7 to 18 (Bowden, Reference Bowden1989; Crowe et al., Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999; Morrison & Gates, Reference Morrison and Gates1988; Tucker et al., Reference Tucker, Kinsella, Gawith and Harrison1987). An arbitrary score of 21 trials to criterion was given to all participants who failed to reach criterion within the 20 trials. To allow direct comparison with previous studies, measures selected for analysis were number of trials to criterion, cumulative errors to criterion and cumulative errors within the first 10 trials.
Planning
The Tower of London test (TOL) (Shallice, Reference Shallice1982) purports to assess higher-order problem solving, specifically executive planning abilities. Participants are required to determine the order of moves required to rearrange three balls on three sticks, from an original position to a new position, according to a set of constraint rules. Participants are reminded the test is not one of speed, but of their ability to solve each problem in as few moves as possible. This reminder is used to encourage examinees to plan their moves before commencing. The TOL was administered in accordance with the test manual (Culbertson & Zillmer, Reference Culbertson and Zillmer2005). Total number of additional moves was selected as the measure of planning (higher score reflects worse performance).
Error Utilisation
The Wisconsin Card Sorting Test (WCST) (Heaton, Chelune, Talley, Kay, & Curtiss, Reference Heaton, Chelune, Talley, Kay and Curtiss1993) purports to measure a person's ability to use environmental feedback to shift cognitive set. Participants are required to sort a set of cards according to rules including colour, shape and number. The sorting rule changes throughout the task and participants must deduce which rule is current based on examiner feedback. The WCST was administrated in accordance with the test manual. The measure used in data analysis was the total number of cards correctly sorted from a pack of 64 cards (higher score reflects better performance).
Visuospatial Ability
The Block Design subtest from the WAIS-IV (Wechsler, Reference Wechsler2008) was used to assess visuospatial ability. It requires participants to construct a three-dimensional formation of blocks based on a two-dimensional model. This test was administered and scored in accordance with the WAIS-IV manual and the raw score was utilised for analysis (minimum 0, maximum 64; higher score reflects better performance).
Visuospatial Memory
The Visual-Spatial Learning Task (VSLT) (Malec, Ivnik, & Hinkeldey, Reference Malec, Ivnik and Hinkeldey1991; Malec et al., Reference Malec, Ivnik, Smith, Tangalos, Petersen, Kokmen and Kurland1992) purports to measure visuospatial memory. This test requires participants to learn and recognise 7 of 15 designs and to recall the correct placement of these designs on a 6 × 4 matrix. Participants have a maximum of five learning trials. The test is concluded when participants achieve two error-free trials. A number of scores can be generated from the VSLT, including number of design errors (minimum 0, maximum 35), number of position errors (minimum 0, maximum 35) and a combined design/position error count (minimum 0, maximum 70; all higher scores reflect worse performance).
Working Memory
The Digit Span Backwards (DSB) subtest of the WAIS-IV (Wechsler, Reference Wechsler2008) was used to assess working memory ability. Participants are required to repeat a progressively increasing sequence of numbers in reverse order. The DSB was administered and scored according to the WAIS-IV manual. DSB raw score was included for analysis (minimum 0, maximum 14; higher score reflects better performance).
Procedure
Ethics approval from the institution where this research was conducted was obtained prior to commencing the study. Participants provided informed consent prior to assessment. Demographic and other information relating to inclusion criteria were obtained first. Participants began with the first administration of the Austin Maze. The five cognitive tests were then administered in randomised order to control for the effects of fatigue. Participants ended the testing session with the second administration of the Austin Maze test. To control for practice effects, the order in which participants completed different versions of the Austin Maze (conventional, computerised) was counterbalanced across participants. Each testing session was approximately 1 hour in duration. The researcher who administered and scored all tests was trained and supervised by a registered psychologist.
Data Analysis
Hierarchical regression was used to examine which cognitive variables (independent variables) were associated with various Austin Maze performance measures (dependent variables). Assumptions of linearity, normality and homoscedasticity of residuals were assessed by examination of scatterplots of the residuals and Normal Probability Plots (P-P) of the regression standardised residuals. Points on the Normal P-P lay in a reasonably straight diagonal line which suggested no major deviations from normality. For the scatterplots of the standardised residuals, assumption of homoscedasticity was not violated as the residuals were in a relatively rectangular distribution, mainly clustering around zero. The presence of multivariate outliers was checked by inspecting the Mahalanobis distances. None of the values exceeded the critical chi-square value, using an alpha level of .001 (Tabachnick & Fidell, Reference Tabachnick and Fidell2007). The correlation matrix was examined to check for multicollinearity. With the exception of the three measures of VSLT, correlations between independent variables were generally low to moderate, reducing the chance of redundancy in the contribution of any one variable to the regression equation. High correlations were observed between the three VSLT scores, ranging from .64 to .88. According to Pallant (Reference Pallant2011), two independent variables with bivariate correlations above .7 should not be included in the same regression analysis. Thus, only one of the three VSLT variables, VSLT position errors, was entered into the regression analysis. This variable was selected for inclusion because it best reflects the spatial, as opposed to visual, demands of the Austin Maze task.
Results
Data describing the Austin Maze and other cognitive test performances are displayed in Table 1.
Means, Standard Deviations and Ranges of Cognitive Performances
Six separate hierarchical regression analyses were conducted to examine which cognitive constructs were associated with different Austin Maze versions (conventional, computerised) and measurement variables (errors to criterion, errors to 10 trials, trials to criterion). Independent variables were common to each analysis. Age, gender and years of education were entered into the regression at Step 1 to control for their influence. Due to the lack of theoretical basis for ordering, the five cognitive variables (planning, error utilisation, visuospatial ability, visuospatial memory and working memory) were entered together into the regression analysis at Step 2.
As shown in Tables 2 and 3, demographic variables including age, gender and years of education did not significantly contribute to any Austin Maze performance across versions (conventional, computer) or measurement variables (errors to criterion, errors to trial 10, trials to criterion) when entered into the regression equation at Step 1 (a criterion of p < .05 was used). By contrast, the cognitive variables (planning, error utilisation, visuospatial ability, visuospatial memory and working memory) significantly contributed to Austin Maze performances across all measurement variables and versions when entered at Step 2. As a group, the cognitive variables generally accounted for approximately 15–30% of Austin Maze performances across measurement variables. The contribution of cognitive variables to computerised Austin Maze performance (26–29%) was slightly higher compared to the conventional version (15–20%).
Summary of Hierarchical Regression Analyses for the Conventional Version of the Austin Maze
* = p < .05
Summary of Hierarchical Regression Analyses for the Computer Version of the Austin Maze
* = p < .05
A consistent pattern emerged across regression models when examining which specific cognitive abilities contributed to Austin Maze performances. Across versions and measurement variables, visuospatial ability and visuospatial memory significantly contributed to five of six Austin Maze performance measures. The only exception was lack of visuospatial contributions to trials to criterion on the conventional version. ‘Executive’ abilities including planning, error utilisation and working memory did not uniquely contribute to any conventional or computerised Austin Maze performances.
Discussion
The first aim of this study was to examine which cognitive constructs were associated with the conventional version of the Austin Maze. Visuospatial ability and visuospatial memory significantly contributed to almost all conventional Austin Maze performance measures. In contrast, ‘executive’ measures, including working memory, planning and error utilisation, did not uniquely contribute to any conventional Austin Maze measures. These findings are consistent with Crowe et al.'s (Reference Malec, Ivnik and Hinkeldey1999) earlier study and further challenge the early assumption of some researchers and clinicians who conceptualised the Austin Maze as a measure of executive function.
The second aim of this study was to examine whether the conventional and computer versions of the Austin Maze were associated with similar cognitive constructs. Current findings generally support this supposition. Consistent with the conventional version, the computer version of the Austin Maze was predominantly associated with visuospatial ability and visuospatial memory and not with ‘executive’ measures such as working memory, planning and error utilisation. The only difference observed was a significant contribution of visuospatial ability to trials to criterion on the computer version but not the conventional version. Interestingly, this significant finding on the computer version is more consistent with previous studies (Crowe et al., Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999; Tucker et al., Reference Tucker, Kinsella, Gawith and Harrison1987) than the non-significant finding observed on the conventional version in the current study. As a whole, current results complement an earlier study which reported equivalence of the conventional and computer versions of the Austin Maze with regard to performance (McKay et al., Reference McKay, Lee, Stolwyk and Ponsford2012). Our findings suggest that the cognitive functions underpinning performance of each version of the Austin Maze, namely visuospatial ability and visuospatial memory, are also similar. These results further support equivalency of the two versions according to American Psychological Association guidelines (1986). Importantly, this demonstration of equivalency supports the translation of previous research findings and established normative data across the conventional and new computerised version of the Austin Maze.
This study explored the cognitive determinants of Austin Maze performance by investigating associations with other established cognitive measures. While our approach avoided certain methodological issues associated with brain lesion studies, it also has some limitations. First, it is acknowledged that most cognitive tests are multifactorial in nature and cannot be considered a ‘pure measure’ of certain cognitive constructs. This issue is particularly pertinent regarding assessment of ‘executive functions’ which, in essence, is an umbrella term encompassing numerous higher-order cognitive abilities (Miyake, Friedman, Emerson, Witzki & Howerter, Reference Miyake, Friedman, Emerson, Witzki and Howerter2000). Previous reviews in this area have noted that executive function tasks rely on a combination of higher-order processes (Stuss, Reference Stuss2011). For example, while we have used the WCST as a measure of ‘error utilisation’ we acknowledge that other executive skills, such as mental flexibility, are implicated in WCST performance (Lezak, Howieson, Bigler, & Tranel, Reference Lezak, Howieson, Bigler and Tranel2012; Strauss, Sherman, & Spreen, Reference Strauss, Sherman and Spreen2006). While great care was taken to select measures that were most representative of the cognitive constructs of interest, further research using alternate tests purporting to measure these constructs would help support the reliability and validity of our findings. For example, we acknowledge that the Digits Backwards task used to measure working memory in this study utilises an auditory–verbal modality and recommend the use of a visuospatial working memory task in future related research. Second, it is acknowledged that the cognitive measures used in this study accounted for a limited amount of the variability in Austin Maze performances (between 15% and 30%). Further research is warranted to explore other cognitive and non-cognitive constructs that may also contribute to Austin Maze performance. In particular, while number of errors on the WCST is a well-recognised indicator of error utilisation, it would be interesting to investigate whether perseverative errors on the WCST would significantly contribute to measures of perseverative errors on the Austin Maze. Third, it is acknowledged that a relatively young, healthy and educated sample was utilised in this study. Based on data presented in Table 1, it does not appear that results were compromised by ceiling effects and/or lack of variability in performances. However, previous research has demonstrated that the cognitive constructs measured by various tools can vary across healthy and clinical populations. For example, verbal fluency has been shown to be more strongly associated with speed and information processing in older, compared to younger, healthy people (Hughes & Bryan, Reference Hughes and Bryan2002). Thus, further research is warranted to investigate whether the cognitive determinants of performance on the Austin Maze vary across broader healthy and clinical populations. It would be particularly interesting to investigate whether ‘executive’ associations with the Austin Maze emerge in clinical populations with known executive dysfunction. Fourth, it is acknowledged that the regression analyses conducted in this study were slightly underpowered. However, considering that the methodology used was closely aligned with that of Crowe et al. (Reference Crowe, Barclay, Brennan, Farkas, Gould, Katchmarsky and Vayda1999) and the same pattern of results emerged, we can be confident that the pattern of findings from current regression analyses is robust.
In conclusion, current findings are consistent with previous research and demonstrate that the Austin Maze is primarily a measure of visuospatial ability and visuospatial memory in a young, healthy population. Furthermore, we have extended previous findings and demonstrated that not only are means and distributions of performance equivalent across the conventional and computer versions of the Austin Maze, but they also measure similar cognitive constructs. A limited number of tools are available that purport to measure visuospatial learning and memory. The Austin Maze has a number of positive attributes, including a parallel form for repeat testing, established normative data and no obvious ceiling effects, and it has demonstrated sensitivity across a range of clinical populations. With a new computerised version increasing accessibility of the measure, the Austin Maze is an attractive option for visuospatial memory measurement across both research and clinical domains.
Author note
This manuscript has not been published elsewhere and has not been submitted simultaneously for publication in another source. All authors have contributed significantly, and are in agreement with the content of this manuscript. No conflict of interest is associated with any authors of this manuscript. This research received no specific grant from any funding agency, commercial or not-for-profit sectors.
