INTRODUCTION
Tests of Design Fluency (DF), also known as “figural” or “nonverbal” fluency, represent a method of assessment of executive functioning, commonly used in research and clinical settings. Examinees draw as many different designs as possible in one minute, while avoiding repeating prior designs. There are several versions of DF tests, most of which require that designs be drawn by connecting dots in a series of five-dot matrices (see Table 1 for a review of common DF tests). Given the general trend in clinical neuropsychology toward interpreting test results in terms of cognitive constructs, rather than lesion locations, understanding the cognitive underpinnings of DF performance is important. However, construct validation studies of DF have almost entirely relied on examining the neuroanatomic substrates of DF performance (Baldo, Shimamura, Delis, Kramer, & Kaplan, Reference Baldo, Shimamura, Delis, Kramer and Kaplan2001; Butler, Rorsman, Hill, & Tuma, Reference Butler, Rorsman, Hill and Tuma1993; Elfgren & Risberg, Reference Elfgren and Risberg1998; Fama et al., Reference Fama, Sullivan, Shear, Cahn-Weiner, Marsh and Lim2000; Kramer et al., Reference Kramer, Quitania, Dean, Neuhaus, Rosen and Halabi2007; Suchy, Sands, & Chelune, Reference Suchy, Sands and Chelune2003; Tucha, Smely, & Lange, Reference Tucha, Smely and Lange1999), with the assumption that sensitivity to frontal lobe pathology implies sensitivity to executive dysfunction.
Design Fluency tests overview
Note
Design Fluency (Jones-Gotman & Milner, Reference Jones-Gotman and Milner1977); Five-Point Test (Regard, Strauss, & Knapp, 1982); Ruff Figural fluency Test (Ruff, Reference Ruff1998), and D-KEFS Design Fluency (Delis, Kaplan, & Kramer, Reference Delis, Kaplan and Kramer2001).
A handful of studies that did examine the neurocognitive constructs that underpin DF performance have been summarized in the RFFT: Ruff Figural Fluency Test Professional Manual (Ruff, 1996). They suggest that, based on factor-analytic findings (Baser & Ruff, Reference Baser and Ruff1987), DF is a measure of “initiation, planning, and divergent reasoning” (p. 15), and that poor DF performance cannot be explained by language, memory, or motor deficits (Ruff, Evans, & Marshall Reference Ruff, Evans and Marshall1986). Beyond these empirical findings, the Delis-Kaplan Executive Function System (D-KEFS) version of DF (Delis et al., Reference Delis, Kaplan and Kramer2001) has been described as “fluency in generating visual patterns” (p. 88), likely based on the self-evident fact that “visual patterns” are the output of DF performance.
To examine the assertions that DF is a measure of (a) planning/initiation, (b) cognitive flexibility/divergent thinking, and (c) fluency in generating visual patterns, we recently conducted a study (Kraybill & Suchy, Reference Kraybill and Suchy2008) in which we used the Ruff Figural Fluency Test (RFFT; Ruff, 1998) as the dependent variable, and several motor and executive measures as predictors. Two of the motor variables used in that study proved to be particularly useful in examining the construct of DF. These were “motor planning” and “motor fluency.”
Motor planning refers to the internal strategy that precedes an intended movement (Keele, Reference Keele and Brooks1981), and presumably contains both general information about the intended goal and specific information about the neuromuscular control that will be required (Keele, Reference Keele and Brooks1981). It was operationalized as the latencies prior to initiation of correctly executed sequences of specified hand movements using the Push-Turn-Taptap task from the Behavioral Dyscontrol Scale–Electronic Version battery (BDS-EV; Suchy, Derbidge, & Cope, Reference Suchy, Derbidge and Cope2005). In particular, participants learned four different sequences (or permutations) of three specified hand movements performed on a specialized response console (Figure 1). The task is described in more detail in Figure 2a.
The figure depicts the Behavioral Dyscontrol Scale–Electronic Version (BDS-EV) response console. It was used in the present study to assess Motor Planning and Motor Sequence Fluency.
The BDS-EV Push-Turn-Taptap task requires that participants learn different sequences (or permutations) of three specified hand movements, using a specialized response console (Figure 1). The three hand movements are “Push” – pushing the joystick forward; “Turn” – turning the joystick clockwise; and “Taptap” – double-tapping on the white dome of the response console. The task begins by presenting a two-movement sequence on the computer screen, until three correct trials are completed. Following these three learning trials, participants continue to perform the sequence from memory, until accomplishing five additional correct trials. After completing the five correct trials, a new sequence is presented on the screen, and the just described process is repeated. There is a total of four different progressively longer sequences that participants learn in the course of this task (only the first two sequences are presented in this figure). Mistakes are followed by an audible tone, along with a presentation of the correct sequence on the screen. Motor Planning (M-PLN) latencies are indicated in the figure by the thick black vertical arrows, and reflect the preparation time before initiation of each trial. Only latencies preceeding correctly executed trials are included.
This figure depicts a hypothetical performance of a participant on the first two blocks of the Motor Sequence Fluency test. Participants are asked to generate as many unique sequences of three previously learned hand movements as they can. The movements are push, turn, and tap-tap, performed on the BDS-EV response console depicted in Figure 1. The test is organized into three blocks of progressively longer sequences, with the first block requiring that exactly three hand movements be strung together to create unique sequences, and the subsequent two blocks requiring that exactly four and five hand movements be strung together into sequences. Repetition of the same movements within a given sequence is permitted. However, repetition of the exact same sequence within a block constitutes an error. Only correct (i.e., unique) sequences are counted to generate the final score. The allotted time is individually determined based on participants’ performance speed on the Push-Turn-Taptap task administered just prior to the Motor Sequence Fluency test.
Motor fluency refers to the ability to generate novel sequences of motor actions. It was operationalized as the total number of unique permutations of three specified hand movements produced on the BDS-EV Motor Sequence Fluency test. The Motor Sequence Fluency test uses the same response console (Figure 1) and the same hand movements as the Push-Turn-Taptap task used for assessment of motor planning, and is described in more detail in Figure 2b.
In a series of hierarchical regressions designed to parse out unique and shared variances among the variables, the DF number of unique designs was predicted by (a) Trail Making Test-Part B (TMT-B), presumably reflecting cognitive flexibility, and (b) the two motor variables described earlier, motor planning and motor fluency. Together, these findings supported the prevailing conceptualization of DF as a test of cognitive flexibility and planning (even if just planning of motor actions) (Delis et al., Reference Delis, Kaplan and Kramer2001; Ruff, 1998). Additionally, these findings introduced the notion that DF may rely on fluency in generating motor actions, in addition to “fluency in generating visual patterns” (Delis, Reference Delis, Kaplan and Kramer2001; p. 88). Finally, consistent with prior findings (Ruff et al., Reference Ruff, Evans and Marshall1986), motor speed, assessed via finger tapping, did not contribute to performance.
However, several questions remained. First, it was not clear whether the association between DF and TMT-B was a reflection of shared cognitive flexibility, or a reflection of shared component skills (i.e., visual scanning, motor speed, sequencing). The latter explanation would be consistent with the assertion of Delis and colleagues (Reference Delis, Kaplan and Kramer2001, p. 89) that the Visual Scanning and Motor Speed conditions of the D-KEFS version of trail making can be used to assess component skills in DF performance.
Second, although we replicated previous findings that finger tapping speed was not related to DF performance (Ruff et al., Reference Ruff, Evans and Marshall1986), it was still possible that graphomotor skills (i.e., the successful wielding of a writing implement) may contribute to the number of generated designs, as suggested by Delis and colleagues (Reference Delis, Kaplan and Kramer2001, p. 89).
Third, while our prior study found that performance on the Motor Sequence Fluency test uniquely and significantly contributed to DF performance, it was not clear whether this relationship was due to some general fluency ability (i.e., one that would be shared by all fluency measures, both verbal and nonverbal), or whether it was specific to fluency in the motor domain.
The purpose of the present study was to address these issues by replicating and extending our prior research. To that end, we administered a battery of cognitive and motor tasks to a sample of community-dwelling elderly. To tease apart cognitive flexibility and component skills, the present study employed the D-KEFS version of trail making (Delis et al., Reference Delis, Kaplan and Kramer2001), which includes not only alpha-numeric sequencing as a measure of cognitive flexibility, but also carefully designed control tasks that assess visual scanning, sequencing, and graphomotor speed. Additionally, to address the question of whether DF is related to general fluency abilities (vs. a specific fluency in generating motor sequences, as assessed via the Motor Sequence Fluency test), we included the D-KEFS Letter Fluency test in this study. Lastly, given that adequate construct validation requires a multi-method approach, the present study employed the D-KEFS version of DF, as opposed to the RFFT used in our prior study (Ruff, 1998).
METHOD
Participants
Participants were 61 right-handed, independently functioning community-dwelling elderly (62 % female). They were recruited from the community via advertisements and received $10.00 an hour for their time. University of Utah Institutional Review Board-approved informed consent procedures and the APA ethical guidelines were followed. See Table 2 for participant characteristics.
Characteristics of the sample
Note
N = 61. M-DRS = Mattis Dementia Rating Scale. Two participants (3.3% of the sample) had M-DRS scaled scores of 5 (moderately impaired range), and two had scores of 6 and 7 (mildly impaired).
Instruments
Delis-Kaplan Executive Function System (D-KEFS; Delis et al., Reference Delis, Kaplan and Kramer2001). For all tests, standard administration and scoring procedures were followed unless otherwise noted. Untransformed raw scores were used as variables in analyses. The following three tasks from this battery were used:
The D-KEFS Design Fluency (DF) test consists of three trials in which participants create novel designs by connecting dots in a series of five-dot matrices. The three conditions are referred to as Filled Dots (connecting filled dots), Empty Dots (connecting empty dots while filled dots function as distractors), and Switch (switching between connecting filled and empty dots). The first two conditions are similar to the Ruff version of DF in that they assess DF both with and without visual distractors, whereas the third condition includes switching, which is not part of the Ruff version of the test. In the present study, the first two conditions and the third condition were examined separately, resulting in two DF variables: (a) The “Non-switch DF” score, comprised of the sum of correct unique designs generated during the first two conditions; and (b) the “Switch DF” score, comprised of the total number of correct unique designs generated during the “switch” condition.
The D-KEFS Trail Making Test consists of five conditions: Visual Scanning, Number Sequencing, Letter Sequencing, Number-Letter Switching, and Motor Speed. All five conditions were administered. For all conditions, participants are asked to work as quickly as they can, and time to completion (in seconds) represents their raw score. The first four conditions present the examinee with a page of pseudo-randomly arranged circles containing letters, numbers, or both. Visual Scanning requires crossing-out circles that contain a particular number. Number and Letter Sequencing require connecting circles in the correct numerical or alphabetical order, respectively. Number-Letter Switching requires connecting circles in an alternating alphanumeric sequence. Motor Speed consists of circles that are already connected by a line, and participants use a pencil to trace the line as fast as they can. The D-KEFS Trail Making Test includes these five conditions so as to allow assessment of the speed of cognitive switching after visual, motor, and sequencing speeds have been accounted for.
For the purpose of this study, to isolate cognitive flexibility, we computed D-KEFS Number-Letter Switching residuals, after controlling for D-KEFS Visual Scanning, Motor Speed, and both Sequencing Speeds. Additionally, D-KEFS Visual Scanning and Motor Speed were examined in analyses as potential component processes of DF, as was previously recommended by Delis and colleagues (Reference Delis, Kaplan and Kramer2001). Note that hereafter, the D-KEFS Motor Speed will be referred to as the D-KEFS Graphomotor Speed, so as to differentiate it from motor speed assessed in our prior study via finger tapping.
The D-KEFS Letter Fluency Test requires that participants generate as many words as they can that begin with a specific letter. The sum of the number of correct responses generated for three different letters (one minute each) represented the raw score used in analyses.
Behavioral Dyscontrol Scale-Electronic Version (BDS-EV; Suchy et al., Reference Suchy, Derbidge and Cope2005). The BDS-EV is an electronically administered battery of motor and executive tasks with excellent reliability (Suchy et al., Reference Suchy, Derbidge and Cope2005) and promising construct validity (Kraybill et al., Reference Kraybill, Suchy and Franchow2009; Suchy et al., Reference Suchy, Derbidge and Cope2005; Suchy & Kraybill, Reference Suchy and Kraybill2007). Two tasks from the battery (i.e., the Push-Turn-Taptap task and the Motor Sequence Fluency test) were used in this study and are described next.
The Push-Turn-Taptap task generates variables that reflect various components of motor programming. However, for the purpose of this study, only the BDS-EV Motor Planning (M-PLN) variable was used, as it has been previously found to share unique variance with the number of unique designs generated during DF performance (Kraybill & Suchy, Reference Kraybill and Suchy2008). The M-PLN variable has been shown to have a strong relationship with executive functions (Kraybill, Suchy, & Franchow, Reference Kraybill, Suchy and Franchow2009; Suchy & Kraybill, Reference Suchy and Kraybill2007). As seen in Figure 2a, the Push-Turn-Taptap task generates the M-PLN variable by measuring the amount of time (measured in milliseconds) that elapses between the completion of one repetition (or trial) of a given sequence and the initiation of the next correct repetition of that same sequence. Because there are four different sequences, and each sequence must be executed correctly eight times (i.e., three with computer prompts and five independently; see Figure 2a) before moving on to the next sequence, the M-PLN variable is based on a total of 32 planning latency observations.
The BDS-EV Motor Sequence Fluency test was administered immediately following the Push-Turn-Taptap task, using the same response console and the same three hand movements. Participants were asked to produce as many unique sequences (i.e., permutations) of the three hand movements as they could within an allotted amount of time. A hypothetical performance is depicted in Figure 2b.
Because the DF test gives all participants the same amount of time (1 minute per condition), whereas the Motor Sequence Fluency test automatically adjusts for participants’ performance speed based on their performance speed on the Push-Turn-Taptap task, the two tasks needed to be placed on a comparable time allotment metric. This was done by computing an unstandardized residual score of the total number of generated unique motor sequences, after controlling for Push-Turn-Taptap performance speed. This score was used in analyses.
Statistical Analyses
All principal analyses in this study employed hierarchical regressions, using DF as the criterion variable. As described in the method section, the “Non-switch” and the “Switch” DF conditions were examined separately.
Predictor variables were selected and entered based on (1) the hierarchical model of cognition, according to which lower-order component processes (i.e., perceptual and motor functions) are governed by, and contribute to, higher-order control processes (i.e., executive functions) (Stuss, Alexander, Benson, Trimble, & Cummings, Reference Stuss, Alexander, Benson, Trimble and Cummings1997), and (2) the assertions by Delis and colleagues regarding the nature of component processes presumed to contribute to DF performance (Delis et al., Reference Delis, Kaplan and Kramer2001; pp. 88–89).
Following this rationale, we first parsed out variance contributions from discrete component skills (i.e., D-KEFS Visual Scanning and Graphomotor Speed). Next, we examined contributions from executive processes not deemed specific to generative fluency (i.e., D-KEFS Number-Letter Switching residuals and the M-PLN variable from the BDS-EV). Lastly, we examined executive processes deemed specific to fluency performance (i.e., the BDS-EV Motor Sequence Fluency and the D-KEFS Letter Fluency). All analyses were conducted in sets of two, reversing the order of variable entries at consecutive steps, so as to allow partialing of unique versus shared variance contributions.
RESULTS
Preliminary Analyses
Descriptive statistics and zero-order correlations for all dependent and independent variables are presented in Tables 3 and 4, respectively.
Descriptive statistics for dependent and independent variables
Note
N = 61. DV = Dependent variable; IV = Independent variable; DF = Design Fluency; D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; NLS residuals = Number-Letter Switching residuals; MSF = Motor Sequence Fluency.
Zero order correlations among dependent and independent variables
Note
N = 61. * p < .05; ** p < .01; Sex: 1 = female, 2 = male; DF = Design Fluency; D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version.
Cognitive structure of the Non-switch Design Fluency (DF) condition
Contribution of D-KEFS Graphomotor Speed and Visual Scanning. D-KEFS Graphomotor Speed and Visual Scanning were entered on Steps 1 and 2, respectively, as well as in the reversed order (i.e., Visual Scanning and Graphomotor Speed on Steps 1 and 2, respectively). Summaries of analyses are presented in Table 5, with the italicized numbers in the R2 Δ column reflecting unique variance contributed by each variable. The results showed that Visual Scanning and Graphomotor Speed together yielded a statistically significant model, F (2, 58) = 5.18, p = .008, that accounted for 15.2% of variance in Non-switch DF (see column R2 in Table 5, last Step in either analysis) with approximately 6.6% of Non-switch DF variance shared between Graphomotor Speed and Visual Scanning (i.e., total variance minus unique variance, or .152 minus the sum of the italicized numbers in the R2 Δ column). However, Visual Scanning and Graphomotor Speed contributed insignificant amounts of unique variance (see p values in Steps 2 in the Table). Because of the significant contribution of the two variables together, both were entered on Step 1 of subsequent analyses.
Summary of hierarchical regressions reflecting variance contributions of Motor Speed and Visual Scanning to the number of Unique Designs generated in the first two trials of D-KEFS Design Fluency test
Note
Italicized numbers in the R 2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System.
Contribution of D-KEFS Number-Letter Switching residuals and BDS-EV Motor Planning (M-PLN). To determine whether the D-KEFS Number-Letter Switching residuals and/or the BDS-EV M-PLN contributed to performance above and beyond the D-KEFS Visual Scanning and Graphomotor Speed, we conducted another set of two hierarchical regressions, entering D-KEFS Visual Scanning and Graphomotor Speed on Step 1, and D-KEFS Number-Letter Switching residuals and the BDS-EV M-PLN variable on Steps 2 and 3, respectively, as well as in reversed order (i.e., M-PLN and Number-Letter Switching residuals on Steps 2 and 3, respectively). See Table 6 for summary of analyses (unique variance is italicized). Taken together, the results showed that the M-PLN variable uniquely contributed 11.9% of variance to Non-switch DF, with approximately an additional 4.2% of variance shared between M-PLN and D-KEFS Number-Letter Switching residuals. Unique contribution from D-KEFS Number-Letter Switching residuals was small (1.7%) and did not reach statistical significance. For that reason, M-PLN was the only variable added to the model in the next set of analyses.
Summary of hierarchical regression reflecting variance contributions of Visual Scanning, Motor Speed, Cognitive Flexibility, and Motor Planning to the number of Unique Designs generated in the first two trials of the D-KEFS Design Fluency test.
Note
Italicized numbers in the R 2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; NLS residuals = Number-Letter Switching residuals.
Contribution of BDS-EV Motor Sequence Fluency and D-KEFS Letter Fluency. We first entered the three significant predictors from previous analyses (i.e., Graphomotor Speed, Visual Scanning, and M-PLN) on Step 1. Next, to determine whether the ability to generate novel designs depended on the ability to generate novel motor sequences versus the ability to simply be generative in any domain (even verbal), we entered Motor Sequence Fluency and Letter Fluency on Steps 2 and 3, respectively, as well as in the reversed order (i.e., Letter Fluency on Step 2 and Motor Sequence Fluency on Step 3). See Table 7. Taken together, the above analyses revealed that Motor Sequence Fluency and Letter Fluency each uniquely contributed 7.0% and 2.8% of variance, respectively, with virtually no overlap in variance (i.e., 1.2%). Only the contribution from Motor Sequence Fluency reached statistical significance, suggesting that it was a better predictor of Non-switch DF than was Letter Fluency.
Summary of hierarchical regression reflecting variance contributions of Visual Scanning, Motor Speed, Cognitive Flexibility, Letter Fluency, and Motor Sequence Fluency to the number of Unique Designs generated in the first two trials of the D-KEFS Design Fluency test
Note
Italicized numbers in the R 2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; MSF = Motor Sequence Fluency.
The final model then included D-KEFS Visual Scanning and Graphomotor Speed, and BDS-EV M-PLN and Motor Sequence Fluency, which together accounted for 39.5% of variance, F (4, 56) = 9.15, p < .001.
Cognitive structure of the Switch Design Fluency (DF) condition
The statistical approach employed with the Non-switch DF (i.e., the order of variable entry, computation of unique vs. shared variance, etc.) was also employed when using the Switch DF as the criterion variable.
Contribution of D-KEFS Graphomotor Speed and Visual Scanning. See Table 8 for a summary of the results. As can be seen, in contrast to Non-switch DF, Switch DF relied more on Visual Scanning, which contributed 13.1% of unique variance and an additional 6.7% of variance that was shared with Graphomotor Speed. This finding likely reflected the need to alternate between searching for filled and empty dots. Graphomotor Speed did not contribute unique variance to the model. Based on these results, we entered the D-KEFS Visual Scanning as the only predictor on Step 1 in the subsequent set of analyses.
Summary of hierarchical regressions reflecting variance contributions of Motor Speed and Visual Scanning to the number of Unique Designs generated in the Switch condition of D-KEFS Design Fluency test
Note
Italicized numbers in the R 2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System.
Contribution of D-KEFS Number-Letter Switching residuals and the BDS-EV Motor Planning (M-PLN). The next set of analyses was conducted with D-KEFS Visual Scanning entered on Step 1, and D-KEFS Number-Letter Switching residuals and BDS-EV M-PLN entered on Steps 2 and 3 and vice versa (Table 9). The results showed that Number-Letter Switching residuals and M-PLN uniquely contributed approximately 2.9% and 3.6% of variance, with approximately an additional 2.5% of variance shared between them, for a total of 9.0% of variance above and beyond D-KEFS Visual Scanning. Although neither variable contributed significantly alone, they contributed significantly together, and thus were both added to the model in the next set of analyses.
Summary of hierarchical regression reflecting contributions of Visual Scanning, Cognitive Flexibility, and Motor Planning to the number of Unique Designs generated in the Switch condition of the D-KEFS Design Fluency test
Note
Italicized numbers in the R 2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; NLS residuals = Number-Letter Switching residuals.
Contribution of BDS-EV Motor Sequence Fluency and D-KEFS Letter Fluency. The next set of analyses was conducted with D-KEFS Visual Scanning, BDS-EV M-PLN, and D-KEFS Number-Letter Switching residuals entered together on Step 1, and BDS-EV Motor Sequence Fluency and D-KEFS Letter Fluency entered on Steps 2 and 3, and vice versa (Table 10). The results showed that, in contrast to Non-switch DF, neither Motor Sequence Fluency nor Letter Fluency contributed further to the model. The final model then included D-KEFS Visual Scanning, D-KEFS Number-Letter Switching residuals, and BDS-EV M-PLN, which together accounted for 28.3% of variance (Table 10, Step 1), suggesting that the Switch DF relied less on fluency than the Non-switch DF did.
Summary of hierarchical regression reflecting variance contributions of Visual Scanning, Cognitive Flexibility, Letter Fluency, and Motor Sequence Fluency to the number of Unique Designs generated in the Switch condition of the D-KEFS Design Fluency test
Note
Italicized numbers in the R 2 Δ column reflect unique variance contributions. df = degrees of freedom. The Step 1 values in the F Δ column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; MSF = Motor Sequence Fluency; NLS residuals = Number-Letter Switching residuals.
Effects of demographic variables
As was done in our previous study, we examined whether demographic variables could account for the findings. Thus, we ran the two final models (one each for Non-switch DF and Switch DF), entering age, education, and sex on Step 1, and all the significant predictors identified in previous analyses on Step 2. The results showed that the previously identified predictors together accounted for 25.4% of variance in Non-switch DF above and beyond demographics, F change (4, 53) = 5.83, p = .001, and 18.4% of variance in Switch DF above and beyond demographics, F change (3, 54) = 4.73, p = .005. A summary of these models appears in Table 11.
Summary of final hierarchical regression models for the number of unique designs in the D-KEFS Design Fluency test
Note
N = 61. M-PLN = Motor Planning; MSF = Motor Sequence Fluency. The Step 1 values in the F Δ column in reality represent F, not F Δ, as they are the values for the base model. D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; MSF = Motor Sequence Fluency; NLS residuals = Number-Letter Switching residuals.
Finally, while hierarchical regressions allow parsing out of unique variance contributions, they do not allow determination of the most parsimonious model (i.e., variables that are the best overall predictors). To provide that information, we present simple linear regressions in Table 12. As can be seen, BDS-EV Motor Sequence Fluency and M-PLN represented the two most prominent predictors for Non-switch DF, as was the case in our prior study. In contrast, D-KEFS Visual Scanning represented the single best predictor for the Switch DF condition.
Summary of final linear regression models for the number of unique designs in the D-KEFS Design Fluency test
Note
N = 61. D-KEFS = Delis-Kaplan Executive Function System; BDS-EV = Behavioral Dyscontrol Scale−Electronic Version; M-PLN = Motor Planning; MSF = Motor Sequence Fluency; NLS residuals = Number-Letter Switching residuals.
Supplementary analyses
Our findings demonstrated that the Switch DF relied much more heavily on Visual Scanning than Non-switch DF did, presumably because of the need to visually separate empty and filled dots. However, the Non-switch DF itself contained data from two trials, one with filled dots only, and one with filled and empty dots (filled dots serving as distracters). To see whether this apparently slight difference reliably affected the degree to which Visual Scanning contributed to performance, we examined the contributions of D-KEFS Visual Scanning and Graphomotor Speed to Trial 1 (Filled dots) and Trial 2 (Filled and Empty dots) separately. The results showed that, as suspected, Trial 2 relied more heavily than Trial 1 on D-KEFS Visual Scanning. In particular, Visual Scanning contributed 5.4% of unique variance to Trial 2, as compared to 3.4% of variance to Trial 1. Graphomotor Speed contributed equally to Trials 1 and 2 (3.1% and 3.0%, respectively). These findings support the interpretation that the need to separate filled and empty dots may in part explain the increased Visual Scanning demands in the Switch DF.
DISCUSSION
Measures of Design Fluency (DF) represent a common component of neuropsychological batteries, both in clinical and in research settings. Although they are generally assumed to assess executive functions (primarily planning, cognitive flexibility, and fluency in generating visual patterns; Delis et al., Reference Delis, Kaplan and Kramer2001; Ruff, 1998), above and beyond motor speed (Ruff et al., Reference Ruff, Evans and Marshall1986), relatively little empirical research has tested these assumptions.
In a recent construct validation study we examined these assumptions, finding that DF was related to: (a) BDS-EV Motor Planning (M-PLN), (b) TMT-B, presumed to measure cognitive flexibility, and (c) BDS-EV Motor Sequence Fluency test assessing the ability to generate novel motor sequence without generation of visual patterns. Additionally, DF was unrelated to finger-tapping speed. However, questions remained.
First, while motor speed assessed via finger-tapping appeared unrelated to DF performance, Delis and colleagues (Reference Delis, Kaplan and Kramer2001) nevertheless suggested that performance can be affected by motor speed “in drawing” (p. 89); thus, they recommended that D-KEFS Graphomotor Speed task (which assesses motor speed in drawing) be examined as a potential component skill of DF. Second, it was not clear from our prior study whether the association between DF and TMT-B reflected a common variance in cognitive flexibility, graphomotor speed, or visual scanning. And third, although we had found that BDS-EV Motor Sequence Fluency test contributed to DF performance, it was not clear whether this reflected motor fluency in particular, or fluency in general. The present study sought to address these questions.
The key findings of the present study were that generation of unique designs (i.e., the first two trials of D-KEFS DF) relied in part on: (a) D-KEFS Graphomotor Speed (in contrast to motor speed assessed via finger-tapping), (b) BDS-EV Motor Sequence Fluency test (as opposed to fluency in general), and (c) BDS-EV M-PLN. In contrast to previous assumptions, none of the three D-KEFS DF trials were substantially related to cognitive flexibility, as assessed with D-KEFS Number-Letter Switching residuals. D-KEFS Visual Scanning appeared related primarily to increases in the visual-attentional complexity of the task (i.e., addition of distractors), rather than generation of unique designs. Interestingly, Visual Scanning represented the only significant predictor of D-KEFS DF Switch condition.
Component Skills: Graphomotor and Visual Scanning Speeds
As suggested by Delis and colleagues (Reference Delis, Kaplan and Kramer2001), D-KEFS Graphomotor Speed and Visual Scanning tasks both appeared to tap into component skills required for performance of Non-switch DF tests. Because our prior study failed to find a relationship between DF and motor speed (assessed via finger-tapping), it appears likely that the speed of wielding a writing implement is what contributed to DF performance. However, the contribution of D-KEFS Graphomotor Speed to performance was relatively small, with less than 5% of unique variance accounted for. Thus, graphomotor abilities likely have minimal impact on performance, except among individuals with considerable graphomotor deficits.
In addition to Graphomotor Speed, D-KEFS Visual Scanning also contributed to test performance, and this contribution appeared to be a function of increases in the complexity of the visual and attentional demands of different DF conditions. In particular, Visual Scanning accounted for 3.4% of unique variance on Trial 1 (i.e., filled dots only), 5.4% on Trial 2 (i.e., filled dots functioning as distracters), and 13% on Trial 3 (i.e., switching between empty and filled dots). Moreover, as can be seen in Table 12, Visual Scanning represented the best single predictor of the Switch DF condition, accounting for nearly 20% of variance overall (Table 9). These findings suggest that the D-KEFS Visual Scanning contribution taps into the skills needed for separation of relevant and irrelevant visual stimuli: That is, the filled and empty dots on the D-KEFS version of DF (used in this study), and, likely, dots and distractors in the latter trials of the RFFT (Ruff, 1996) version of DF.
Executive Functions: Planning and Flexibility
As was the case in our prior study, BDS-EV M-PLN again emerged as a powerful predictor of the ability to generate novel designs (i.e., the first two conditions of D-KEFS DF), accounting for 12% of unique variance above and beyond component skills (Table 5), and represented the strongest single predictor (Table 12). In contrast, flexibility, at least as assessed by the residual score from D-KEFS Number-Letter Switching (after controlling for D-KEFS Graphomotor, Visual Scanning, and Sequencing Speeds), failed to account for a meaningful amount of variance in the first two D-KEFS DF conditions (i.e., less than 2%, see Table 6). This finding clarified that the association between cognitive flexibility and DF seen in our prior study was simply a function of the association between DF and component skills. Although contributions of flexibility increased somewhat when the switching demands were added to the task, the unique variance accounted for was still quite minimal (about 3%, Table 9). Together, these findings suggest that planning of a motor response (assessed via BDS-EV M-PLN), but not cognitive flexibility, may represent one of the key executive processes assessed by DF.
“Motor planning,” the type of planning assessed by the BDS-EV M-PLN variable, is a covert aspect of complex motor output that can be thought of as the process of preparation for a movement prior to the movement initiation (Keele, Reference Keele and Brooks1981). Thus, it reflects planning at the level of immediate motor action. Although M-PLN latencies have been shown to be related to executive abilities (Suchy & Kraybill, Reference Suchy and Kraybill2007), it is not clear whether they are related to the more purely cognitive planning skills that do not involve immediate motor output.
Fluency: Specific versus General Construct
The present findings replicated our prior results (Kraybill & Suchy, Reference Kraybill and Suchy2008), again demonstrating that the Motor Sequence Fluency test was a unique predictor (Table 7) of the number of generated designs (i.e., the first two conditions of D-KEFS DF), even after demographics and other aspects of motor performance (i.e., M-PLN, Graphomotor Speed) have been accounted for (Table 11). Additionally, together with BDS-EV M-PLN, the BDS-EV Motor Sequence Fluency test represented one of the two strongest predictors of the Non-switch DF overall (Table 12). Incidentally, these same two variables emerged as the two strongest predictors of DF in our prior study as well (Kraybill & Suchy, Reference Kraybill and Suchy2008).
Moreover, the results confirmed that specifically motor fluency (i.e., the ability to generate novel motor actions), not fluency in general, predicted DF performance. This was demonstrated by the lack of association between Letter Fluency and DF. In particular, if the ability to generate novel designs relied on some general fluency ability (a construct that is sometimes invoked as a componential aspect of executive abilities), then this general ability would have to share variance with all three fluency tasks. However, that was not the case.
Switch DF Condition
The results showed that by adding the Switch condition, the cognitive structure of the D-KEFS DF task changed dramatically. As the Switch demand was introduced, performance appeared unrelated to BDS-EV Motor Sequence Fluency and M-PLN, both potent predictors of performance in the Non-switch DF trials. These findings also suggest that switching may represent a construct that is: (a) separate from generative fluency, and (b) perhaps more heavily relying on attentional resources. These results provide support for considering the third (i.e., Switch) trial of D-KEFS DF as a separate construct when interpreting test results.
Limitations
Although our prior study, which yielded very similar results, was conducted with a sample that included the full adult life span (ages 18 to 68 years), it is not clear whether similar results would also be yielded by a sample of young healthy adults only, or by clinical samples, such as dementia patients, epilepsy patients, and so forth. In fact, different factor structures are known to emerge in different populations, particularly when it comes to timed (Psychological Corporation, 1997) and memory tests (Delis, Jacobson, Bondi, Hamiltoon, & Salmon, Reference Delis, Jacobson, Bondi, Hamilton and Salmon2003). For example, timed visual-spatial tests generally load on a different factor than processing-speed tests, except among individuals in their eighties, for whom timed visual-spatial and processing-speed performances load on a single factor (Psychological Corporation, 1997). Similarly, while immediate and delayed recall on memory measures load on a single factor in most populations, they load on separate factors among individuals with mesial temporal dysfunction (Delis et al., Reference Delis, Jacobson, Bondi, Hamilton and Salmon2003). Thus, it is possible that among a sample of patients with focal frontal lesions, cognitive flexibility would become dissociated from motor planning and would uniquely account for substantial variance in DF. These examples highlight the limitation of correlational methodology in construct validation research (Delis et al., Reference Delis, Jacobson, Bondi, Hamilton and Salmon2003), and point to the importance of replications with other, preferably clinical, populations.
Additionally, as is usually the case, the present study yielded models that accounted for approximately between 30% and 40% of variance. This means that more than 60 % of variance remained unexplained. While some of the unexplained variance undoubtedly reflects random error, other systematic sources of variance could likely be identified. In particular, additional specific cognitive processes, such as working memory, attentional vigilance, and visual-constructional skills, may play a role. Similarly, differences in intellectual capacity could likely explain some variance, as Full Scale IQ is known to be differentially related to different cognitive tests (Psychological Corporation, 1997). Lastly, individual differences in temperament and personality, including motivation, autonomic arousal, state and trait anxiety, and interest and curiosity in new experiences all likely contribute differentially to performance (Williams, Suchy, & Rau, Reference Williams, Suchy and Rau2009). Including such a large number of variables would, of course, require a much larger sample.
ACKNOWLEDGMENTS
The authors had no conflict of interest when conducting this research or reporting the results. Funding for this research was provided by the principle author’s University of Utah faculty start-up account.
