INTRODUCTION
Performing skilled object-related actions such as hammering, pouring, sawing, and ironing entails a complex relationship between the visible and known properties of objects, the goals of the actor, motor programs responsive to current metrical information about the environment and body, and stored representations of the skilled action to be performed. One way to understand the nature and properties of the latter representations is to study patients with ideomotor apraxia (hereafter, IMA), in whom the representations are damaged or poorly accessible.
Patients with IMA are markedly deficient in pantomiming to command and imitating pantomimes of object-related actions, despite adequate object recognition and elementary sensory and motor functions (Heilman & Gonzalez-Rothi, 1993). Their attempts at object-related pantomime are frequently characterized by inaccuracy in hand posture, arm posture, amplitude, and timing (Buxbaum et al., 2005b). The disorder is frequently observed with involvement of the left inferior parietal lobe (IPL; Buxbaum, 2001; Buxbaum et al., 2005a; Haaland et al., 2000; Heilman et al., 1982), but has also been reported after damage to left middle frontal gyrus (BAs 6, 8, 9, and 46; Haaland et al., 2000) and left ventral premotor cortex (PMv; Watson et al., 1986). These regions are likely components of a neural circuit specialized for skilled object-related actions (see Buxbaum et al., 2006).
TWO ACTION SYSTEMS
Perhaps one of the most compelling aspects of IMA is its predilection to disproportionately impair skilled object-related pantomime. Reaching to and grasping objects, selection of hand postures based on object structure, and demonstrating actions with tools in hand all tend to be better preserved (Buxbaum et al., 2003, 2005a; DeRenzi et al., 1982; Goldenberg & Hagmann, 1998; Mozaz et al., 2002; Sirigu et al., 1995). In fact, patients with IMA frequently appear to rely on object structural cues to inform hand postures for skilled use even when the structural information guides them toward an inaccurate response. For example, when shown a computer keyboard and asked to select a hand posture consistent with skilled use, IMA patients will frequently select a prehensile posture consistent with grasping based on the global size and shape of the object (Buxbaum et al., 2003).
We (and others, see Rizzolatti & Matelli, 2003) have previously suggested that two action systems guide object-related actions. The dorsal visual processing stream, specialized for object prehension (grasping), relies relatively heavily on moment-to-moment computations of the locations of objects and the body, and is computed by a circuit including the anterior intraparietal sulcus (AIP) and the inferior prefrontal lobe (F5 in monkeys; with a possible homologue of 44 and 45 in humans; Fagg & Arbib, 1998; Gallese et al., 1997; Murata et al., 1996, 1997, 2000). This circuit is active in response to the sight of manipulable objects, to views of others manipulating objects, and when the subject manipulates objects (Buccino et al., 2004; Buxbaum et al., 2005a; Fogassi et al., 1998; Rizzolatti & Fadiga, 1998; Rizzolatti & Luppino, 2001; Rizzolatti et al., 2002). The observed activation is sometimes tied to objects' meaningful function (Chao & Martin, 2000; Frey, 2007), but has been observed even with novel shapes (Creem-Regehr & Lee, 2005).
Unlike the grasp system, which is bilaterally organized and may be preferentially engaged by the presence of objects, the IPL-based functional use system is most often present in the dominant (usually left) hemisphere (Frey, 2007). This “use system” may store abstracted representations of hand–object interactions that may be used in pantomime, motor imagery, and delayed action tasks. Recent neuropsychological and neuroimaging investigations have demonstrated the importance of left IPL structures in abstracting the functional representations of objects. Notably, these studies have consistently shown that the use system, while clearly dissociable from the classic notion of the ventral visual processing stream, nevertheless relies on neural regions that are more ventral than those associated with grasping objects (Buxbaum et al., 2005a, 2006; Creem-Regehr & Lee, 2005; Gerardin et al., 2000; Milner et al., 2001).
Based on the two-action-system model, we have previously suggested that, in IMA, damage to representations for skilled use mediated by the left IPL forces an over-reliance on calculations of hand posture based on current visual information about object structure calculated by the dorsal visual processing stream. Consistent with this suggestion, previous functional magnetic resonance imaging (fMRI) work from our lab indicates that the left IPL is more strongly activated in tasks requiring recognition of skilled use hand posture representations than in tasks requiring response to overall object structure, despite equation of the “use” and “grasp” tasks for behavioral difficulty (Buxbaum et al., 2006).
Role of Structural Object Information in New Learning
For Gibson (1979), objects in the environment offer visuomotor cues to functional interaction based on the needs of the particular individual and the given context. In other words, when an action is highly afforded by an object, the structural properties of the object happen to be suited to the individual's physical attributes and the individual's goals in the moment. Thus, a stone affords graspability and throwability only to someone who can grasp and throw it (Young, 2006).
In light of IMA patients' ability to respond to object structure, as exemplified by clinical studies demonstrating patients' improved performance when allowed to manipulate real objects (Clark et al., 1994; Goldenberg et al., 2004; Hermsdorfer et al., 2006; Wada et al., 1999), we speculated that adding affordance cues might result in better object-related learning of hand posture. We created a stimulus set of novel gestures and novel tools, paired in such a way as to reflect highly afforded and low-afforded action–tool relationships. IMA patients, non-IMA patients, and unimpaired age-matched controls were trained to recognize and produce the novel actions. We predicted that IMA patients (but not non-IMA patients or controls) would have trouble recognizing and producing the actions, but would perform better with highly afforded actions.
We also collected lesion data on all patients. We predicted that IMA patients would evidence greater involvement of the left IPL than non-IMA patients. A secondary prediction was that patients who could benefit from affordance information would have sparing of AIP and Broca (44 and 45) regions previously shown to be involved in response to object structure.
METHOD
Subjects
A total of 15 patients (11 men, 4 women; age, M = 60.3 years; range, 46–81 years) were tested. All patients suffered left hemisphere cerebrovascular accident and were at least 6 months postonset at the time of participation. We obtained names from the Patient Research Registry at Moss Rehabilitation Research Institute, with the following recruitment criteria: (1) between 21 and 80 years of age; (2) English as a native or primary language; and (3) evidence of linguistic and cognitive capacity to give informed consent. Subjects were included without respect to gender, race, or ethnic background.
Eleven patients were identified as apraxic (IMA) by averaging their scores on 10 trials of pantomime to sight of object and 10 trials of meaningless gesture imitation, and comparing these individual scores against the mean of a previously tested group of unimpaired controls (see Buxbaum et al., 2005b, for details). [There is disagreement about the most appropriate test for IMA. In keeping with previous work in our laboratory (e.g., Buxbaum et al., 2005b), we used a composite score of novel gesture imitation and gesture to sight of objects to capture patients who would be defined as having IMA by either measure. Table 1 shows that the two measures are reasonably correlated (r = .49). Both tests have a maximum score of 100%.] All patients in the IMA group scored 2 SD or more below the mean of the non-IMA group. The IMA and non-IMA subjects did not differ in age (p = .25), education (p = .76), or total lesion size (p = .23). Scores on behavioral measures for both the IMA and non-IMA patients, along with other demographic information, are shown in Table 1.
Behavioral measures and demographic information on IMA and non-IMA patients
In addition to the patients, we tested 10 unimpaired control subjects (hereafter, CTRL), all of whom were right-handed and native English speakers without history of neurological injury. Two CTRLS were men, their average age was 58.3 years (range, 37–74 years), and they did not differ from the patients in age (p = .62) or education (p = .12). CTRLs were recruited from the Control Subject Research Registry of Moss Rehabilitation Research Institute. All participants gave informed consent according to the guidelines of the Albert Einstein Medical Center Institutional Review Board and were paid for their participation.
Materials
The materials used in this study were normed across three separate groups of unimpaired college-age subjects (see details in Appendix A). Using the normative data, we created action–picture pairs wherein the actions were of equal complexity, the pictures of novel tools were of equal discriminability, and the relationship between the actions and associated pictures were either highly or low-afforded. Examples of each condition (picture and a screen-capture of the gesture) are given in Figure 1.
Examples of High- and Low-Afforded gesture and object pairs.
Procedure
Subjects were introduced to eight objects from a “foreign planet,” and their task was to learn how to properly use these objects. The experiment was composed of two 45-minute sessions, held on separate days with at least 2 days between sessions. During each session, the subject learned four of the eight action–picture pairings (e.g., in Session 1, Subject 1 learned pairings 1–4, and in Session 2 that same subject learned pairings 5–8). Order of learning was counterbalanced across subjects with the constraint that in each session, subjects learned two gesture–picture pairings that were High Affordance, and two others that were Low Affordance.
Within each session, there were 10 blocks of eight trials. All blocks began with an Exposure Phase (four trials), followed by a 30-second distractor task (to minimize the possibility of rehearsal), followed by a Test Phase (four trials). Recognition tests were alternated with Production tests across blocks (e.g., Block 1 contained an Exposure Phase followed by Recognition tests, and Block 2 contained an Exposure Phase followed by Production tests), and order of testing was counterbalanced across subjects. At the end of the second session, a subject had completed five Recognition tests and five Production tests for each of the gesture–object pairings.
The Exposure Phase trained the subjects to match the actions and objects. On each trial in the Exposure Phase, one object picture from the learning set appeared at the top of a computer screen, followed by a video clip of a woman performing the matching action in the bottom of the screen. The action was shown twice, and the picture remained on the screen during the clip. The subject imitated the action at any point during the trial. Timing between trials was individually paced, and responses were not timed. Although we did not give the subjects corrective feedback during the Exposure Phase, the subjects were reminded to use the time to “watch the video closely and learn to imitate the action as accurately as possible, down to the last detail.”
In the Recognition Test Phase, subjects performed a multiple-choice task. On each trial, the subject watched a video clip at the bottom of the screen, while at the top of the screen there appeared four object pictures. One picture was the target, one picture was a foil from within the learning set, and the remaining two pictures were foils from outside the learning set. The subject was to point to the picture that “went with” the given action. Subjects were not informed of the accuracy of their performance at any time during the experiment.
On each trial in the Production Test Phase, the subject was shown a single object picture and asked to perform the action that “went with” the picture. All movements were made with the ipsilesional (left) hand. No corrections were made to the subjects' production of the gesture at any time during the experiment. Subjects were videotaped so that actions could be scored after the session.
Scoring Protocols
During the test session, subjects' pointing responses in the Recognition Test were entered on prepared data sheets. Performance in the Production Test was scored from the videotapes by one of two independent coders. (An assessment of inter-rater reliability was determined by having both coders score four randomly selected subjects: mean agreement between the coders on this subset of the data was 94.7%.) Coders compared each subjects' action to the model videoclip and, consistent with the usual scoring protocol in our laboratory, to rate five components of the movement: Content, Hand Posture, Arm Posture, Amplitude, and Timing. As will be discussed below, this study assessed predictions relevant only to the Hand Posture component. Additional details are given in Appendix B.
Failures of Encoding, Acquisition Rate, and Consolidation
Using procedures similar to those of Gonzalez-Rothi and Heilman (1984), we calculated encoding failures, consolidation failures, and acquisition rates for all subjects on the Recognition data. Briefly, encoding failures were the number of actions (out of eight) that the subject never recognized. Acquisition rate was determined by taking the total number of actions learned in the experiment (zero to eight possible) and dividing by the number of exposures (i.e., trainings) required for the subject to first recognize the actions. Consolidation failures were determined to be the number of instances the subject failed to correctly recognize an action following at least one successful recognition trial, divided by the total number of correctly recognized trials. To avoid overpenalizing the patients, if any subject (independent of group membership) missed more than one trial following a correct recognition (e.g., a correct trial followed by two incorrect trials followed by a correct trial), it was counted as a single consolidation failure.
Lesion Analysis
We obtained high-resolution structural MRI or research-quality computed tomography (CT) scans on 12 of the 15 patients in the study (9 of the 11 IMA; 3 of the 4 non-IMA).
MRI examination parameters
The parameters were as follows: (a) T1-weighted (500/20, one signal acquired) sagittal localizing sequence with 5-mm slice thickness, 1.5-mm gap, 24-cm field of view; (b) T2-weighted (3,000/100, one-half signal acquired) sequence with 5-mm slice thickness, no gap, 20-cm field of view; (c) fast fluid-attenuated inversion recovery, or FLAIR (6,706/120, 2,000-ms inversion time) sequence with 5-mm slice thickness, no gap, 20-cm field of view; and (d) T1-weighted spoiled gradient-recalled-echo (MPRAGE; 35/7, 45 flip angle, one signal acquired) transverse images with 1-mm slice thickness, no gap, 1 mm in-plane resolution. A 192 × 256 matrix was used throughout acquisition.
CT parameters
Forty-four contiguous axial slices covering the entire brain (2.55-mm slice thickness) without contrast were used.
Segmentation and Coregistration
Images were obtained on a 1.5-T scanner at a single site (Hospital of the University of Pennsylvania). Of the remaining patients, one IMA patient had no scan, one IMA patient had a small subcortical lesion in conflict with perfusion data showing a large region of left hemisphere hypoperfusion (and was, therefore, excluded from lesion analyses), and one non-IMA patient had only an acute clinical scan available.
Under the supervision of a behavioral neurologist, two trained observers naive to experimental conditions manually drew lesion boundaries onto a standard MNI template using MRIcro software (www.psychology.nottingham.ac.uk/staff/crl/mricro.html). Team members established reliability in lesion segmentation using the method devised by Fiez et al. (2000): percent volume difference (7.99%), percent discrepant voxels (4.18%), and percent nonoverlapping voxels (31.5%) were comparable to or better than Fiez and colleagues.
RESULTS
For all analyses, noncorrected α was .05. We used the error term from the omnibus analysis of variance (ANOVA) when conducting post hoc tests for simple main effects. This procedure maximizes power, and thus the ability to detect significant differences is more favorable in comparison to using simple t tests (which by definition will have lower degrees of freedom). Bonferroni corrections were made where appropriate: we will remind the reader of the relevant α criterion as necessary. Also, effect sizes in the form of partial eta-squared (ηp2) are presented where appropriate.
Recall that each item was trained and tested 10 times (5 Recognition Tests, 5 Production Tests). Learning was operationalized as the factor Time. Performance on the first two Recognition tests (or first two Production tests) was averaged together and labeled “Early” (in the session), and the performance on the last two test points (again separately for kind of test) was averaged together and labeled “Late” (in the session). Data from the middle test point were discarded. Separate repeated measures ANOVAs were performed on the subject means (mean Recognition score or Production score), with Affordance (high, low) and Time (early, late) as within-subjects variables, and Group (IMA, non-IMA, and CTRL) as a between-subjects variable.
Recognition Learning
Overall learning effects
The ANOVA yielded a main effect of Time [F(1,22) = 27.29; p < .0001; ηp2 = .55], qualified by a Time × Group interaction [F(2,22) = 5.03; p = .016; ηp2 = .31]. In post hoc testing, with corrected α of p = .016 (three tests performed), within-groups analysis revealed that CTRLs performed well and equally at both time points (Early: M = .944; SD = .048; Late: M = .975; SD = .036) [F(1,22) = .697; p = .413; ηp2 = .03]. Critically, both IMA patients [F(1,22) = 24.36; p < .0001; ηp2 = .53] and non-IMA patients [F(1,22) = 11.78; p = .002; ηp2 = .35] demonstrated significant improvements in recognition accuracy from early to late in the session (IMA Early: M = .670; SD = .046; IMA Late: M = .847; SD = .076; non-IMA Early: M = .781; SD = .034; non-IMA Late: M = .984; SD = .056). In between-groups comparisons (corrected for two tests with critical α of p = .025), there were significant effects of Group at the early time point [F(2,22) = 8.50; p = .002; ηp2 = .44], and a trend at the late time point [F(2,22) = 4.16; p = .028; ηp2 = .27]. Pairwise comparisons (corrected α = .016) at the early time point revealed that IMA patients performed similarly to non-IMA patients (p = .225), but different from CTRLs (p < .0001), and non-IMA patients did not differ from CTRLs (p = .084). Additional pairwise comparisons conducted at the late time point showed that IMA patients still differed from CTRLs, although less so, and that non-IMA patients did not differ from CTRLs: IMA versus non-IMA (p = .049), IMA versus CTRLs (p = .016), non-IMA versus CTRLs (p = .890).
Recognition learning parameters
We performed three univariate ANOVAs on the Recognition data with acquisition rate, consolidation failures, and encoding failures as the dependent variables. There were significant differences between groups in action acquisition (IMA: M = .659; SD = .04; non-IMA: M = .765; SD = .07; CTRLs: M = .944; SD = .04) [F(2,22) = 12.73; p < .0001; ηp2 = .54]. On average, IMA patients learned to recognize only 66% of the actions after a single Exposure Phase, whereas the CTRLs learned nearly 100% of the actions after only one Exposure Phase. Pairwise comparisons (corrected α of p = .016) indicated that IMA patients and CTRLs differed in acquisition rate (p < .0001); the other comparisons were marginally nonsignificant (IMA vs. non-IMA patients; p = .18; non-IMA patients vs. CTRLs, p = .03). There were also significant differences in rate of consolidation failure between groups (IMA: M = .154; SD = .03; non-IMA: M = .038; SD = .05; CTRL: M = .031; SD = .03) [F(2,22) = 4.56; p = .02; ηp2 = .29]. Pairwise comparisons again indicated significant differences between IMA patients and CTRLs (p = .010); the other comparisons were nonsignificant (IMA vs. non-IMA patients; p = .059; non-IMA patients vs. CTRLs; p = .898). (Moreover, none of the three learning indices varied as a function of affordance.) Finally, the number of encoding failures failed to distinguish between IMA patients, non-IMA patients and controls. One apraxic subject committed the single encoding failure observed across the entire experiment.
Effect of affordance in recognition
Critical to our predictions, the ANOVA on the Recognition data revealed an Affordance × Group interaction [F(2,22) = 9.42; p = .001; ηp2 = .46]. The main effect of Affordance did not approach significance [F(1,22) = 0.002; p = .966; ηp2 = .00]. Within-groups post hoc testing (with corrected α of p = .016) revealed that only IMA patients demonstrated better recognition of High (M = .795; SD = .036) than Low (M = .722; SD = .039) Afforded actions, and this difference was significantly reliable [F(1,22) = 12.40; p = .002; ηp2 = .36]. Neither non-IMA patients [F(1,22) = 0.21; p = .66; ηp2 = .01] nor CTRLs [F(1,22) = 6.53; p = .02; ηp2 = .23] demonstrated significant effects of affordance, with the caveat that the CTRLs tended to perform slightly better on Low-Afforded actions (see Figure 2).
Within-groups comparison of action recognition data. Ideomotor apraxia IMA patients alone demonstrated improved performance on High-Afforded trials. Error bars denote 1 SE of the mean of the within-groups comparison.
Moreover, between-group comparisons (with corrected α of p = .025) failed to reveal significant differences in recognition of High-Afforded actions [F(2,22) = 3.85; p = .052; ηp2 = .24]. In contrast, there was a highly significant difference between groups' recognition of Low-Afforded actions [F(2,22) = 11.15; p < .0001; ηp2 = .50]. Pairwise comparisons (with corrected α of p = .016) revealed that IMA patients' recognition of Low-Afforded actions differed from CTRLs (p < .0001), but not non-IMA patients' (p = .22). Non-IMA and CTRL performance did not differ (p = .44).
Affordance benefit
We quantified this apparent “benefit” of affordance in recognition learning for each subject by taking the difference in mean recognition performance between High- and Low-Affordance trials and dividing that number by the mean performance across all trials (Affordance Benefit Ratio, or ABR). [We used the denominator to correct for baseline performance. However, a reviewer requested that we reanalyze the data using a raw difference score as the measure of “affordance benefit.” We did, and found similar results: Affordance Benefit (AB) is still negatively correlated with Apraxia score (r = −.58, p = .02). Group membership in the AB+ and AB− groups did not change.] Of the 15 patients, 7 demonstrated an affordance benefit (hereafter, “AB+”: M = .147; SD = .11; range, .059–.267), and 8 did not (hereafter, “AB−”: M = −.021, SD = .04; range, −.054–.00). Importantly, none of the non-IMA patients or unimpaired control subjects demonstrated an Affordance Benefit. All of the patients in the AB+ group were apraxic: the ABR negatively correlated with Apraxia score (r = −.61; p = .01), indicating that the more severe apraxics showed greater benefit of affordance. Outside the scope of this paper, all patients were tested on an extensive battery intended to characterize various praxis abilities (see Buxbaum et al., 2003, for details). AB+ subjects exhibited significantly greater deficits than AB− subjects in the ability to imitate familiar object-related [t(12) = −3.10, p = .009] but not symbolic gestures [t(12) = −.369, p = .72].
Production Learning
Overall learning effects
Following Buxbaum et al. (2003), we focused on the Hand Posture component of gesture production. There was a main effect of Group [F(1,22) = 7.91; p = .003; ηp2 = .42], such that IMA patients' hand postures were significantly worse than both non-IMA patients' (p = .008) and CTRLs (p = .002). There was also a main effect of Time [F(1,22) = 21.67; p < .0001; ηp2 = .50], indicating that all groups improved in their ability to produce novel actions.
Effect of affordance on production
Analysis of the effect of affordance on learning to produce object-related hand postures was inconsistent with our prediction. There was no main effect of Affordance [F(1,22) = 1.13; p = .30; ηp2 = .05], nor was there an Affordance × Group interaction [F(1,22) = 0.47; p = .63; ηp2 = .04]. We present possible reasons for these null effects in the Discussion section.
Lesion Analysis
Comparison of the IMA/non-IMA groups
Subtraction of the lesions of the non-IMA group (n = 4) from the lesions of the IMA group (n = 9) was performed to identify the loci of damage more common in the latter group. Results were consistent with numerous previous studies. The classification of IMA was relatively likely to be associated with damage to left IPL, inferior and middle frontal and prefrontal gyri, as well as the superior temporal gyrus (STG). Eight of the nine IMA patients had lesions that included the IPL (Figure 3).
Map of differences in the proportions of patients in the ideomotor apraxia (IMA) group (n = 9) versus the non-IMA group (n = 4; Left) having lesions in different brain regions generated by MRIcro (Rorden & Brett, 2000). Map of the proportion differences in IMA group versus non-IMA subjects for whom postacute, research-quality magnetic resonance imaging or computed tomography scans were available (n = 3; Right). The colors denote the proportion difference in the number of subjects in each group having lesions to a given region. For example, the orange color labeled “66.6–83.2%” indicates that this region was involved in between 66.6% and 83.2% more patients in the IMA group compared with the non-IMA group. Classification of IMA was associated with damage to left inferior parietal lobe, inferior and middle frontal and prefrontal gyri, as well as the superior temporal gyrus.
Comparison of the AB+/AB− groups
Subtracting the lesions of the AB− group (n = 7) from the lesions of the AB+ group (n = 6) to identify loci of damage more common in the AB+ group revealed that AB+ patients tended to have greater involvement in a region spanning the PMv and mid-STG (Figure 4). Comparison of the critical lesion patterns associated with IMA and AB+ in Figure 5 reveals, qualitatively, that the affordance benefit (AB+) is associated with a relatively ventral pattern of involvement, with relative sparing of the parietal lobe (both IPL and SPL).
Map of differences in the proportions of patients in the Affordance Benefit-positive (AB+) group (n = 6) versus the AB− group (n = 7) having lesions in various brain regions (Left). Map of proportion differences in the AB+ group (n = 6) versus the AB− subjects for whom postacute, research-quality magnetic resonance imaging or computed tomography scans were available (n = 6; Right). For additional interpretation of color key, see Figure 3. AB+ patients tended to have greater involvement in a region spanning the ventral premotor cortex and mid-superior temporal gyrus.
Overlay of regions displayed in Figures 3 (dark blue) and 4 (green) to facilitate comparison. Regions common to both patterns appear in light blue. Note that the lesion loci associated with the Affordance Benefit-positive (AB+) pattern of performance (green) is more ventral than that associated with the ideomotor apraxia pattern of performance (blue).
DISCUSSION
Based on prior evidence that IMA patients show preserved ability to respond to object structure, this study investigated whether affordance cues could benefit their learning of novel object-related actions. We trained IMA, non-IMA, and CTRLs to recognize and produce eight novel object-related actions, half of which were highly afforded by their paired objects.
Our main prediction was that the IMA patients, but not the other two groups, would learn to both recognize and produce highly afforded actions better than low-afforded actions. The critical finding in the study is that only the IMA patients demonstrated improved action recognition as a function of affordance. Moreover, IMA patients' recognition of low-afforded actions was significantly worse than non-IMA patients and controls, whereas their recognition of highly afforded actions was statistically equivalent to that of the other two groups.
We also demonstrated that, with repeated exposures, IMA patients improved initially poor recognition of novel object-related actions. Whereas CTRL subjects recognized novel actions well and equally early and late in the session, IMA patients were significantly impaired relative to CTRLs early in the session, but differed from CTLs only at a trend level by the end of the session. That IMA patients learned to recognize novel object-related actions is noteworthy, especially in light of the brevity of training and absence of specific feedback during practice. In this context, IMA patients' learning to recognize object-action pairs had several abnormal characteristics. IMA patients had significantly slower acquisition rates and a greater number of consolidation failures than the other groups. On average, IMA patients acquired only two thirds of the actions after a single exposure round compared with nearly perfect performance by the CTRLs. Also, the IMA patients were five times more likely to forget a new action after learning it successfully. These results are striking in that they highlight the extreme difficulty IMA patients have when learning new actions: even when the memories are acquired, they remain highly unstable.
In terms of production learning, IMA patients' object-related hand posture production showed improvement with repeated exposure. Nevertheless, they were impaired relative to the other groups at both early and late time points.
Affordance and Production of Novel Gestures
The Production data were less conclusive than the Recognition data with regard to an AB. We found no significant improvement in hand posture production quality with highly afforded actions. One possible reason for this finding is that we used two-dimensional (2-D) rather than three-dimensional (3-D) objects during training. To the extent that translation of affordance cues into motor codes for production of skilled action relies on interaction with a three-dimensional object, our design may have failed to provide a sufficient training milieu. Consistent with this possibility are recent data from Symes et al. (2007), showing that affordance effects are larger and more robust when objects appear graspable and are oriented in depth so that a leading edge appears to point toward the hand of the viewer. On this account, however, it is not clear why we obtained an AB with 2-D objects in action recognition, unless 3-D objects are necessary to benefit action production but not action recognition. As action recognition is not typically included in experiments designed to assess AB, this remains a possibility.
An alternative (but not mutually exclusive) explanation might lie in the differences between the recognition and production tests we used. Recall that, in the Recognition Test, the stimulus was an action and subjects picked the corresponding object from an array. It is possible, then, that affordance cues can serve to strengthen the ease with which an observed action can be linked to an object. This possibility is consistent with recent studies that have shown improved processing of visual stimuli consistent with intended or observed actions (Bekkering & Neggers, 2002; Craighero et al., 1999; Hannus et al., 2005), a phenomenon that has been termed “motor attention.”
In contrast, in the Production Test, subjects were given an object in isolation and were required to produce the appropriate action. In healthy subjects, there is evidence that passive observation of tools evokes activity in cortical motor areas and facilitates motor responses consistent with object affordances (Ellis & Tucker, 2000; Grafton et al., 1997; Tucker & Ellis, 1998; Mecklinger et al., 2004). It appears, however, that IMA patients' motor responses did not benefit from affordance information present in objects.
There are several possible reasons for this finding. One is that IMA patients frequently suffer production impairments that affect all complex motor tasks (see Buxbaum et al., 2003) and that may be additional to their impairments in object-related gesture representations. It is also possible that IMA patients have damaged or poorly accessible stored action representations and that “noisy” activation interferes with the ability to benefit from affordance cues, given an object in isolation with the goal to produce a learned gesture. This explanation is consistent with previous data from our lab showing that, with familiar objects, IMA patients continue to produce inappropriate hand postures even when the “correct” hand posture could have been gleaned from object structure alone (Buxbaum et al., 2003).
Neural Basis of the Affordance Benefit
In terms of the neuroanatomic basis for this effect, we demonstrated that the critical locus of the lesions in patients showing a benefit of affordance tended to differ from the lesions producing IMA. The lesions associated with the “affordance benefit” in patients' action recognition were predominantly in the more ventral portions of the action system. Consistent with our predictions, these lesions completely spared AIP and Broca's 44 and 45, and instead largely involved superior portions of the temporal lobe, with encroachment on ventral portions of PMv. In contrast, the lesions associated with IMA, consistent with previous reports, included the IPL and inferior frontal gyrus (44 and 45). As we (and others, see Buxbaum, 2001; Buxbaum et al., 2003; Garbarini & Adenzato, 2004; Norman, 2002; Schindler et al., 2004) have suggested, the ability to capitalize on structural features of objects is offered by relatively intact frontoparietal structures of the dorsal stream (AIP and 44/45). In the face of damage (or noisy access) to stored representations for skilled object use and recognition, IMA patients may bootstrap gesture recognition by way of reliance on these dorsal stream processes. The interactive nature of the two action systems may allow for this process to occur.
CONCLUSIONS
In summary, we have identified a potentially valuable mechanism for improving new action learning in at least some apraxic patients by way of object affordances. A limitation of this work is that we found the affordance benefit only for action recognition and not production. Future investigations focused upon 3-D objects are required to assess whether more realistic stimuli may have more potent effects on production. Alternatively, it may be possible that the stimuli are not as critical as the directionality of the “bootstrapping” process; that is, whether in IMA patients observed actions may drive access to affordance information present in objects, but not vice versa.
ACKNOWLEDGMENTS
This work was supported by NIH Grant R01-NS036387 awarded to Laurel J. Buxbaum. We thank Katie Kyle and Susan Lipsett for assistance with data collection. Some of these data were presented at the 2006 Cognitive Neuroscience Society Meeting.
APPENDIX A: NORMATIVE DATA
All subjects (n = 42) who provided normative data were students from the University of Pennsylvania without history of brain injury. Subjects gave informed consent in accordance with the guidelines of the Albert Einstein Medical Center and the University of Pennsylvania Institutional Review Boards and were paid for their participation.
Gesture Complexity
Twelve subjects provided measures of the perceived complexity of gesture stimuli in isolation. Thirty-two novel gestures were created by systematically varying the following parameters: Hand Posture (i.e., Pinch, Poke, Palm, and Clench), Arm Posture (i.e., mostly straight or mostly bent), Dynamics (moving versus static arm), Plane (parallel vs. perpendicular to body), Amplitude/Timing (large/slow movement vs. small/fast movement), and whether there was Contact with the Body (yes vs. no).
Subjects pressed a button to view a brief (7–9 s) videoclip of a woman performing a gesture and were required to imitate the gesture immediately thereafter. They then rated the complexity of the gestures on a 5-point scale. The ratings data were averaged for each gesture and from these means two stimulus groups were created, each containing 16 gestures of comparable complexity (p = .86).
Picture Discriminability
Ten subjects established the discriminability of the picture stimuli. We selected a set of 16 pictures (black and white line drawings) of novel tools from a larger stimulus set. Subjects viewed two pictures in succession and decided if the two stimuli were the SAME or DIFFERENT.
For DIFFERENT trials only, we examined the mean discrimination latency for each stimulus pair, and used this measure to divide the pictures into two stimulus groups, each with eight pictures of comparable discriminability.
Affordance
Last, 10 subjects provided ratings of affordance, defined as the extent to which the physical characteristics of a given object provides information about an associated gesture. Each picture stimulus was paired with two different gesture clips over the course of the pilot study: one pairing selected a prioi as likely to reflect a HIGH AFFORDANCE and one likely to be LOW. Subjects rated the extent to which the gesture was appropriate for using the tool shown, using a 5-point scale. These ratings were used to create the final stimulus set: four gesture–picture pairings were rated Highly Afforded (Mean = 3.7), and the other four pairs were rated Low-Afforded (Mean = 1.5). For each affordance category, there was no difference in the picture discriminability. Finally, each affordance category had one Clench gesture, one Pinch gesture, one Poke gesture, and one Palm gesture, and there was no difference in the complexity of gestures across affordance category.
APPENDIX B: PROTOCOL FOR SCORING GESTURE PRODUCTION DATA
A production was given one point for content if the production was recognizable as the correct gesture. Across all subjects and conditions, content errors occurred 26% of the time. When they did occur, it was noted if the gesture produced matched an “in set” object, an “out of set” object, or neither.
Within the Hand Posture component, one point was given if the subject produced the correct initial hand posture, one point was given if the final hand posture was correct, and one additional point was given if the subject's hand movements reflected the gross dynamics of the target posture (i.e., moving vs. static).
Within the Arm Posture component, one point was given if the plane of movement was correct (i.e., the joint angle of the arm was correct and/or the motion of the arm was in the correct plane relative to the body), one point was given if the placement of the arm was correct (e.g., the arm was placed such that the hand was appropriately by the ear instead of in front of the face), and one point was given if the trajectory of the arm was correct (e.g., arm moved down and up instead of vice versa).
To be given one point for Amplitude, the gesture needed to be of reasonably correct size compared with the model. Finally, one point was given for correct Timing of the movement (i.e., was the tempo of the movement correct and the subject performed the correct number of repetitions).
