Two major goals of SLA research are to define and describe learners’ second-language (L2) linguistic knowledge and to explain how this knowledge develops over time (R. Ellis, Reference Ellis2004, Reference Ellis2005). Many theoretical issues, including the debate over how linguistic knowledge changes as proficiency increases (the interface and noninterface position), hinge on the ability to measure learners’ explicit and implicit language knowledge. However, there has been little research to date on how learners’ explicit and implicit interlanguage knowledge can be measured (Ellis et al., Reference Ellis, Loewen, Edler, Erlam, Philp and Reinders2009). One difficulty in measuring L2 learners’ implicit and explicit language knowledge is that their interlanguage contains both types of knowledge, making measurement tricky. This study explores the validity of the tests designed by Ellis (Reference Ellis2005) to provide separate measures of implicit and explicit knowledge, with a population of heritage language (HL) learners of Spanish (Valdés, Reference Valdés, Peyton, Ranard and McGinnis2001) who have very little explicit knowledge by virtue of the fact that they learned their HL naturalistically and have had little formal instruction.
In addition to validating the measures of explicit and implicit knowledge, having a sample of L2 learners and a sample of HL learners complete the same tests of linguistic knowledge can help to provide a more complete picture of the ways in which HL learners may be similar to, or different from, L2 learners. This is not a trivial question, because it has implications both for linguistic theory and for language pedagogy.
POSITIONS ON THE RELATIONSHIP BETWEEN IMPLICIT AND EXPLICIT LANGUAGE KNOWLEDGE
In SLA, there are three main theoretical positions taken with regard to the relationship between implicit and explicit language knowledge. According to the noninterface position, implicit and explicit L2 knowledge are acquired differently (Hulstijn, Reference Hulstijn2002; Krashen, Reference Krashen1981), are thought to be located in or accessed by different areas of the brain (Paradis, Reference Paradis and Ellis1994), and are accessed by different processes (N. C. Ellis, Reference Ellis1993). The hallmark of the noninterface position is that the two knowledge types are completely separate to such an extent that explicit knowledge can never become implicit. This is the position espoused by Krashen (Reference Krashen1981) and his learning-acquisition hypothesis. According to the noninterface position, explicit language knowledge will always remain explicit, despite years of exposure, practice, and proficiency in the language.
The strong interface position is in polar opposition to the noninterface position: This position holds that explicit knowledge can be derived from implicit knowledge and, even more importantly, that explicit knowledge can become implicit knowledge with enough practice. According to the strong interface position, learners can first learn a rule as declarative (explicit) knowledge and then, through repeated use and practice, can convert it into an implicit representation. The interface position was first formally advanced by Sharwood Smith (Reference Sharwood Smith1981) and has subsequently been promoted by DeKeyser (Reference DeKeyser, Doughty and Williams1998).
The weak interface position similarly posits that explicit knowledge can become implicit; however, it is not as categorical as the strong interface position in its formulation. This position stipulates some constraints on the circumstances under which explicit knowledge can become implicit. One version of the weak interface position, espoused by R. Ellis (Reference Ellis1993), proposes that the conversion of explicit to implicit knowledge can occur through practice, but only if and when the learner is developmentally ready to acquire the linguistic form in question. A second version, espoused by N. C. Ellis (Reference Ellis1994), maintains that explicit knowledge contributes indirectly to the creation of implicit knowledge because “declarative rules can have ‘top-down’ influences on perception” (p. 16). The third and final version of the weak interface position, espoused by Schmidt and Frota (Reference Schmidt, Frota and Day1986) and Sharwood Smith (Reference Sharwood Smith1981), proposes that learners use their explicit linguistic knowledge to produce (presumably planned) output, which, in turn, becomes input for their implicit system.
These positions have been debated in the SLA literature for decades. However, to date, there is no conclusive evidence that favors one position over another. One main reason for this is that although studies have sought to examine the relationship between explicit and implicit linguistic knowledge for many years (Hulstijn & Hulstijn, Reference Hulstijn and Hulstijn1984; Seliger, Reference Seliger1979; Sorace, Reference Sorace1985), most have not been concerned with establishing a valid and reliable set of measurement instruments to do so (R. Ellis, Reference Ellis2004, Reference Ellis2005). It was for this reason that Ellis set out to examine the constructs of implicit and explicit linguistic knowledge and to develop a rigorous set of instruments designed to measure them.
FEATURES PROPOSED TO DISTINGUISH BETWEEN TYPES OF LINGUISTIC KNOWLEDGE
In his recent research, R. Ellis (Reference Ellis2004, Reference Ellis2005) considered features that could be used to distinguish between the constructs of implicit and explicit linguistic knowledge. His review of theoretical and empirical work in SLA revealed seven distinguishing features: Awareness, type of knowledge, systematicity, accessibility, use of knowledge, self-report, and learnability.
Awareness
Children learning their first language (L1) initially demonstrate intuitive knowledge about what is possible and impossible in their language. Only once they reach a certain age and developmental level (at or beyond age 5, or school age) do children start to demonstrate some ability to analyze their language, or to develop metalinguistic knowledge and conscious awareness of why structures are grammatical or ungrammatical (Karmiloff-Smith, Reference Karmiloff-Smith1979). In a recent study with English-speaking L2 learners of Spanish and German, the amount of language-learning experience in formal (classroom) settings was found to be the strongest predictor of metalinguistic knowledge (Roehr & Ganem-Gutierrez, Reference Roehr and Ganem-Gutierrez2009).
Type of Knowledge
In psychology, there is a long-standing distinction between declarative and procedural knowledge (Anderson, Reference Anderson1983). Declarative knowledge is factual in nature and accessed slowly, whereas procedural knowledge is highly automated and accessed quickly. Declarative and explicit knowledge are parallel, as are procedural and implicit knowledge.
Systematicity and Certainty of L2 Knowledge
In cognitive psychology, Reber, Walkenfield, and Hernstadt (Reference Reber, Walkenfield and Hernstadt1991) demonstrated evidence to support the idea that implicit knowledge is less variable, or more consistent, than explicit knowledge. Along similar lines, SLA researchers have claimed that learners’ interlanguages (their implicit knowledge) are highly systematic (e.g., Tarone, Reference Tarone1988). Explicit knowledge, in contrast, tends to be more anomalous and much less consistent (Sorace, Reference Sorace1985). Taken together, these pieces of evidence suggest that implicit knowledge is more structured than explicit knowledge and that learners are more confident in making judgments based on implicit, rather than explicit, knowledge.
Type of Processing and Accessibility of Knowledge
Implicit knowledge involves automatic processing, whereas explicit knowledge entails controlled processing. However, the type of processing cannot be directly measured and is instead generally inferred from the speed with which the knowledge is accessed. Some SLA researchers (e.g., Hulstijn, Reference Hulstijn2002) have indicated that practice will only “speed up the execution of algorithmic rules to some extent” (p. 211) and that there remains a fundamental difference between automated explicit knowledge and implicit knowledge in terms of their accessibility. Others (e.g., DeKeyser, Reference DeKeyser, Doughty and Long2003) have indicated that explicit knowledge can be fully automatized and can thereby become functionally equivalent to implicit knowledge.
Use of L2 Knowledge
Bialystok (Reference Bialystok1982) provided evidence that the use of explicit or implicit knowledge varies according to the demands of the tasks learners are asked to perform. Bialystok (Reference Bialystok1982) distinguished task demands in terms of binary dimensions of analysis and control. For example, she provided evidence to show that a written task that requires learners to detect and then correct errors is [+ analyzed, – automatic] and taps into explicit knowledge, whereas an aural task requiring participants to follow the same instructions is [+ analyzed, + automatic], tapping implicit knowledge.
Self-report
Implicit knowledge cannot be verbalized, whereas explicit knowledge is potentially verbalizable. Although learners are likely unable to verbalize the entire contents of their explicit knowledge, they are able to verbalize at least a subset of it.
Learnability
Some researchers have proposed that there are maturational effects on implicit and explicit knowledge (DeKeyser, Reference DeKeyser, Doughty and Long2002). More specifically, Bialystok (Reference Bialystok and Ellis1994) claimed that “explicit knowledge can be learned at any age” (p. 566), but there are age-related constraints on L2 learners’ ability to acquire implicit knowledge.
In summary, implicit knowledge can be described as intuitive, procedural, automatic, variable in a limited and systematic way, and available in fluent, spontaneous language use. Implicit knowledge cannot be verbalized, and there is debate over the extent to which implicit knowledge can be learned in adulthood. Explicit knowledge, to the contrary, is conscious, declarative, highly variable, and only accessible through controlled processing. Therefore, explicit knowledge is largely unavailable in spontaneous language use, although learners can rely on it for monitoring in planned language use. Explicit knowledge can be verbalized, therefore entailing at least some degree of metalinguistic knowledge. Like any other type of declarative knowledge, explicit language knowledge can be learned regardless of age. When linguists and L1 acquisition researchers refer to linguistic knowledge, they typically refer to linguistic competence (i.e., implicit knowledge). However, in many empirical SLA studies, learners’ knowledge of targeted structures in a language is often measured through tests that favor the use of explicit knowledge.
REVIEW OF R. ELLIS (Reference Ellis2005)
R. Ellis (Reference Ellis2005) created a battery of five English language tests designed to tap explicit and implicit knowledge by manipulating awareness, type of knowledge, self-report, learnability, systematicity and certainty of L2 knowlege, type of processing and accessibility of knowledge, and use of L2 knowledge. The tests included (a) an oral imitation test that contained both grammatical and ungrammatical sentences, (b) an oral narration test, (c) a timed grammaticality judgment test (GJT), (d) an untimed GJT with the same grammatical structures, and (e) a metalinguistic knowledge test. L2 learners of English at a range of proficiency levels (n = 91) and a group of English native speakers (n = 20) took the battery of tests, and their responses were analyzed using exploratory factor analysis. The results indicated that the scores from the oral imitation test, oral narration test, and timed GJT loaded on one factor, whereas the scores from the metalinguistic knowledge test and scores from the ungrammatical sentences on the untimed GJT loaded on a second factor. Ellis interpreted the two factors as corresponding to implicit and explicit knowledge, respectively.
R. Ellis’s (Reference Ellis2005) use of an exploratory factor analysis rather than a confirmatory factor analysis was criticized by Isemonger (Reference Isemonger2007) on the grounds that a confirmatory factor analysis is recommended in cases in which a priori hypotheses are being tested (Byrne, Reference Byrne2001; Kline, Reference Kline1994; Thompson, Reference Thompson2004; Thompson & Daniel, Reference Thompson and Daniel1996). Because Ellis’s study was indeed verificational rather than exploratory in nature and given that he did in fact intend to test a number of hypotheses, Ellis and Loewen (Reference Ellis and Loewen2007) subsequently reanalyzed the data from the original Ellis study using confirmatory factor analysis. The reanalysis confirmed the study’s original findings—that a two-factor model with the oral imitation test, the oral narrative test, and the timed GJT loading on one factor and the untimed GJT and the metalinguistic knowledge test loading on a second factor was an appropriate fit, with a χ2 of 1.191, whereas an alternate model proposed by Isemonger (Reference Isemonger2007) was not a good fit.
THE CURRENT STUDY
The purpose of this study was to validate that the battery of tests reported in R. Ellis (Reference Ellis2005) indeed provide relatively separate measures of implicit and explicit language knowledge. Whereas Ellis used a battery of tests in English and tested them on both native speakers (NSs) and L2 learners of English, this study includes a battery of tests designed closely following Ellis’s guidelines for testing in Spanish. Additionally, the tests were run on three groups: Spanish NSs, L2 learners, and HL learners. If the tests tap relatively separate pools of linguistic knowledge, then the L2 learners—who learned Spanish in a classroom setting after puberty and had no more than 1 week of study-abroad experience in a Spanish-speaking country—should score higher on tests of explicit knowledge. In contrast, the HL learners—who learned Spanish naturalistically since birth and had less than 2 years of formal classroom instruction in Spanish—should score higher on tests of implicit knowledge. The NSs, who were raised monolingually and had completed at least secondary school in Spanish, are expected to perform near ceiling on the oral imitation, oral narration, and both timed and untimed GJTs. Only a small amount of variation is expected because NS knowledge should be relatively stable. However, NSs are not expected to perform near ceiling on the metalinguistic knowledge test because it relies on grammatical terminology and rules that the NSs most likely do not access frequently. A larger percentage of variation (i.e., larger standard deviations in scores) is thus expected from one NS to the next on this test than on the other four.
Two specific research questions guided the current study.
1. Do scores on the battery of five tests load on two factors, in the way that they did in R. Ellis (Reference Ellis2005)?
2. Does a comparison of HL and L2 learners’ test scores provide construct validity evidence for the tests (i.e., do L2 learners score higher on tests of explicit knowledge, whereas HL learners score lower on those measures because of their naturalistic language acquisition process)?
Participants
A total of 30 participants completed the battery of five tests. Native speakers (n = 10) were all raised monolingually in Spanish-speaking countries and had immigrated to the United States as adults. Among the NSs, there were five males and five females, ranging in age from 24 to 36, with an average age of 30.66. They were from a broad range of countries of origin: one each from Puerto Rico, Costa Rica, Colombia, Chile, and Mexico, two from Peru, and three from Spain. They took the battery of tests primarily to ensure that the usage in the test items was normative and to establish baseline mean reaction times on each sentence used in the timed GJT.
Both the L2 and HL learners were enrolled in intermediate-level Spanish language classes at the university where the study was conducted. However, the language background profiles of the two groups were quite different. The L2 learners (n = 10) all reported that they had been raised in monolingual English-speaking households and had been exposed to Spanish in a classroom setting at or after puberty. Among the L2 learners, there were six females and four males, ranging in age from 19 to 22, with an average age of 20.2. They reported that they had not had any significant study-abroad experience (to qualify for the study, L2 learners could have spent a maximum of 1 week in a Spanish-speaking country). On average, the L2 learners had taken 6 years of Spanish courses (including secondary and postsecondary courses). The HL learners of Spanish (n = 10), in contrast, reported that they had been raised in bilingual English-Spanish households and that they routinely used Spanish throughout childhood and to the present day with at least one family member. It is important to note that all HL learners were English-dominant, given that they had been schooled in the United States and were attending a university where English was the language of instruction. To qualify for the study, HL learners could have had no more than 2 years of formal instruction in Spanish. This was done to ensure that the HL learners would have as little explicit knowledge as possible while still being literate in Spanish. Among the HL learners, there were seven females and three males, ranging in age from 19 to 23, with an average age of 20.6. They had taken an average of 1.3 years of Spanish courses (including secondary and postsecondary courses).
Test Content
Just as in R. Ellis (Reference Ellis2005), the tests in this study were designed to provide measures of the learners’ knowledge of 17 different grammatical structures. The specific structures were selected primarily because they have been shown to be problematic for both L2 and HL learners of Spanish. Ample research has shown that both L2 and HL learners of many languages, including Spanish, display morphological variability and have persistent difficulty with nominal and verbal inflection (see Montrul, this issue). In the nominal domain, the structures tested were gender agreement (Bruhn de Garavito & White, Reference Bruhn de Garavito, White, Pérez-Leroux and Liceras2003; Håkansson, Reference Håkansson1995; Montrul, Foote, & Perpiñán, Reference Montrul, Foote and Perpiñán2008; Polinsky, Reference Polinsky2008; White, Valenzuela, Kozlowska-Macgregor, & Leung, Reference White, Valenzuela, Kozlowska-Macgregor and Leung2004) and case-marking (Montrul & Bowles, Reference Montrul and Bowles2009; Polinsky, Reference Polinsky2006; Song, O’Grady, Cho, & Lee, Reference Song, O’Grady, Cho, Lee and Kim1997), both with dative psych verbs and with animate direct objects. In terms of verbal inflection, several structures that involve tense-aspect-mood morphology were tested, including subject-verb agreement, present subjunctive in both nominal and adverbial clauses (Montrul, Reference Montrul, Potowski and Cameron2007), past (imperfect) subjunctive (Potowski, Jegerski, & Morgan-Short, Reference Potowski, Jegerski and Morgan-Short2009), perfective versus imperfective aspect (Montrul, Reference Montrul2002; Silva-Corvalán, Reference Silva-Corvalán1994), and conditionals (representative stimuli for the tests are included in Appendixes A, B, and C; the entire stimulus is available upon request from the author). The selection of problematic structures, including those that are resistant to instruction or that are consistently problematic after years of instruction (e.g., aspect and mood distinctions), helped to ensure that there was a range of scores on the tests and that learner performance did not reach ceiling. Both early- and late-acquired forms were selected, as were structures introduced at a range of instructional levels. For instance, subject-verb agreement with regular present indicative verbs, as shown in the examples in (1) and (2), is early acquired and taught in beginner-level courses, whereas mood selection (indicative vs. subjunctive) in adverbial clauses, as shown in examples (3) and (4), is late acquired and is taught in intermediate and even advanced-level courses.Footnote 1
(1) Mamá y yo tomamos mucho café.
Mom and I drink.1pl.pres.ind. much coffee
“Mom and I drink a lot of coffee.”
(2) *Las chicas hablo inglés.
The.fem.pl girls speak.1sg.pres.ind. English
“The girls speak English.”
(3) Cuando el chico llegue a casa, te llamará.
When the.masc.sg boy arrive.3sg.pres.subj. to house, you.2sg.cl.
call.3sg.future
“When the boy gets home, he will call you.”
(4) *Cuando Juan tiene tiempo, estudiará francés.
When John have.3sg.pres.ind. time, study.3sg.future French
“When Juan has time, he will study French.”
Description of the Tests
Imitation Test. The oral imitation test consisted of 34 statements, half of which were grammatical sentences containing the target structures and half of which were ungrammatical sentences containing the target structures; that is, there was one grammatical and one ungrammatical sentence per target structure. The sentences were audio-recorded by a female NS of Spanish, and the resulting audio files were randomized to prevent ordering effects. The sentences were played one at a time for the participants, who had to state aloud whether they agreed or disagreed with the propositional content of the statement by saying sí “yes” or no “no.” Immediately after making this judgment, participants were required to repeat the sentence in correct Spanish. Participants’ responses were audio-recorded and subsequently analyzed by identifying obligatory occasions for the use of the target structures. Each correctly imitated sentence received a score of 1, whereas each sentence for which the target structure was avoided or attempted but incorrectly supplied received a score of 0.Footnote 2 For each participant, scores on this test were expressed as the percentage of sentences repeated or restated correctly out of a maximum of 34.
Oral Narrative Test.The picture story used in this test was designed to elicit the use of a number of the target structures, including reflexive verbs, subject-verb agreement, present or simple past tense inflectional morphology, gender agreement, and aspect (perfective vs. imperfective). The participants saw the vignettes from “The tricky alarm clock,” a wordless picture story, which has been used in previous SLA elicitation tasks (Adams, Reference Adams2003; Swain & Lapkin, Reference Swain and Lapkin2002). The vignettes were accompanied by a story written in Spanish to go along with the pictures. Participants read the story and looked at the corresponding pictures twice. Then the third time, only the pictures remained on the screen and the participants were asked to retell the story aloud. Their narratives were audio-recorded and subsequently transcribed. A transcript of one of the HL learners’ narrations is provided in Appendix D. As with the oral imitation test, an obligatory occasion analysis was carried out to establish the percentage of correct suppliance of each target structure. For each participant, scores on this test were calculated by averaging the percentage scores for each structure.
Timed GJT. The timed GJT consisted of 68 sentences, evenly divided between grammatical and ungrammatical items. Each sentence was presented in written form on a computer screen, and there were four sentences for each of the 17 grammatical structures. The test was delivered using E-Prime software (version 1.2), and the sentences were automatically randomized. Participants were required to indicate whether each sentence was grammatical or ungrammatical by pressing one of two keys on the computer keyboard within a fixed time limit. The time limit for each sentence was established on the basis of the NSs’ average response time for each stimulus. Following R. Ellis (Reference Ellis2005), the average response time for the NSs was increased by 20% to allow for the slower processing speed of the learners (time limits were rounded to the nearest millisecond). The specific time allowed for judging the individual sentences ranged from 1,742 to 5,467 ms. Each item was scored dichotomously as correct or incorrect, with items left unanswered at the end of the time limit scored as incorrect. A percentage accuracy score was calculated for each participant.Footnote 3
Untimed GJT. The untimed GJT included the same types of sentences (but different tokens) as the timed GJT. It was also delivered in E-Prime (version 1.2) and sentences were presented in written form and randomized automatically. Participants were required to indicate whether each sentence was grammatical or ungrammatical, just as they had done in the timed GJT—except that they were instructed to answer at their own pace because the test had no set time limit.
Metalinguistic Knowledge Test. This test was based on the test designed in Alderson, Clapham, and Steel (Reference Alderson, Clapham and Steel1997) and reported in R. Ellis (Reference Ellis2005). It was a two-section untimed computerized multiple-choice test. The first section contained 17 ungrammatical sentences, 1 per target structure, and instructed participants to select the rule that best explained why each sentence was ungrammatical. In the second section of the test, participants read a short text written for beginning-level learners of Spanish (see Appendix C) and were instructed to find examples of 21 specific grammatical features from the text (e.g., verb in the imperfect, subject pronoun). A total percentage accuracy score was calculated out of the total number of points possible.Footnote 4
These tests, like those in R. Ellis (Reference Ellis2005), were designed to provide a relatively separate measure of either implicit or explicit knowledge. The oral imitation test and the oral narration test were predicted to measure implicit knowledge because the participants would need to rely mainly on feel or intuition; under pressure to perform in real time, they would be focused primarily on communicating meaning rather than on grammatical form and would have no reason to access their metalanguage. In contrast, the metalinguistic knowledge test was predicted to measure explicit knowledge because it involved a high degree of awareness, had no time pressure, focused attention on grammatical form, and—obviously—required the use of explicit knowledge. Both of the GJTs required participants to focus attention primarily on grammatical form by their very nature, but the two differed in that the timed GJT was predicted to measure primarily implicit knowledge, whereas the untimed GJT was predicted to measure primarily explicit knowledge. The timed task encouraged the use of feel and was time-pressured, allowing participants little access to explicit knowledge. In contrast, the untimed GJT encouraged a high degree of awareness and was self-paced, with no time pressure. These features of the untimed GJT distinguish it from the timed GJT, because participants could take their time and use explicit knowledge if they wished.Footnote 5
As in R. Ellis (Reference Ellis2005), the tests were administered in a consistent order, with those designed to tap implicit knowledge administered before those designed to tap explicit knowledge. This order of presentation ensured that the explicit knowledge tests would not prime learners. The oral imitation test was administered first, followed by the oral narration test, the timed GJT, the untimed GJT, and, finally, the metalinguistic knowledge test.
RESULTS
Given the methodological issues raised by Isemonger (Reference Isemonger2007), a confirmatory factor analysis, rather than an exploratory factor analysis as in R. Ellis (Reference Ellis2005), was used to answer the first research question. Following Nunnally (Reference Nunnally1978), who recommended having 10 times as many participants as factors, and Kass and Tinsley (Reference Kass and Tinsley1979), who recommend having between 5 and 10 participants per factor, it was considered that the minimum acceptable sample size for this type of analysis to be meaningful would include 20 participants. By this metric, the final sample size of 30 participants should be sufficient to find meaningful loadings on the two factors.
Table 1 shows the reliability coefficient for each of the five tests. Cronbach’s alpha coefficients were calculated for all except the oral narration test, for which it was necessary to calculate interrater reliability. All coefficients were well above .80, indicating that all of the tests were internally consistent.
Reliability coefficients for the five tests
Table 2 shows the means and standard deviations of scores on the five measures by NSs and L2 and HL learners. Native speakers, as predicted, scored near ceiling (92% or higher), with very little variance on all measures except the metalinguistic knowledge test. Their scores on all tests except the metalinguistic knowledge test were higher than those of both the L2 and HL learners. The HL learner group outperformed the L2 learners on all measures and with scores in the 75–80% range, except for the metalinguistic knowledge test, in which they scored less than 50% and the L2 learners scored at nearly the same levels as NSs.
Accuracy scores for the five tests by group
The mean scores on the five tests were submitted to separate one-way ANOVAs, with Group (NS, L2, HL) as the between-subjects factor. There were statistically significant between-group differences on all five tests. Table 3 includes the F statistics, p values, and results of the Scheffé post hoc analyses that were performed to determine the sources of the between-group differences.
Results of one-way ANOVAs on test scores for the three groups
Note
The symbol > indicates that the first group scored significantly higher than the second group.
* p < .05.
** p < .0001.
Table 4 shows the correlation matrix for L2 and HL learners’ scores (n = 20) on the battery of tests. All correlations were significant at the .01 level, and the tests designed to measure implicit knowledge were highly intercorrelated. The tests designed to tap into explicit knowledge showed a different pattern of results: The metalinguistic knowledge test correlated negatively with the other four measures, and the untimed GJT was far less highly correlated with the other measures.
Correlation matrix for the five tests (L2 and HL learners, n = 20)
A confirmatory factor analysis was then run using AMOS version 17.0 (Arbuckle, Reference Arbuckle2008), with the two-factor model reported in R. Ellis (Reference Ellis2005) and Ellis and Loewen (Reference Ellis and Loewen2007) specified a priori. The results of the analysis are presented in Figure 1, and the summary statistics for the model fit are χ2 = 2.9, df = 4, p = .90, normed fit index (NFI) = .974, root mean square error of approximation (RMSEA) = .000. As in previous studies, the χ2 statistic was used to test the goodness of fit. A significant p value indicates that the model is statistically unlikely to occur, whereas a nonsignificant value, such as the one obtained, indicates an acceptable fit (Byrne, Reference Byrne2001). Goodness of fit was also analyzed according to the metrics used in Ellis and Loewen, in which a NFI value of greater than .95 indicates a superior fit and a RMSEA value of less than .05 indicates a good fit for the model.
Implicit-explicit model. TGJT refers to timed grammaticality judgment test and UGJT refers to untimed grammaticality judgment test.
DISCUSSION
This study was primarily concerned with providing evidence for the construct validity of the battery of five tests designed in R. Ellis (Reference Ellis2005) to provide relatively separate measures of implicit and explicit knowledge. The first research question addressed the issue of validity broadly and asked whether scores on the five tests would load on two separate factors—one representing implicit knowledge and the other representing explicit knowledge. If the tests in R. Ellis (Reference Ellis2005) are replicable and tap different sources of knowledge, then the tests adapted in this study for Spanish and tested on both classroom L2 learners and naturalistic HL learners should load on separate factors. The results provide evidence that these tests, like those developed by Ellis in English, indeed tap relatively separate pools of knowledge.Footnote 6
The second research question addressed the construct validity of R. Ellis’s (Reference Ellis2005) test battery in a more fine-grained way. If the tests really measure relatively separate pools of linguistic knowledge, then L2 learners who have learned Spanish in a classroom setting should score higher on tests of explicit knowledge and lower on tests of implicit knowledge. In contrast, HL learners, who learned Spanish naturalistically but have received less than 2 years of formal instruction, should score lower on tests of explicit knowledge and higher on tests of implicit knowledge.
The test scores from this study provided such evidence. L2 learners scored highest overall on the two tests that were designed to measure explicit knowledge—the metalinguistic knowledge test (72.4%), followed by the untimed GJT (66.9%). Their scores on the remaining three tests—those that were designed to measure implicit knowledge—were all under 50% accuracy. The HL learners’ scores, however, showed the opposite pattern. Whereas L2 learners scored highest on the metalinguistic knowledge test, HL learners as a group scored the lowest on this test (with an average accuracy of just 57.4%). In a similar manner, whereas L2 learners scored lowest on the timed GJT, oral narration, and oral imitation tests, HL learners scored high on all three of these tests, with the oral narration in fact being the test on which they scored highest as a group (95.9%). These findings point to the content validity of the battery of tests because the measures appear to distinguish between the L2 learners, who should have higher explicit knowledge, and the HL learners, who should have minimal explicit knowledge. In a recent study that examined the relationship between background-contextual variables and English language knowledge, Philp (Reference Philp, Ellis, Loewen, Elder, Erlam, Philp and Reinders2009) found similar results. The participants who scored highest on her implicit language tests clustered together, and subsequent analysis of their language background questionnaires revealed that they had all begun to acquire English at or before the age of 6 and were regular users of English in their homes or community. Although Philp (Reference Philp, Ellis, Loewen, Elder, Erlam, Philp and Reinders2009) did not use the nomenclature heritage learner in her article, from her descriptions, it appears that the participants in this cluster were a mixture of L1 English speakers and heritage learners of English.
A comparison of the participants’ scores on the untimed versus timed GJTs provides additional evidence in support of the validity of the explicit-implicit model proposed by R. Ellis (Reference Ellis2005). NSs, L2 learners, and HL learners all scored significantly lower on the timed GJT than they did on the untimed GJT: NS, t(9) = 2.98, p < .05, d = 1.19; L2, t(9) = 4.41, p < .01, d = 2.64; HL, t(9) = 5.02, p < .001, d = 1.44. This indicates that time pressure significantly affected accuracy for all three groups and to a nontrivial extent, as the large effect sizes indicate. The largest effect size (d = 2.64), which was found for the L2 learners, is nearly twice as large as that of the HL learners (d = 1.44) and more than twice as large as that of the NSs (d = 1.19), thus demonstrating that time was a much greater factor for L2 learners than for either HL learners or NSs of Spanish, most likely because the L2 learners relied more heavily on their explicit knowledge to complete the untimed GJT than did participants in the other groups.
This study therefore provides support for the construct validity of the battery of tests in R. Ellis (Reference Ellis2005), in a different language and with another population of learners. It brings the field of SLA research one step closer to having a series of reliable and valid measures available to administer to learners to measure their implicit and explicit linguistic knowledge. Such measures should allow SLA researchers, in R. Ellis’s (Reference Ellis2005) words, to “investigate issues of central theoretical importance in the study of L2 acquisition” (p. 169). Research on the controversial interface-noninterface positions certainly comes to mind, but such measures could be used in part to address other long-standing issues, such as the role of age effects in language acquisition or the relationship between type of instruction and the nature of resulting linguistic knowledge. In the field of HL acquisition, which has been able to benefit from fewer instructional intervention studies than SLA, it would appear crucial to determine how HL learners’ linguistic knowledge differs from that of monolingually raised NSs or from that of L2 learners who were first exposed to the language in early adulthood. Then, armed with that knowledge, researchers and teachers will be better equipped to design instruction (and empirical studies on the effects of different types of instruction) to determine in what ways various components, such as explicit information, negative evidence, and structured input, affect these bilinguals’ HL systems. A few studies have begun to address these issues (Montrul & Bowles, Reference Montrul, Bowles, Bowles, Foote, Perpiñán and Bhatt2008, Reference Montrul and Bowles2009; Potowski et al., Reference Potowski, Jegerski and Morgan-Short2009), but clear measures of implicit and explicit knowledge will help to further hone and refine measurement and instruction in this growing field of investigation.
Appendix A
REPRESENTATIVE SAMPLE OF SENTENCES IN THE ORAL IMITATION TEST
Grammatical
(1) Los estudiantes generalmente se levantan muy temprano.
“Students generally get up very early.”
(2) Los padres quieren que sus hijos saquen malas notas.
“Parents want their children to get bad grades.”
(3) Los estudiantes bailarán mucho cuando tengan tiempo.
“The students will dance a lot when they have time.”
(4) Barack Obama quería que muchos jóvenes votaran en las últimas elecciones.
“Barack Obama wanted many young people to vote in the last elections.”
(5) Los políticos conocen a muchas personas.
“Politicians know many people.”
Ungrammatical
(6) *Los estudiantes acuestan tarde.
“Students go to bed late.”
(7) *Los maestros quieren que sus estudiantes estudian poco.
“Teachers want their students to study little.”
(8) *Los estudiantes beben mucho alcohol cuando irán de vacaciones.
“Students drink a lot of alcohol when they go on vacation.”
(9) *Sarah Palin quería que muchos demócratas voten en las últimas elecciones.
“Sarah Palin wanted many democrats to vote in the last elections.”
(10) *Los estudiantes visitan sus familias los fines de semana.
“Students visit their families on weekends.”
APPENDIX B
REPRESENTATIVE SAMPLE OF SENTENCES IN TIMED AND UNTIMED GRAMMATICALITY JUDGMENT TESTS
Reflexive Verbs
(1) Me ducho por la mañana.
“I shower in the morning.”
(2) *Lavo los dientes tres veces al día.
“I brush my teeth three times a day.”
Present Subjunctive with Nominative Clauses
Present Subjunctive with Nominative Clauses
(3) Mamá quiere que Paco estudie mucho.
“Mom wants Paco to study a lot.”
(4) *Ana quiere que salemos con ella.
“Ana wants us to go out with her.”
Present Subjunctive with Adverbial Clauses
(5) Cuando el chico llegue a casa, te llamará.
“When the boy gets home, he will call you.”
(6) *Cuando Juan tiene tiempo, estudiará francés.
“When Juan has time, he will study French.”
Imperfect Subjunctive
(7) Ana quería que su hijo mirara menos televisión.
“Ana wanted her son to watch less television.”
(8) *Papá quería que Nuria lave la ropa.
“Dad wanted Nuria to wash the clothes.”
Differential Object Marking
(9) María visita a su abuela cada semana.
“Maria visits her grandmother every week.”
(10) *Pedro ve el profesor el lunes.
“Pedro sees the profesor on Monday.”
Noun and Adjective Gender Agreement
(11) Manuel vive en una casa roja.
“Manuel lives in a red house.”
(12) *Ana tiene una falda negro.
“Ana has a black skirt.”
Subject-Verb Agreement (Regular Present Tense Verbs)
(13) Paula mira la televisión.
“Paula watches television.”
(14) *José y yo toman clases por la noche.
“Jose and I take classes at night.”
Adjective Placement (Postnominal)
(15) Tengo un carro rojo.
“I have a red car.”
(16) *Quiero una blanca camisa.
“I want a white shirt.”
Ser or Estar (with Professions)
(17) Mi padre es médico.
“My father is a doctor.”
(18) *Mi tío está ingeniero.
“My uncle is an engineer.”
Ser or Estar (with Locations of Objects)
(19) Los libros están en la biblioteca.
“The books are in the library.”
(20) *Mi cartera es en el apartamento.
“My wallet is in the apartment.”
Ser or Estar (with Events)
(21) La fiesta es el 28 de febrero.
“The party is on February 28.”
(22) *El baile está a las 9:00.
“The dance is at 9:00.”
Gustar
(23) A los estudiantes no les gusta el examen.
“The students do not like the exam.”
(24) *Los niños les gusta los caramelos.
“Children like candies.”
Past Hypothetical Conditionals
(25) Si Roberto hubiera estudiado, no habría sacado malas notas.
“If Roberto had studied, he would not have gotten bad grades.”
(26) *Si mi familia había tenido dinero, habríamos ido de vacaciones.
“If my family had had money, we would have gone on vacation.”
Possessives
(27) Como mi apartamento es pequeño, prefiero el tuyo.
“Since my apartment is small, I prefer yours.”
(28) *Mi clase favorita es español pero la tu es matemáticas.
“My favorite class is Spanish but yours is math.”
Aspect
(29) Ayer los niños jugaron hasta muy tarde.
“Yesterday the children played until very late.”
(30) *Anoche los chicos estudiaban hasta las 3:00.
“Last night the boys studied until 3:00.”
Article Use with Generics
(31) Los delfines son animales inteligentes.
“Dolphins are intelligent animals.”
(32) *Perros ladran mucho.
“Dogs bark a lot.”
Double Object Construction
(33) Le escribo cartas a mi hermano.
“I write letters to my brother.”
(34) *Le doy mi mamá rosas.
“I give my Mom roses.”
APPENDIX C
REPRESENTATIVE SAMPLE OF QUESTIONS FROM THE METALINGUISTIC KNOWLEDGE TEST
Part 1: Each sentence below is ungrammatical in Spanish. Please select the option that explains why each sentence is ungrammatical.
1. Los estudiantes duchan en el gimnasio.
“Students shower at the gym.”
a. Missing reflexive pronoun
b. Missing indirect object pronoun
c. Missing subject pronoun
d. Missing demonstrative pronoun
2. Paula sugiere que estudias para el examen.
“Paula suggests that you study for the exam.”
a. A verb is conjugated in the wrong person
b. A verb is conjugated in the wrong tense
c. A verb is conjugated in the wrong aspect
d. A verb is conjugated in the wrong mood
Part 2: In the paragraph below, find one example of each grammatical feature and type it in the space provided. Note: There are many possible answers.
1. Demonstrative adjective
2. Verb in third person
Text Source: Lejos de casa, Centro Virtual Cervantes, Texto de nivel inicial, http://cvc.cervantes.es/aula/lecturas/inicial/lectura_10/texto/
APPENDIX D
TRANSCRIPT OF ONE OF THE HL LEARNERS’ ORAL NARRATIONS
Un día al amanecer había una chica tumbada en la cama que … que no se quería levantar para la escuela. A las seis en punto en la mañana suena la alarma del despertador pero ella no se quería levantar. Decide dormirse unos minutos más después de que sonó su alarma. A las seis y uno la muchacha se volvió a dormir muy rápido. A las seis y dos en punto de la mañana sale una mano de su despertador y le hizo cosquillas en sus pies. La muchacha se despertó. Se levantó y se fue al baño y se peinaba mientras se lavaba los dientes. Y después se fue a vestir. Rápido estuvo lista para ir a la escuela. Se puso su mochila, justo a tiempo para llegar a la escuela a tiempo.
“One day at dawn there was a girl lying in bed who … who didn’t want to get up to go to school. At six o’clock in the morning sharp the alarm from the alarm clock goes off but she didn’t want to get up. She decides to sleep a few minutes more after her alarm went off. At 6:01 the girl went back to sleep very quickly. At 6:02 in the morning sharp a hand comes out of her alarm clock and it tickled her feet. The girl woke up. She got up and went to the bathroom and brushed her hair while she was brushing her teeth. And then she went to get dressed. Quickly she was ready to go to school. She put on her backpack, just in time to get to school on time.”
