Introduction
The working memory model
First used by Miller, Galanter and Pribram in Reference Miller, Galanter and Pribram1960, the term ‘working memory’ refers to the “temporary storage and manipulation of information that is assumed to be necessary for a wide range of complex cognitive activities” (Baddeley, Reference Baddeley2003, p. 189). In 1974, Baddeley and Hitch proposed that the earlier unitary concept of working memory could be elaborated into three subsystems, thus forming the theoretical framework commonly known as the working memory model. This three-component system comprises a “limited capacity attentional controller” (Baddeley, Reference Baddeley2002, p. 86), the central executive. This is aided by two slave systems, one concerned with visual and spatial information (the visuo-spatial sketchpad, or VSSP), and the other performing a similar function for acoustic and verbal information (the phonological loop). More recently, Baddeley has proposed a fourth component, the episodic buffer (Baddeley, Reference Baddeley2000). This multi-component system has received substantial independent support from both cognitive neuropsychology and brain-imaging studies, which have provided convergent evidence that phonological and visuo-spatial memory processing are functionally separate within the brain (ibid.). It has thus remained the dominant model of short-term memory for some years.
Of particular interest to this study are the phonological loop and the visuo-spatial sketchpad. The phonological loop is comprised of two subcomponents: a phonological store and a subvocal rehearsal system. Memory traces within the store are thought to decay after about two seconds unless they are refreshed using the subvocal rehearsal process (repeating items to oneself, aloud or in one's head). Information in spoken form gains obligatory access to the phonological loop. In addition, visually-presented information (printed words or familiar visual objects) can be phonologically recoded and stored there temporarily. Therefore interference mechanisms, such as articulatory suppression, tend to hinder phonological memory, as concurrent articulation blocks the use of the phonological rehearsal system and thus interferes with the ability to retain phonological information. A similar phenomenon occurs when visually-presented material shares phonological characteristics. The phonological similarity effect (PSE) – first recorded by Conrad and Hull in Reference Conrad and Hull1964 – refers to the fact that phonologically similar sequences of letters, such as T, C, V, D, G, B are more difficult to recall in the correct order than phonologically distinct items, such as W, Y, K, R, X. Words which sound alike – for instance, man, cat, cap, map, can – are also subject to the PSE (Baddeley, Reference Baddeley1966).
Whilst the phonological loop is responsible for encoding speech-based material, the VSSP serves the function of temporarily maintaining and manipulating spatial and visual material (and, possibly, kinaesthetic information; see Baddeley, Reference Baddeley2003). Long neglected in favour of phonological loop, experimental research has only recently begun to investigate this component of working memory, perhaps because ‘pure’ measures of visuo-spatial functioning – material that cannot be semantically or phonologically recoded – are difficult to develop (Pearson, Reference Pearson and Andrade2001). The VSSP functions primarily to integrate visual and spatial representations, forming an interface between the two which is accessed either through the senses or from long-term memory (Baddeley, Reference Baddeley2002). However, visual and spatial elements may also work in isolation (Pearson, Reference Pearson and Andrade2001). Depending on the memory task, storage may be primarily spatial (this is reflected in the Corsi block-tapping task; see Baddeley & Lieberman, Reference Baddeley, Lieberman and Nickerson1980) or primarily visual as represented by colour and shape (reflected more strongly in pattern span; see Logie, Reference Logie1986). Like the phonological loop, the sketchpad's functioning may be disrupted by requiring participants to tap repeatedly a specified pattern of keys or locations, which disturbs the use of visuo-spatial imagery (Baddeley & Lieberman, Reference Baddeley, Lieberman and Nickerson1980). Unattended patterns or visual ‘noise’ may also disrupt the visual component of the system (Quinn & McConnell, Reference Quinn and McConnell1996). Further, items which look similar are more difficult to recall – a phenomenon known as the visual similarity effect or VSE (Hitch, Halliday, Schaafstal & Schraagen, Reference Hitch, Halliday, Schaafstal and Schraagen1988).
The present study has examined the role of the phonological loop and the VSSP in encoding visually-presented material, in this case, pictures of familiar objects. Both systems are clearly implicated in the process: the VSSP is involved in correctly identifying the image, and in other functions such as maintaining a stable representation of the page and its layout (Baddeley, Reference Baddeley2003). The phonological loop allows the stimuli a label, and thus helps to maintain its distinctiveness in storage; it is also essential for recall (Gathercole & Baddeley, Reference Gathercole and Baddeley1993). Yet experimental research seems to suggest that the roles played by the VSSP and the phonological loop in encoding visually-presented material are not equal; in fact, the interplay between the two components changes according to age. One important line of evidence on children's memory for visually-presented material comes from studies involving the recall of pictured objects (e.g. Halliday, Hitch, Lennon & Pettipher, Reference Halliday, Hitch, Lennon and Pettipher1990; Hitch et al., Reference Hitch, Halliday, Schaafstal and Schraagen1988). Such studies have found that young children tend to encode pictures of familiar objects in a visual form: they are subject to the VSE, recalling visually distinct items better than those which are visually similar (for example, long items with the major axis in the same oblique orientation: nail, bat, key, spade, comb, saw, fork and pen; see Hitch et al., Reference Hitch, Halliday, Schaafstal and Schraagen1988). These children do not benefit from being instructed to name the objects during presentation (ibid.). However, from the age of eight onwards, children tend to use a more phonological approach when remembering the same pictures (ibid.). These children are not subject to the VSE, though they benefit from naming objects during presentation; furthermore, they become confused when confronted with phonetically similar items, thus showing the PSE (Halliday et al., Reference Halliday, Hitch, Lennon and Pettipher1990). It seems, therefore, that young children are more reliant on visuo-spatial working memory when storing and manipulating visually-presented information, a reliance which declines through childhood in favour of phonological strategies (Hitch, Halliday, Dodd & Littler, Reference Hitch, Halliday, Dodd and Littler1989; Pickering, Reference Pickering2001).
Several reasons have been proposed for this decreased reliance on the VSSP (see Hitch et al., Reference Hitch, Halliday, Schaafstal and Schraagen1988; Longoni & Scalisi, Reference Longoni and Scalisi1994; Palmer, Reference Palmer2000). One of the major factors responsible for this change in coding behaviour is the acquisition of literacy, especially in an alphabetic orthography – writing systems in which each letter roughly represents a phoneme. Fastenau, Conant and Lauer (Reference Fastenau, Conant and Lauer1998) suggest that reading instruction – itself a process of phonological recoding – is an important factor in the increased use of phonological techniques in visuo-spatial exercises. Cross-cultural studies provide support for this finding (Conant et al., Reference Conant, Fastenau, Giordani, Boivin, Chounramany, Xaisada, Choulamountary, Pholsena and Olness1997, Reference Conant, Fasternau, Giordani, Boivin, Opel and Nseyila1999, Reference Conant, Fastenau, Giordani, Boivin, Chounramany, Xaisada, Chounramany and Pholsena2003).
Working memory within logographic languages: the case of Chinese
Until quite recently, it was assumed that reading development was largely dependent on the phonological loop (Huang & Hanley, Reference Huang and Hanley1994). In recent years, however, there has been a growing interest in comparative investigations of word recognition in the other major writing systems, namely syllabic (e.g. Japanese kana) and logographic systems (Chinese or early Egyptian), in which each character represents a syllable or a morpheme, respectively (Ho & Bryant, Reference Ho and Bryant1997). As such, cross-cultural research has discovered that the process of reading, and the interaction between the phonological and the visual components of working memory during word recognition, may be different in other orthographies (Chen, Reference Chen1996).
Research has shown particular interest in comparing English and Chinese – two of the most orthographically distant languages in current usage (Perfetti, Liu & Tan, Reference Perfetti, Liu and Tan2002) and some interesting results have begun to emerge. In a seminal study, Huang and Hanley (Reference Huang and Hanley1994) found that visuo-spatial working memory, not phonological awareness, was more important as a predictor of reading skill for Chinese children than for English controls (Hitch et al. Reference Hitch, Halliday, Schaafstal and Schraagen1988). A comparable study by Smythe, Everatt, Gyarmathy, Ho and Groeger (Reference Smythe, Everatt, Gyarmathy, Ho and Groeger2003) found that poor readers in alphabetic orthographies (English and Hungarian) show deficits in tests of verbal memory, whilst poor readers in Chinese perform badly on visual memory tests (see also Liu, Liu & Xu, Reference Liu, Liu and Xu2004). These studies suggest that reading different orthographies activates very different word processing mechanisms – the orthographic variation hypothesis (Chen & Juola, Reference Chen and Juola1982).
A rudimentary analysis of the writing system of the Chinese language soon reveals why it may contain more of a visuo-spatial emphasis than alphabetic orthographies. Chinese does not possess the segmental, ‘atomistic’ structure that is rudimentary to the alphabetic system (Ho & Bryant, Reference Ho and Bryant1997). Instead, phonetic notation is holistic (Siok, Perfetti, Jin & Tan, Reference Siok, Perfetti, Jin and Tan2004; Wang & Geva, Reference Wang and Geva2003). It is estimated that a child must learn to recognise as many as 2570 different characters by the age of twelve – 470 of these during their first year at school (Shu, Chen, Anderson, Wu & Xuan, Reference Shu, Chen, Anderson, Wu and Xuan2003). Consequently, it is possible that learning to read Chinese may depend more in making appropriate visual distinctions than on phonological skills. The nonlinear spatial layout of Chinese characters – interwoven strokes in a square-shape form – and its high spatial frequency may also make more demands on spatial memory than the linear arrangement of alphabetic scripts (Hoosain, Reference Hoosain2002; Wang, Koda & Perfetti, Reference Wang, Koda and Perfetti2003). Research suggests that the gradual command of written Chinese builds complex visuo-spatial skills that may affect all aspects of mental functioning, from basic processing mechanisms to memory strategies and even spatial IQ (Demetriou, Zhang, Spanoudis, Christou, Kyriakides & Platsidou, Reference Demetriou, Zhang, Spanoudis, Christou, Kyriakides and Platsidou2005; Jensen, Reference Jensen1998; Lynn, Reference Lynn and Nyborg2003).
However, the nature of written Chinese is often misunderstood. It is not a wholly pictographic system with an arbitrary script-to-sound relationship (Baron & Strawson, Reference Baron and Strawson1976). Certainly, Old Chinese – the language of the Spring and Autumn period of the Zhou dynasty (722–481 bc) – was pictographic: characters were pictures of objects which immediately evoked meaningful interpretations (Shu & Anderson, Reference Shu and Anderson1999). Alongside these pictograms were ideograms, frequently formed by putting several pictograms together: compare, for example, the symbols for “tree” (
), “grove” (
), and “forest” (
). However, this system ultimately proved inefficient (Hung & Tzeng, Reference Hung and Tzeng1981). Instead, modern Chinese follows what might more accurately be known as a morphosyllabic system (Huang, Reference Huang1996; Perfetti & Liu, Reference Perfetti and Liu2006), and over 80% of figures contain a component that hints at the pronunciation of the character (Jackson, Chen, Goldsberry, Kim & Vanderwerff, Reference Jackson, Chen, Goldsberry, Kim and Vanderwerff1999). These components can be divided into two parts: a semantic radical and a phonetic radical (Ho, Ng & Ng, Reference Ho, Ng and Ng2003; Lee, Tsai, Huang, Hung & Tzeng, Reference Lee, Tsai, Huang, Hung and Tzeng2006). The semantic radical, like the ancient pictogram, suggests something about the literal or metaphorical meaning of the character. Characters containing the semantic radical “tree”, for example, have meanings related to the concept: names for parts of a tree, or things made of wood (Jackson, Lu & Ju, Reference Jackson, Lu, Ju and Berninger1994). The phonetic radical provides a cue as to the character's pronunciation. While most phonetic radicals may also exist as real characters (known as ‘free-standing’), others cannot exist on their own as legitimate and independent characters (‘non-free-standing’; Law, Weekes & Wong, Reference Law, Weekes and Wong2006). Thus, in the Chinese script, unlike in alphabetic scripts, there are no character components that correspond to phonological units smaller than a syllable (ibid.).
Though this phonetic cue is not always reliable or consistent (Huang & Hanley, Reference Huang and Hanley1994; Wang et al., Reference Wang, Koda and Perfetti2003), there is a suggestion that phonology plays a significant role in processing Chinese characters (Lee et al., Reference Lee, Tsai, Huang, Hung and Tzeng2006). Indeed, some studies have reported a small but significant effect of homophone interference in meaning-related tasks (e.g. Chua, Reference Chua1999) and a ‘tongue-twister’ effect in oral and silent reading (Zhang & Perfetti, Reference Zhang and Perfetti1993), suggesting that, like English, Chinese engages phonological codes in working memory – albeit at a lexical, rather than a pre-lexical, level (Wang et al., Reference Wang, Koda and Perfetti2003). It is therefore possible that a child with good phonological skills may learn Chinese more quickly than a child with less phonological ability (Chan & Nunes, Reference Chan and Nunes1998; Ho & Bryant, Reference Ho and Bryant1997). This is certainly the belief of the Chinese and Taiwanese educational authorities, who, in the 1950s, introduced a phonetic system for learning to read Chinese (Hanyu Pinyin in China and Zhu-Yin-Fu-Hao in Taiwan) alongside Chinese characters (Huang & Hanley, Reference Huang and Hanley1994).
However, though phonology and orthography both contribute to the reading of Chinese, research suggests that the balance is the opposite of that observed in literate alphabetic cultures: orthographic information plays a stronger role than phonological information (Zhou, Shu, By & Shi, Reference Zhou, Shu, Bi and Shi1999). This suggestion has been supported by numerous experimental paradigms. Studies using the Stroop effect, for instance, have reported a greater lexical interference effect for Chinese participants in a Chinese version of the colour-naming task than for American participants in an English version (Hung & Tzeng, Reference Hung and Tzeng1981). However, it is studies using brain imaging techniques that have provided the most convincing support of all for the visuo-spatial emphasis of Chinese. PET and fMRI scans of word recognition in participants from alphabetic backgrounds have shown that, though there is some visual processing in the posterior fusiform and lingual giri, processing focuses primarily on left mid-superior temporal regions. These areas are specifically identified as being responsible for fine-grained phonemic analysis and conversion of written symbols to phonological units of speech (Kuo, Yeh, Duann, Wu, Ho, Hung, Tzeng & Hseih, Reference Kuo, Yeh, Duann, Wu, Ho, Hung, Tzeng and Hseih2001; Simos, Breier, Fletcher, Foorman, Castillo & Papanicolaou, Reference Simos, Breier, Fletcher, Foorman, Castillo and Papanicolaou2002).
Studies of logographic orthographies such as Chinese, however, suggest some important differences. Research conducted by Tan and colleagues (Tan et al. Reference Tan, Spinks, Gao, Liu, Perfetti, Xiong, Pu, Liu, Stofer and Fox2000, Reference Tan, Liu, Perfetti, Spinks, Fox and Gao2001) found more right hemisphere cortical regions (particularly the right frontal and parietal regions) to be involved in reading Chinese relative to reading English or Pinyin (see Matthews, Fu, Chen & Iversen, Reference Matthews, Fu, Chen and Iversen2002). In addition, they discovered increased activation in the right occipital cortex, which is associated with the spatial recognition of visual symbols (Shen, Hu, Yacoub & Ugurbil, Reference Shen, Hu, Yacoub and Ugurbil1999). Of particular interest is activation in the left middle frontal gyrus, an area pertinent to spatial working memory which is not activated by alphabetic languages (Courtney, Petit, Maisog, Ungerleider & Haxby, Reference Courtney, Petit, Maisog, Ungerleider and Haxby1998). Tan et al. (Reference Tan, Liu, Perfetti, Spinks, Fox and Gao2001) propose that the extremely strong activation of this area in reading Chinese is associated with the unique square configuration of the logographs. More specifically, they believe that the left middle frontal area may act as a central executive system, responsible for coordinating cognitive resources i.e. integrating the visuo-spatial analysis demanded by the logographs’ square configuration, and semantic (or phonological) cues (D'Esposito, Detre, Alsop, Shin, Atlas & Grossman, Reference D'Esposito, Detre, Alsop, Shin, Atlas and Grossman1995).
The issue here, however, is not simply whether Chinese and English employ different processing mechanisms per se, but whether the effect posited by the orthographic variation hypothesis (Chen & Juola, Reference Chen and Juola1982) persists between languages. Indeed, studies suggest that logographic readers learning to read alphabetic scripts tend to rely more on orthographic information than native English readers (Leong, Tan, Cheng & Hau, Reference Leong, Tan, Cheng and Hau2005; Wang et al., Reference Wang, Koda and Perfetti2003). An experiment by Wang and Geva (Reference Wang and Geva2003), for example, showed that, whilst Chinese children learning English as a second language (ESL) showed poorer performance than native English controls on spelling to dictation of pseudowords, they outperformed their L1 counterparts in a spelling task of visually-presented orthographically legitimate and illegitimate letter strings. It seems that readers from logographic backgrounds are better able to process orthographic information in alphabetic L2s than grapheme–phoneme correspondences (Jackson et al., Reference Jackson, Lu, Ju and Berninger1994; Koda, Reference Koda1988).
Brain imaging technology has also been used to investigate the cortical activation of individuals from a logographic L1 reading an alphabetic script. As a continuation of the aforementioned 2001 study, Tan, Spinks, Feng, Siok, Perfetti, Xiong, Fox and Gao (Reference Tan, Spinks, Feng, Siok, Perfetti, Xiong, Fox and Gao2003) used fMRI to investigate differences between native English participants and participants who were Chinese L2-English speakers (henceforth Chinese ESL). One might expect that the left mid-superior temporal regions would be active, as is the case in phonological processing of alphabetic words by English monolinguals (Poldrack, Wagner, Prull, Desmond, Glover & Gabrieli, Reference Poldrack, Wagner, Prull, Desmond, Glover and Gabrieli2001). However, Chinese bilinguals showed activity in the same neural system used to process Chinese words, namely the left middle frontal and posterior parietal gyri (associated with spatial working memory and the central executive). This suggests that Chinese participants were applying the same strategy for processing Chinese to processing English words. Tan and colleagues conclude that their study provides “the most compelling data in support of the hypothesis that language experience tunes the cortex” (Tan et al. Reference Tan, Spinks, Feng, Siok, Perfetti, Xiong, Fox and Gao2003, p. 164).
The present study
The present study was designed to investigate the distribution of cognitive resources shown by Chinese ESL learners and native English participants when recalling sequences of pictured objects, focusing specifically on the use of phonological and visuo-spatial encoding strategies. Serial recall of pictures has been successfully used with both adults and children in cognitive research, and is a particularly acute way of exploring how visuo-spatial representations differ from verbal representations in working memory (Yoon, Feinberg, Luo, Hedden, Gutchess, Chen, Mijels, Jiao & Park, Reference Yoon, Feinberg, Luo, Hedden, Gutchess, Chen, Mijels, Jiao and Park2004). The study employed the same procedures as Hitch et al. (Reference Hitch, Halliday, Schaafstal and Schraagen1988, Reference Hitch, Halliday, Dodd and Littler1989, Reference Hitch, Halliday, Schaafstal and Heffernan1991) used to study the development of the VSSP and phonological loop in young English children: pictures of everyday objects measured participants’ response to the PSE and VSE. Participants’ auditory memory (through recall of digits forward), visual memory (using the visual patterns test; Della Sala, Gray, Baddeley & Wilson, Reference Della Sala, Gray, Baddeley and Wilson1997) and articulation rate were also tested.
It was hypothesised that orthographic background would affect working memory performance within the two groups. English participants, from an alphabetic background, should demonstrate a stronger use of phonology when completing the task. They should show a PSE, but not a VSE, as visually similar items will remain phonologically distinct (Halliday et al., Reference Halliday, Hitch, Lennon and Pettipher1990; Hitch et al., Reference Hitch, Halliday, Schaafstal and Schraagen1988). Conversely, because Chinese ESL learners may transfer their logographic L1 word recognition processes to an alphabetic L2, it was expected that they would show a stronger reliance on visual information than native English speakers (Bialystok, Majumder & Martin, Reference Bialystok, Majumder and Martin2003). It was therefore hypothesised that they would not be subject to the PSE, but may be subject to the VSE. It was also possible that, due to the logographic nature of the language, Chinese participants would outperform English participants on the Visual Patterns Test, thus replicating Demetriou et al.'s (Reference Demetriou, Zhang, Spanoudis, Christou, Kyriakides and Platsidou2005) findings.
However, two alternative hypotheses must be considered. As the Chinese ESL participants are skilled users of English, they may have developed a separate mental lexicon for their second language, and, in addition, a ‘language switch mechanism’ which controls which lexicon is active (Van Wijnendaele & Brysbaert, Reference Van Wijnendaele and Brysbaert2002). If this is the case, performances of the English and Chinese participants should be comparable: both will be subject to the PSE, and unaffected by the VSE. This finding would replicate that of Chen (Reference Chen, Chen and Tzeng1992), who discovered that Chinese bilingual students’ viewing strategies differed depending on whether they were reading Chinese or English. This outcome would be consistent with the assumption that level of skill in L2 and not L1 is the critical factor (Lee & Chan, Reference Lee and Chan2000; Wang et al., Reference Wang, Koda and Perfetti2003). It could also arise from the fact that Chinese ESL participants’ experience with Pinyin or Zhu-Yin-Fu-Hao facilitates their use of English phonological information.
A third hypothesis is that Chinese participants do not have fixed picture–word recognition strategies like their English counterparts. Rather, they coordinate strategies from their logographic L1 and their alphabetic L2. In this case, the Chinese participants may rely on both alphabetic and logographic strategies when encoding the material. They should be subject to neither the PSE nor the VSE, as they do not prioritise one encoding strategy over another.
Method
Participants
Two groups of adult English speakers participated in the study: 14 native English speakers, and 14 native Chinese speakers. The mean age of the Chinese participants (2 male, 12 female) was 27.07 years (SD = 2.16; range = 24–32). The mean age of the English participants (3 male, 11 female) was 29.93 years (SD = 8.54; range = 23–55). Both groups were graduate students at the University of Bristol who participated voluntarily in the experiment; all had normal or corrected to normal vision.
Participants in the ESL group were chosen using strict criteria. As Wang et al. (Reference Wang, Koda and Perfetti2003) assert, homogeneity of L2 proficiency is an important consideration when conducting such studies: one must be sure that performance differences arise from participants’ orthographic background, and not from lack of skill in L2. All participants had a similar, high level of L2 language proficiency. Seven started to learn English in elementary school (6–11 years of age), while seven began formal schooling in English at the start of junior middle school (ages 12–15). Participants had studied English for an average of 15.29 years (SD = 2.33; range = 12–20). All had passed examinations allowing them to study for a Masters degree at a British university, which requires a minimum of International English Language Testing System (IELTS) Band 6.5 (described as a “competent user”). Seven participants met this criterion, and seven exceeded it (with IELTS scores of 7.0 or higher). Ten had spent one year in an English-speaking country at the time of testing, and four had spent two years.
In addition, only Mandarin speakers from Taiwan and mainland China took part in the study (seven were from mainland China and seven from Taiwan). Cantonese speakers from Hong Kong were excluded, as they would have learned to read Chinese via a logographic ‘look-and-say’ method rather than through a phonetic system (Holm & Dodd, Reference Holm and Dodd1996). This has been shown to affect their phonological awareness, both in Chinese and in an alphabetic L2 (Cheung, Reference Cheung1999; Leong, Reference Leong2002). Hong Kong Chinese also have a greater exposure to both oral and written English from an early age, as it is essentially a bilingual culture (Holm & Dodd, Reference Holm and Dodd1996; Leong & Hsia, Reference Leong and Hsia1996). This, too, could potentially confound results (see Jackson et al., Reference Jackson, Chen, Goldsberry, Kim and Vanderwerff1999).
Materials
Three sets of twelve five-, six- and seven-item sequences of line drawings of common objects were used in the experiment. The pictures were downloaded as freeware from the UCSD Center for Research in Language website. This particular set of Snodgrass & Vanderwart (Reference Snodgrass and Vanderwart1980) pictures were chosen as they have been used extensively in memory research (see Yoon et al., Reference Yoon, Feinberg, Luo, Hedden, Gutchess, Chen, Mijels, Jiao and Park2004 for a review), and have been standardised across different countries, including China and Taiwan (Bates et al., Reference Bates, D'Amico, Jacobsen, Szekely, Andonova, Devescovi, Herron, Lu, Pechmann, Pleh, Wicha, Federmeier, Gerdjikova, Gutierrez, Hung, Iyer, Kohnert, Mehotcheva, Orozco-Figueroa, Tzeng and Tzeng2003).
An initial survey of all 260 pictures identified and eliminated those pictures which could be confusing, or which involved complicated vocabulary (e.g. llama). This is because image and name agreement, as well as word frequency, have been shown to affect picture naming speed (Barry, Morrison & Ellis, Reference Barry, Morrison and Ellis1997). All words chosen were monosyllabic, as polysyllabic words (e.g. ladder, ladybird) could interfere with the variables being tested, due to the word length effect (see Baddeley, Thomson & Buchanan, Reference Baddeley, Thomson and Buchanan1975). Set 1 contained pictures of objects which are visually distinct, and which have names that are phonologically distinct: for example, “star”, “bridge”, “cheese”, “nest”, “rose”. Set 2 contained objects which are visually distinct, but which have names that are phonologically similar chosen for their similarity of internal vowel sound and, where possible, their beginning or end consonant sound: e.g. “bra”, “car”, “star”, “scarf”, “glass”. Such ‘rhyming lists’ are believed to be most influenced by the PSE (see Lian, Karlse & Eriksen, Reference Lian, Karlse and Eriksen2004). Visually similar items were chosen according to the criteria specified by Hitch et al., (Reference Hitch, Halliday, Schaafstal and Schraagen1988). Long and thin items – such as “nail”, “fork”, “gun”, “spade” and “broom” – were therefore placed together along the same axis, thus creating the third set of pictures. Lists were then piloted. Adult native English participants, as expected, showed significantly decreased performance on the phonologically similar condition, but not on the visually similar condition.
A standard forward digit span test (adapted from The Working Memory Test Battery for Children, Pickering & Gathercole, Reference Pickering and Gathercole2001), and the Visual Patterns Test (Della Sala et al., Reference Della Sala, Gray, Baddeley and Wilson1997) were used to assess verbal and visual working memory. Though the latter has been standardised for adult use, the former needed some adaptation; two randomised ten- and eleven-span sequences were therefore added. Articulation rate was calculated by asking participants to engage in standard Rapid Automatised Naming (RAN) – naming all pictured items as fast as possible in the order in which they were presented during the experiment – and recording the result (see Denckla & Rudel, Reference Denckla and Rudel1976). An overall articulation rate (in seconds per picture), as well as separate rates for each condition, were thus calculated.
Design
The experiment had a within-participants design. To eliminate practice effect or fatigue, trials on the main test were counterbalanced using a Latin square model of exposure.
Procedure
The study was carried out during two sessions, one week apart. In the first session, participants were shown all of the pictures to which they would be exposed in the eventual test, in alphabetical order, and asked to name items aloud in English. Any failures to give the designated name were corrected and retested, ensuring that all participants knew the correct name of each picture before commencing the experiment. A test of digit span and the Visual Patterns Test were also administered at this first sitting, using the procedures described above.
In the second session, participants were exposed to lists of items arranged according to their phonological and visual distinctiveness as described above. Participants first completed a practice set of items, and received feedback. Participants were then exposed to three sets of four five-, six- and seven-item picture lists for 7, 8, or 9 seconds, respectively. The usual formula of one second per item was not used, as pictured items have been found to take longer to name than written words (see Ferrand, Reference Ferrand1999). After exposure time, measured using a stopwatch, the list was covered up and participants wrote down their answers in a response booklet with five, six or seven boxes. There was no time limit for this, and participants were allowed to correct their responses. A two-minute break was provided between sets to prevent fatigue. Participants received a score of one for each individual item recalled correctly (i.e. according to its designated name: wig not hair, broom not brush, and so on) and in the order in which it was presented (an item named in third place, for instance, had to be recorded in the third box in the response booklet). Overall scores were thus calculated for each condition, with a maximum score of 72.
The final part of the second session involved participants naming the full list of pictures verbally, into a tape recorder, in the order in which they were exposed, as fast as possible. Participants were then asked to fill in a questionnaire, indicating their personal details (age, gender, country of origin, etc), language experience, and their understanding of how information was encoded during the test.
Results
Serial recall of pictured objects
Scores for all participants were collated for the three experimental conditions, and for overall performance. A mean and standard deviation were calculated for each of these variables (see Table 1).
a Phonologically and visually distinct.
b Phonologically similar.
c Visually similar.
A mixed (3 × 2) ANOVA was conducted to assess whether there were differences between scores in participants’ performance across conditions according to first language. (The following assumptions were tested: (a) independence of observations, (b) normality, and (c) sphericity. All assumptions were met.) Results indicated a significant effect for condition, F(2,52) = 24.07, p < .01, partial eta2 = .48, but not for language, F(1,26) = .78, p = .39, partial eta2 = .029. However, the main effect for condition was qualified by a significant interaction between condition and first language: F(2,52) = 25.78, p < .01, partial eta2 = .498. This suggested that there were differences between the two cohorts’ performance across different conditions.
Post-hoc paired-samples t-tests of English and Chinese participants’ results confirmed that there were indeed differences within cohorts across conditions. Using Bonferroni adjustment for multiple comparisons, the alpha level per comparison was set at .008 (.05/6). An exploration of the English participants’ results showed that there was a significant difference between performance on the distinct and phonologically similar condition (t = 10.67, df = 13, p < .008, two-tailed), and on the visually similar and phonologically similar conditions (t = −9.33, df = 13, p < .008, two-tailed). In both cases, mean scores were significantly lower on the phonologically similar condition, thus showing evidence of the phonological similarity effect. There were no significant performance differences between the distinct and visually similar conditions: t = 1.20, df = 13, p = .25 (two-tailed), showing no evidence of a visual similarity effect. The Chinese participants, however, showed no significant differences in mean scores across all three conditions (see Table 1): the distinct and phonologically similar conditions (t = .22, df = 13, p = .83, two-tailed); the phonologically and visually similar conditions (t = .50, df = 13, p = .63, two-tailed); and the distinct and visually similar conditions (t = .61, df = 13, p = .55, two-tailed). They did not appear to show evidence of either the phonological or the visual similarity effect.
In addition, post-hoc independent-scores t-tests were carried out to investigate performance differences between the groups. There was no significant difference in the total performance between the two groups: t = .88, df = 26, p = .39 (two-tailed, equal variances assumed). There were, however, significant differences in performance under the different experimental conditions. On the distinct condition, t = 2.38, df = 26, p < .05 (two-tailed, equal variances assumed), and on the visually similar condition, t = 2.22, df = 26, p < .05 (two-tailed, equal variances assumed). In both conditions, mean scores were higher for the English participants. Though mean scores were higher for Chinese participants on the phonologically similar condition, this difference failed to reach significance: t = −1.91, df. = 26, p = .07 (two-tailed, equal variances assumed).
Articulation rate
Articulation rates (in seconds per item) for all participants were collated in each of the three experimental conditions. For each experimental condition, a mean and standard deviation were calculated. These descriptive statistics are shown in Table 2.
a Phonologically and visually distinct
b Phonologically similar
c Visually similar
A mixed (3 × 2) ANOVA was conducted to assess whether there were differences in articulation rates across languages and conditions. (The following assumptions were tested: (a) independence of observations, (b) normality, and (c) sphericity; all assumptions were met.) Results indicated a significant effect for condition, F(2,52) = 32.09, p < .01, partial eta2 = .552, but not for language, F(1,26) = 1.84, p = .19, partial eta2 = .066. Post-hoc multiple-comparison analysis revealed significant differences between articulation rate for the distinct and phonologically similar items (p < .001), and for the distinct and visually similar items (p < .001); no significant differences were found between phonologically and visually similar items (p = .12). There was no significant interaction effect between condition and first language: F(2,52) = .36, p = .50, partial eta2 = .017. This suggests that the articulation rate for both English and Chinese participants was significantly slower for phonologically and visually similar items than for distinct items.
Digit span and visual pattern span
Scores for all participants were collated for the two tests of verbal and visual memory. For each test, a mean and standard deviation were calculated (see Table 3).
A visual inspection of the raw data revealed some small differences in performance on both tests for Chinese ESL and native English participants. Independent-scores t-tests were therefore carried out to investigate performance differences between the groups. No significant differences were found. For the test of digit span, t = 1.88, df = 18.20, p = .77. (two-tailed; equal variances not assumed). For the visual patterns test, t = –.22, df = 26, p = .83 (two-tailed; equal variances assumed).
Discussion
As hypothesised, the English participants, from an alphabetic background, appeared to use phonological strategies when encoding the picture sequences. They performed significantly better not only on the distinct condition, but also on the visually similar condition, than on the phonologically similar condition. They were thus subject to the PSE, but not to the VSE, as visually similar items were rendered distinct through phonological coding. This result confirms in adults the findings of Hitch et al. (Reference Hitch, Halliday, Schaafstal and Schraagen1988), Halliday et al. (Reference Halliday, Hitch, Lennon and Pettipher1990) and others, which suggest that individuals literate in an alphabetic orthography prioritise phonological over visuo-spatial strategies when encoding visually-presented material.
The findings for the Chinese participants, however, were markedly different. Chinese participants were not subject to the PSE or to the VSE. In fact, their performance was remarkably stable across all three conditions, showing almost identical mean scores on the visually/phonologically distinct and phonologically similar conditions, and a drop of only one item on the visually similar condition. The performance differences between English and Chinese participants were also revealing. Though there was no overall performance difference between English and Chinese participants, the two groups did perform differently on the three conditions. English participants significantly outperformed Chinese participants on the distinct and visually similar conditions. Chinese participants outperformed their English counterparts in the phonologically similar condition, though this difference failed to reach significance.
The results show that articulation rate for both Chinese and English participants was longer for the phonologically and visually similar items than for distinct items. There has been some conflicting research on the effect of phonological similarity on articulation rate. Some researchers have argued that it provides a facilitative effect on word recognition and naming (e.g. Vitevitch, Reference Vitevitch2002), whilst others, that phonologically-related words produce slower and less accurate speech production (Schacter, Reference Schacter1999; Sevald & Dell, Reference Sevald and Dell1994). The finding of this experiment supports this latter hypothesis that phonologically similar words compete during speech production. Recorded articulation for phonologically similar sequences was more hesitant, and less accurate (more ‘misnaming’ took place – brush for broom, for example). It is interesting to note that visually similar items also took longer to name than visually distinct items. This may be because visual similarity confuses lexicalisation – the process of finding the right name in the mental lexicon which then makes phonological form available (Ferrand, Reference Ferrand1999). Lexicalisation speed is partly determined by picture discriminability; it may therefore be slowed by the visual similarity of the surrounding stimuli, which compete with the image within the mental lexicon (Barry et al., Reference Barry, Morrison and Ellis1997).
Articulation rate was comparable for both groups of participants. Though the mean articulation rate in seconds per word was longer for Chinese participants under all three conditions, this difference failed to reach significance. This suggests that the poorer performance of Chinese participants in conditions 1 and 3 was not a result of slower picture naming (see Jackson et al., Reference Jackson, Chen, Goldsberry, Kim and Vanderwerff1999).
There was no significant difference between English and Chinese participants on the digit span test. However, it should be noted that verbal memory – though in some ways necessary as a complementary tool to the visual memory test – is difficult to measure in a standardised, culture-fair way. Cross-language measures of digit span have been the subject of much controversy, as some languages – such as Chinese – have digits with a much shorter pronunciation duration than English (Hedden, Park, Nisbett, Ji, Jing & Jiao, Reference Hedden, Park, Nisbett, Ji, Jing and Jiao2002; Li, Liu & Li Reference Li, Liu and Li2003). Measuring digit span in the participants’ shared language, in this case English, also poses problems. Chinese ESL participants were remembering not only the digits, but how that digit is signified in English; their load was therefore potentially greater than that of the English participants. This suggests a need, articulated by Hedden et al. (Reference Hedden, Park, Nisbett, Ji, Jing and Jiao2002), for established culture-invariant measures for the assessment of verbal working memory.
There is conflicting research concerning visual memory in comparative studies of English and Chinese participants. Some studies have found Chinese adults to be strong in tests of visuo-spatial perception and memory (e.g. Demetriou et al., Reference Demetriou, Zhang, Spanoudis, Christou, Kyriakides and Platsidou2005; Flaherty, Reference Flaherty2003; Tavassoli, Reference Tavassoli2002). Others have found no difference between Chinese and English participants (Hedden et al., Reference Hedden, Park, Nisbett, Ji, Jing and Jiao2002) – a finding replicated here. However, there were some notable differences in the way that both groups completed the Visual Patterns test. English participants tended to mark boxes using a cross or a tick; Chinese participants, particularly when representing the more complex patterns, tended to use a set of lines reminiscent of a Chinese character. This suggests that Chinese participants were trying to remember the pattern using the skills acquired in their native orthography, seeing each visual ‘chunk’ as representing a single stroke. Why, then, were they not more able when completing the task? One plausible explanation is that the patterns themselves did not resemble the typical configuration of Chinese characters. Chen and Kao (Reference Chen and Kao2002) considered the visuo-spatial properties inherent in Chinese characters, looking specifically at the effect of linearity and symmetry. They found that characters with ‘high linearity’ were recognised significantly more quickly than those with fewer linear strokes; symmetry, too, was found to have a facilitative effect on character recognition. Interestingly, few of the patterns on the Visual Patterns Test are perfectly symmetrical, and many have extensive non-linear orientation. Therefore the test may not measure the particular visuo-spatial skill in which Chinese participants excel.
Of the three hypotheses regarding Chinese performance on the picture-naming test, the first two are contradicted by the data. If, as the first hypothesis stated, Chinese participants were using the visuo-spatial strategies inherited from their logographic L1, they would have been subject to the VSE. However, most Chinese participants performed as well in this condition as in the others. The second hypothesis must also be discarded. If, as fluent English speakers, Chinese participants were using solely phonological strategies, more appropriate to their alphabetic L2, they would have been subject to the PSE, like the native English participants (see Chen, Reference Chen, Chen and Tzeng1992). Again, this was not the case. It seems, therefore, that the results support those expected from the third hypothesis. What can explain the fact that Chinese participants were subject to neither the phonological nor the visual similarity effect?
One way to answer this question is to query the design of the experiment: did the second experimental condition really test phonological similarity for Chinese ESL participants? There are two ways in which Chinese participants may have confounded this condition. The first of these is by encoding their answers in Mandarin – where, of course, the pictured items may not be phonologically similar – and then translating this into English for transcription. However, this did not seem to be happening in this experiment: in the questionnaire, all Chinese participants reported that they encoded the items in English. A second potential confound regards the Chinese participants’ pronunciation of English (British versus American), which may have rendered items more phonologically distinct. Although this was the case for two of the phonologically similar items (glass /ɡˡɑːˢ/, occasionally Americanised as /ɡæˡˢˢ/, and comb /kǝum/, pronounced /koum/), pronunciation of sequences was largely regular across both groups. Furthermore, several of the English-speaking participants – two from Northern England, one from Ireland and one from America – also pronounced some items (glass, bra, vase) in a distinct manner. However, all four participants were subject to the PSE.
Self-report data from the questionnaire provide further support for hypothesis 3: that Chinese participants were using both phonological (remembering the name of the item) and visuo-spatial strategies (remembering what the item looked like), either intermittently or simultaneously. Whilst English participants reported using phonological strategies, all Chinese participants reported using both visual and phonological strategies for remembering the sequences. Visual strategies reported included remembering the ‘visual distinctiveness’ of items (features such as axis, shape and orientation), making a ‘picture story’ with items, and ‘visualising’ or ‘reading’ items in their mind. One participant reported remembering the first half of a sequence in a phonological code, and the second half visually. Two others said that they accessed the phonological form of the items through their visual form – a technique reminiscent of the way Chinese characters are read (Perfetti et al., Reference Perfetti, Liu and Tan2002). Several reported that they used phonological strategies on the visually similar items, and visual strategies on those which were phonologically similar, thereby optimising their performance in both conditions.
Different memory strategies – phonological for the English participants, and dual-purpose for Chinese ESL participants – were also evidenced by the participants’ behaviour during the test. Chinese participants could only be seen engaging in phonological rehearsal during the visually similar condition. English participants, in contrast, engaged in subvocal rehearsal during all three experimental conditions. Furthermore, English participants filled in their answers quickly, and reported rapid decay if items were not written down immediately. Incorrect answers were often phonologically related to the stimuli: cheese instead of skis, for example. However, for Chinese participants, incorrect answers were either phonologically related to the stimuli, or visually/generically related: toe instead of thumb, for instance. They completed the sequences carefully, and often took their time over answers; some were retrieved as long as 50 seconds after exposure. This suggests that items were retained in memory for a longer period of time – perhaps because more than one strategy was used to maintain the items’ distinctiveness in working memory.
Research suggests three potential explanations for the utilisation of a dual encoding strategy by Chinese participants. The first is that fluency in two diverse orthographies – Chinese and English – allows a multi-strategy approach to visual recognition to develop. The visuo-spatial emphasis comes from practice in reading the logographic Chinese script (Tan et al., Reference Tan, Spinks, Gao, Liu, Perfetti, Xiong, Pu, Liu, Stofer and Fox2000, Reference Tan, Liu, Perfetti, Spinks, Fox and Gao2001) and the phonological activation, from fluency in English (Goswami, Reference Goswami2000). Readers fluent in both scripts thus have more than one set of cognitive resources available for the task. This takes Chen's (Reference Chen, Chen and Tzeng1992) finding – that Chinese bilingual students’ viewing strategies differ depending on which language they are reading – one step further, suggesting that fluent Chinese ESL readers may also coordinate viewing strategies from their logographic L1 and alphabetic L2.
This ‘holistic’ view of bilingualism challenges ‘fractional’ theories, such as that proposed by Forster and Jiang (Reference Forster, Jiang and Nicol2001). They posit that a bilingual is “the equivalent of two monolingual word recognizers living within the same person” (Brysbeart, Reference Brysbaert2003, p. 324); there is, they believe, a functional, cerebral division between L1 and L2 lexicons, with only the L1 lexicon inhabiting the ‘language module’ of the brain. This study, however, suggests that L1 and L2 have interactive, reciprocal effects. Support for this ‘holistic’ view is provided by a variety of masked priming studies, which suggest that the phonological and orthographic representations of both languages are activated in parallel, and to interact, even when there is an inconsistency between the letter–sound correspondence in L1 and L2 (see Brysbeart, Reference Brysbaert2003; Davis, Kim & Sanchez-Casas, Reference Davis, Kim and Sanchez-Casas2003).
A second explanation for the use of a dual encoding strategy focuses not on the interaction between L1 and L2, but on the participants’ early educational experiences. It is possible that exposure to alphabetic scripts – Pinyin or Zhu-Yin-Fu-Hao – alongside Chinese characters in primary school may have facilitated their phonological processing skills, developed in conjunction with visuo-spatial processing. Indeed, research suggests that this does provide a phonological advantage when reading English: mainland ESL Chinese and Taiwanese children perform significantly better on phonological awareness tests in English than their Hong Kong counterparts (Holm & Dodd, Reference Holm and Dodd1996; Huang & Hanley, Reference Huang and Hanley1997; Wang & Geva, Reference Wang and Geva2003). Other studies have noted that young Chinese children use Pinyin to help them spell unfamiliar English words (Ding & Peng, Reference Ding and Peng1998). It is therefore possible that Chinese and Taiwanese participants have educational tuning which allows them to draw on both phonological and visuo-spatial resources during the picture-recognition task.
The third possible factor contributing to the dual encoding strategy concerns the configuration of the Chinese character, and the way that it is recognised and encoded. Chinese participants are used to coordinating pictographic and phonological cues during reading, as the typical Chinese character consists of a semantic and a phonetic radical, both of which may be necessary for word recognition (Lee et al., Reference Lee, Tsai, Huang, Hung and Tzeng2006). Thus, some researchers claim that Chinese is not purely a ‘logographic’ script, but might additionally be considered morphemic or morphosyllabic (Honorof & Feldman, Reference Honorof and Feldman2006; Perfetti & Liu, Reference Perfetti and Liu2006). Support for this is provided by neuroimaging studies, which suggest that fluent Chinese readers have a brain area specialised for this very function: the left middle frontal gyrus, analogous in function to the central executive (Kuo et al., Reference Kuo, Yeh, Duann, Wu, Ho, Hung, Tzeng and Hseih2001; Tan et al., Reference Tan, Liu, Perfetti, Spinks, Fox and Gao2001). They may therefore be used to the kind of cognitive coordination required of this particular task, whilst English participants are programmed to code visually-presented material in a phonological form.
A study by Chen and Tsoi (Reference Chen and Tsoi1990) provides an experimental example of these different approaches to encoding visual stimuli. They used a Stroop-like naming task to compare recognition of the four arithmetic operations (e.g. + and –), which operate as logographic symbols in both Chinese and English, under conditions of interference between symbols and words. They found that Chinese participants were equally fast in naming calculation symbols with Chinese characters, whilst English participants were faster in reading the words than naming the symbols. They concluded that Chinese readers process characters and other logographic symbols in a similar manner, coordinating visual and phonological cognitive resources and thus enabling rapid identification regardless of stimulus type. Readers of English, however, process pictures and logographic symbols in one way and words in another, the former more slowly than the latter (Demetriou et al., Reference Demetriou, Zhang, Spanoudis, Christou, Kyriakides and Platsidou2005). During a picture-naming exercise such as that seen in the present experiment, it is possible that similar processes were at work. Chinese participants were coordinating the phonological and visual information provided by the pictures, thus performing equally well across all conditions; English participants, however, struggled to coordinate cognitive resources. They were more likely to rely on phonological coding in working memory, a strategy which proved more or less effective according to conditions.
To discriminate between these diverse explanations, it is recommended that further studies fine-tune this experiment in order to work out exactly what is causing the noted pattern of results amongst the Chinese ESL participants. One suggestion involves replicating the central picture–word recognition test using three different groups of participants. The first group would consist of high-proficiency ESL speakers from another alphabetic orthography (e.g. Spanish). This would establish whether the results seen in this experiment are unique to Mandarin–English coordinate bilinguals, or whether this is a pattern more generally exhibited by other coordinate bilinguals. High-proficiency Cantonese ESL speakers would make up the second group; this experiment would investigate whether educational experience has encouraged phonological encoding strategies, as hypothesised. The final group would consist of Mandarin–English bilinguals (i.e. those who learned both languages together in the same context: Reber & Reber, Reference Reber and Reber2001, p. 88), and would examine whether this dual-encoding strategy is unique to those who learned an alphabetic orthography after a logographic one, or whether these encoding strategies can be developed in coordination.
Conclusions
This study investigated the different encoding strategies used by native English and Chinese ESL participants when engaging in immediate serial recall of pictures of familiar objects. It found that, whilst English participants were using predominantly phonological encoding strategies – evidenced by the fact that they were subject to the phonological but not to the visual similarity effect – Chinese ESL participants seemed to be using both visuo-spatial and phonological strategies, performing comparably across all three experimental conditions. Though this did not provide an overall performance advantage, it did provide some advantage on the phonologically similar condition, where Chinese participants outperformed their English counterparts.
It was suggested that these different encoding strategies may arise from the L1 background of the two groups (see Chen & Juola, Reference Chen and Juola1982). Reading in an alphabetic orthography such as English is dependent on the development of phonological awareness (Goswami, Reference Goswami2000). Previous studies have shown that this dependence on phonological strategies in reading also extends to visuo-spatial tasks, like that seen in this present study, in which literate individuals rely more exclusively on phonological skills than those who are non-literate (e.g. Hitch et al., Reference Hitch, Halliday, Schaafstal and Schraagen1988). Reading in a logographic or morphosyllabic orthography such as Chinese, however, is dependent on a combination of visuo-spatial and phonological strategies (e.g. Chee, Soon & Lee, Reference Chee, Soon and Lee2003). It is therefore possible that Chinese ESL readers may transfer these skills to picture recognition in their alphabetic L2, and are thus more able to use both visual and phonological information when encoding the sequences than those exposed to an alphabetic orthography. Educational background, particularly experience of Pinyin or Zhu-Yin-Fu-Hao, and the semantic–phonetic configuration of the Chinese character itself, may also facilitate this dual-coding strategy. More experimental research is needed to investigate these diverse hypotheses, and to further discriminate the memory processes at work considered in this study.
