When listening to the message spoken by a communication partner, successful comprehension depends on many factors, including the storage, manipulation, and retention of large quantities of information in working memory. The burden on memory, however, typically is eased by the hierarchical, syntactic organization of linguistic units (i.e., words combine into phrases, and phrases combine into sentences) with highly conventionalized, and therefore predictable, ordering patterns. Past work has demonstrated that short-term and working memory for sequences of words benefit from higher rather than lower order approximations to naturally occurring English word sequences (for native English speakers; e.g., Miller & Selfridge, Reference Miller and Selfridge1950) and ultimately for syntactically correct sentences rather than for random strings of words (Brener, Reference Brener1940; Cattell, 1886 in Scheerer, Reference Scheerer1981; Marks & Miller, Reference Marks and Miller1964; Miller & Isard, Reference Miller and Isard1963). This latter effect is often termed the “sentence superiority effect” (SSE; e.g., Bonhage, Fiebach, Bahlmann, & Mueller, Reference Bonhage, Fiebach, Bahlmann and Mueller2014; Scheerer, Reference Scheerer1981) and is theorized to be a result of linguistic “chunking,” or relational binding of items, according to syntactic and grammatical rules (Epstein, Reference Epstein1961, Reference Epstein1962; Marks & Miller, Reference Marks and Miller1964), and also according to sentence-level semantic meaning (Bonhage et al., Reference Bonhage, Fiebach, Bahlmann and Mueller2014; Potter & Lombardi, Reference Potter and Lombardi1998). Recent evidence suggests that linguistic chunking occurs automatically during the encoding stage of processing spoken input, both outside of conscious awareness (Baddeley, Hitch, & Allen, Reference Baddeley, Hitch and Allen2009) and regardless of task demands (e.g., whether the materials are actively memorized or not; Bonhage, Meyer, Gruber, Friederici, & Mueller, Reference Bonhage, Meyer, Gruber, Friederici and Mueller2017).
For the recall of longer strings of linguistic materials, the findings from several studies suggest that ongoing discourse is processed and organized as clauses and sentences in memory, and that as each clause is processed for meaning, recall for the precise words therein decreases substantially (e.g., Bever, Lackner, & Kir, Reference Bever, Lackner and Kir1969; Marslen-Wilson & Tyler, Reference Marslen-Wilson and Tyler1976; Jarvella, Reference Jarvella and Bower1979; Sachs, Reference Sachs1967). Jarvella presented long narrative passages of speech interrupted by experimenter-imposed pauses. After each pause, the participants were asked to recall verbatim a certain number of words preceding the pause (Jarvella, Reference Jarvella1970) or as many words preceding the pause as possible (Jarvella, Reference Jarvella1971; Jarvella & Herman, Reference Jarvella and Herman1972). This work indicated that there exist sharp gradients in recall accuracy across clause and sentence boundaries for running speech. Specifically, the sentences immediately preceding the test pause are recalled better than earlier sentences (Jarvella & Herman, Reference Jarvella and Herman1972). Within the immediately preceding sentences, recall is better for words forming a full clause than clause fragments, and overall recall of immediate clauses is superior to recall of previously occurring clauses (Jarvella, Reference Jarvella1970, Reference Jarvella1971; Jarvella & Herman, Reference Jarvella and Herman1972). The superior recall for more recent clauses (termed the “clause recency effect”) can be influenced by several factors. For example, the size of the clause recency effect can be reduced—that is, the recall of earlier clauses may be improved—by increasing the contextual relatedness among the clauses (Jarvella, Reference Jarvella1971, Reference Jarvella1973). The prosody and pauses of the speaker also have been shown to influence the gradients in recall across clause boundaries within sentences, although clause recency effects still may be found with visually presented linguistic material (see Jarvella, Reference Jarvella and Bower1979).
There is evidence that clause- and sentence-based structure in memory for connected speech remains even while forgetting occurs for the specific words comprising the clauses and sentences. In support of this point, some studies have shown that, as clauses and sentences are forgotten, their word-by-word recall functions assume a bowed or U shape, with better recall observed at the beginning and end of the clauses (Jarvella, Reference Jarvella1971, Reference Jarvella and Bower1979; Marslen-Wilson & Tyler, Reference Marslen-Wilson and Tyler1976). In these studies using test materials composed of multiple clauses, there exist multiple connected U shapes corresponding to each clause. In effect, these recall-function shapes may be interpreted as strong evidence for clause-based organization of words in memory. It is important to note that these enhancements of recall for speech at clause boundaries relative to the surrounding word positions have only been observed for open-set discourse materials with natural prosody and/or semantic context extending over multiple sentences. This begs the question as to whether and to what degree conformance to syntactic structure in the absence of natural prosody and semantic context yields evidence of clause-level organization and aids in the recall of auditory materials in general. That is, does the conformance to an expected order of word category lead to clause-level organization—and generally improve storage and recall of the constituent words—compared to conditions without this structure?
Previous work has indicated that linguistic chunking according to grammar and syntax occurs automatically during the encoding of speech in memory (Baddeley et al., Reference Baddeley, Hitch and Allen2009; Bonhage et al., Reference Bonhage, Meyer, Gruber, Friederici and Mueller2017). Baddeley et al. (Reference Baddeley, Hitch and Allen2009) reported that the size of the SSE remained the same regardless of whether subjects performed concurrent tasks thought to disrupt executive processes during the recall tasks. This was taken as evidence that the SSE, or formulation of linguistic chunks, does not depend on a “central executive.” Automatic linguistic chunking during encoding is presumably also responsible for clause-based organization in memory (manifesting as peaks at clause boundaries in recall curves). However, Baddeley et al. did not evaluate evidence of clause-based organization with and without concurrent tasks.
The present study extended this work to examine how resistant the SSE and apparent clause-based organization is to mental manipulation of the materials themselves. Furthermore, the study examined whether mental manipulation could yield the SSE and apparent clause organization. Specifically, we devised a test condition in which syntactically correct sequences (aiding in chunking) were not apparent at stimulus presentation, but became syntactic sequences after subjects performed a mental reversing of the materials to make a response. Syntactic sequences were presented in backwards order, and participants recalled these materials either in the same order they were presented or in reverse order (yielding a syntactic sequence). If the subject’s response conformed to syntactically correct linguistic clauses but the presented material did not, would the enhancements for recall at clause boundaries be apparent and might there still exist general recall performance benefits due to linguistic structure (i.e., a SSE)?
This study was designed to address two main questions. First, we asked whether there was evidence of clause-based organization in memory under conditions where normal prosodic and semantic context cues were weak or absent. The goal was to learn whether relying primarily on syntactic relations among word strings would be sufficient to produce benefits for the recall of speech, either for enhancements to performance at clause boundaries or for overall sentence recall performance (SSE). In order to achieve the degree of stimulus control necessary to support the empirical observations, a closed-set speech corpus was employed in a word identification task. This approach was chosen in preference to other approaches found in the literature (Jarvella, Reference Jarvella1970, Reference Jarvella1971, Reference Jarvella1973; Jarvella & Herman, Reference Jarvella and Herman1972; Marslen-Wilson & Tyler, Reference Marslen-Wilson and Tyler1976) that assessed the recall of open-set materials presented as discourse which, as remarked on above, typically have natural prosody and considerable semantic context spanning clauses and sentences. The corpus used here consisted of words arranged in six syntactic categories: name (i.e., subject), verb, number, adjective, object, and conjunction with eight tokens in each category (see Hagerman, Reference Hagerman1982; Kidd, Best, & Mason, Reference Kidd, Best and Mason2008; Wagener, Kühnel, & Kollmeier, Reference Wagener, Kühnel and Kollmeier1999, for similar closed-set identification tests). All of the words were spoken by young adult talkers uttering each word in isolation with neutral inflection so as to avoid the phrase-level intonation effects that have been shown to induce chunking (see Simpson & del Prado Martín, Reference Simpson and del Prado Martín2015) and to affect segment organization (Jarvella, Reference Jarvella and Bower1979). With these materials, sentences/clauses were constructed by making random word selections from the name, verb, number, adjective, and object categories in order with items from the conjunction category used to link five-word clauses. Our impressions of these materials are consistent with the conclusions of Baddeley et al. (Reference Baddeley, Hitch and Allen2009), who observed that closed-set materials are less reliant on long-term memory and semantic coding than are open-set sentences. The result of this approach is to yield syntactically correct sentences/clauses with no phrase-level intonation and low semantic value. The low semantic value of the materials is due to the low predictability of the items in sequence (i.e., although the category of the next word in a sequence is known, the precise word within this category cannot be predicted from the meaning of the preceding words). In addition, the clauses formed by the materials constructed in this way are not dependent on, or linguistically related to, preceding or following clauses. That is, these items exhibit low and controlled semantic predictability (e.g., each within-category selection has a probability of occurrence of 1/8) for words within clauses, and low predictability across clauses, while at the same time exhibiting high syntactic (word class/category) predictability. Thus, our underlying assumption was that any evidence for clause-based memory organization may be attributed more directly to the unique contribution of syntactic structure than would be possible using traditional approaches. In Experiment 1, participants listened to sequences of words that were either randomly drawn from the full corpus (nonsyntactic) or drawn according to a constant syntactic structure (syntactic) and were asked to recall the words in the order in which they were presented. The effect of syntax (nonsyntactic vs. syntactic) was examined for a range of word sequence lengths (4 to 14 words). A range of sequence lengths was tested as part of this exploratory experiment given that it was not clear at the outset how listeners would perform in these conditions with these materials.
For the second main question, we asked whether the overall benefit of syntactic structure for recall (SSE) and the benefits of syntactic structure for recall at clause boundaries (clause-based organization) would be maintained with mental manipulation of the materials (i.e., revising the order of stored lexical items to make a response), and whether these effects would arise when mental manipulation led to standard syntax. In Experiment 2, the subjects heard the materials (described above) played in three presentation orders: syntactic presented forwards, syntactic but presented backwards, and nonsyntactic (random). The sequences were always 11 words long, which corresponded to two full clauses in syntactic (forwards or backwards) conditions. For each of these presentation conditions, the subjects were asked to recall the sequences presented either in the same order heard (i.e., first-in, first-out) or in reverse order (i.e., last-in, first-out). With these conditions, it was possible to compare cases in which syntactic ordering was apparent during presentation and response (i.e., syntactic stimuli with in-order response) to those where the presentation was syntactic but mental manipulation of materials yielded nonsyntactic sequences (i.e., syntactic stimuli with reverse-order response), and to compare those with the case where the presented material was not syntactic but became syntactic with the mental manipulation required to make a response (i.e., syntactic presented backwards with reverse-order response). Thus, this design allowed us to evaluate whether the benefits of syntactic structure on the recall of speech (clause-based organization and the SSE) are vulnerable to or emerge with mental manipulations of the materials.
Experiment 1
Method
Subjects
Ten young adults (Mean age = 21.5; SD = 2.16 years) who learned English from birth and who consider English their primary language participated in the study. All listeners had normal hearing (thresholds <20 dB HL for audiometric frequencies from 250 to 8000 Hz) and no history of otologic disorders.
Stimuli
The corpus of words (shown in Table 1 arranged by category) was recorded from eight female talkers. Each word was recorded in isolation and spoken with neutral inflection. Four conjunction words were used (“and,” “but,” “then,” and “while”).Footnote 1
The word corpus used in Experiments 1, 2, and 2a. The corpus is organized into six categories: names, verbs, numbers, adjectives, objects, and conjunctions. In Experiment 1, only four words in the conjunction category were used, whereas in Experiments 2 and 2a, all eight words in the conjunction category were used
Procedures
Listeners were seated in a double-walled sound-attenuating booth in front of a computer monitor. Auditory stimuli were processed through MATLAB 2011b (Mathworks Inc., Natick, MA) software at a sampling rate of 44100 Hz, delivered to a RME HDSP 9632 24-bit soundcard (ASIO) and presented through Sennheiser HD 280 Pro circumaural headphones. All words were presented at 60 dB SPL.
Words were presented in random, nonsyntactic order (words in each serial position drawn at random from the full corpus without replacement) or in syntactic order (words in each position drawn at random from each column in order without replacement). Sequence lengths of 4, 6, 8, 10, 12, and 14 words were tested. In the syntactic condition, for sequence lengths longer than 6 words, the columns were wrapped in order (e.g., the 7th word position drew from the 1st column again, the 8th word position drew from the 2nd column again, etc.). In the random condition, there was the possibility of drawing unintentionally syntactic sequences. To verify that this was an uncommon occurrence, a post hoc analysis of all word draws in the random condition was performed for all participant data. In this analysis, a syntactic sequence of words was defined as a string of words drawn from adjacent columns in the order of Table 1 (as in the syntactic condition). The majority of words presented in the random condition were not part of a grammatical sequence as defined (78%), whereas 17.9% of words were part of a grammatical pair, and 3.3% of words were part of a grammatical triplet. Less than 1% of words presented (corresponding to 19 total presentations across all subjects and trials) were part of a grammatical quadruplet. There were no instances of a grammatical quintuplet.Footnote 2
Each condition (Sequence Length × Syntax) was presented in 10-trial blocks twice in random order, yielding 20 trials per condition. For each subject, each trial of each condition consisted of a new draw of words. In each trial, after hearing each sequence, subjects were presented with one correct-category column at a time from which to select each word via mouse click (resulting in a 1/8 chance level for all word categories except for the conjunction category, which was 1/4). For example, if subjects were presented with “shoes two Sam red” in the random condition, during the response portion they were presented with the object, then number, then name, and then adjective categories. Each new category of words appeared after each response. Thus, subjects were restricted in the order in which they could respond. The first column of words for responding was presented immediately following presentation of the final word of the sequence. Prior to the experiment, participants were shown the full grid of words used in the experiment (much like Table 1, but with the conjunction column only showing the four words used) and then were given 10 practice trials responding to five-word sequences in the syntactic and random conditions (5 trials each).
Results for Experiment 1
The proportion of words correct for each condition, broken down by serial position in the sequence, is shown in the top row of Figure 1. The bottom row of Figure 1 shows the difference between syntactic and random conditions as a function of word position for each sequence length. To test for effects of syntax, generalized linear mixed models were fitted to the data for each sequence length separately, in each case specifying subject as a random effect and specifying syntax and word position as fixed effects. The analysis was conducted using the R statistical software, and the glmer function in the lme4 package was used. Post hoc comparisons of syntactic versus random at different word positions were done using the emmeans function in the R statistical software, and multiple comparisons were adjusted for using Tukey’s method. Asterisks in the bottom row of Figure 1 indicate word positions at which the syntactic condition performance was significantly greater than random performance. The effect of syntax condition itself was not significant for any sequence length: for sequence lengths of 6 words (Coefficient = –.23, SE = .68, z value = –.34, p = .73), 8 words (Coefficient = .48, SE = .50, z value = .96, p = .33), 10 words (Coefficient = .66 SE = .39, z value = 1.68, p = .09), 12 words (Coefficient = –.40, SE = .34, z value = –1.18, p = .24), or 14 words (Coefficient = –.36, SE = .27, z value= –1.34, p = .18). However, as indicated in the bottom row of Figure 1, there were clear benefits of syntax at various word positions.
Top row: Average proportion correct recall broken down by word position for syntactic and random conditions for each sequence length (different panels). The end of clauses in the syntactic condition are indicated by dashed vertical lines. Bottom row: Average benefit of the syntactic condition relative to the random condition at each word position (random subtracted from syntactic performance). Asterisks indicate word positions at which the syntactic condition yielded significantly better performance than random.
One of the purposes of Experiment 1 was to test for evidence of clause-based organization in the syntactic conditions with these materials, manifesting as improvements in recall at clause boundaries. As is apparent in Figure 1, in the random condition, for sequences longer than 4 words, there is evidence of primacy and recency effects, which have been displayed in several previous studies of recall of nonsyntactic materials (e.g., Baddeley et al., Reference Baddeley, Hitch and Allen2009; Deese & Kaufman, Reference Deese and Kaufman1957; Murdoch, Reference Murdoch1962). In the syntactic condition, for sequences containing at least one full clause, there is evidence of primacy and recency more broadly, but there is also a midsequence peak corresponding to the end of the first clause. Furthermore, in the bottom row, there is evidence of sharp increases in syntactic benefit at clause boundaries. To test for clause organization statistically, the data for sequences containing at least one full clause (sequence lengths of 6, 8, 10, 12, and 14 words) were analyzed for the presence of a U shape corresponding to the first clause. The second clause was not evaluated for the U shape given that it was potentially confounded with recency effects. The U shape was defined as significantly better performance at word positions 1 and 5 than at word position 3 (the middle of the first clause). These post hoc comparisons were done using the emmeans function in the R statistical software, and multiple comparisons were adjusted for using Tukey’s method. In the syntactic condition, word positions 1 and 5 were not greater than word 3 for sequence lengths of 6 words or 8 words, but they were statistically significantly greater for sequence lengths of 10 words (1 vs. 3: odds ratio [OR] = 4.306, SE = 1.56, z ratio = 4.04, p = .002; 3 vs. 5: OR = 0.102, SE = 0.0497, z ratio = –4.674, p < .001), 12 words (1 vs. 3: OR = 5.046, SE = 1.34, z ratio = 6.071, p < .001; 3 vs. 5: OR = 0.238, SE = 0.06, z ratio = –5.633, p < .001), and 14 words (1 vs. 3: OR = 2.483, SE = 0.58, z ratio = 3.889, p = .007; 3 vs. 5: OR = 0.403, SE = 0.09, z ratio = –3.889, p = .007).
Experiment 2
Method
Subjects
A new set of 10 young adult normal-hearing listeners participated in Experiment 2. No subjects participated in both Experiments 1 and 2. The mean age was 21.7 years (SD = 2.58). All listeners were native English speakers who consider English their primary language. As before, all listeners had normal hearing and no history of otologic disorders.
Stimuli
The same recorded materials used in Experiment 1 were used in Experiment 2, except now all eight conjunction words were included to ensure an equal number of options in each category (see Table 1).
Procedures
The same equipment used in Experiment 1 was used for Experiment 2. In Experiment 2, the aim was to compare conditions varying in the syntactic form of the input as well as in the order of subject response. Only a sequence length of 11 words was tested given the clear clause boundary effects observed in Experiment 1 for sequences of 10 words and longer and because 11-word sequences contain two full clauses. There were 6 conditions: 3 presentation orders × 2 response orders tested. Examples of these conditions are displayed in Table 2. The syntactic forwards, same order, and random, same order conditions were tested in Experiment 1, but all other conditions were new to Experiment 2. The syntactic forwards condition presents listeners with syntactic materials; however, in the reverse-order response mode, the output is no longer syntactic. Furthermore, the syntactic backwards condition presents backwards seemingly nonsyntactic materials, but in the reverse-order response mode, the output is syntactic. Note, however, that hearing or responding with backwards syntax results in stimuli that are not as purely nonsyntactic as the random materials are. There still exist word clusters that may be considered grammatical (e.g., “saw Sue” or “and hats”).
Examples of responses in each presentation order/response order condition combination
These six conditions were tested in 10-trial blocks. Each condition was tested once in a set with randomized condition order. Three such sets were tested, yielding 30 trials per condition. Listeners were informed of the condition before each block and were given a printed example of a sequence and the corresponding correct response for that condition. Thus, subjects were aware that their response in the syntactic backwards, reverse-order response condition should be a syntactic sequence. As in Experiment 1, subjects responded one word at a time by selecting from correct-category columns.
Results for Experiment 2
Experiment 1 showed that when listeners heard and responded to the syntactic condition, recall was superior to when listeners heard and responded to the random condition at most word positions. Furthermore, there existed U shapes in recall functions with serial position in the first clause in the syntactic condition, which was taken as evidence for clause-based organization of materials in memory. Experiment 2 examined whether there was an overall benefit of syntactic structure and whether the evidence of clause organization would be maintained when participants were required to mentally manipulate materials.
In Figure 2, average proportion correct recall (averaged across listeners) is shown for each condition and each response position. Generally, reverse-order responses (Figure 2b) led to better performance than same-order responses (Figure 2a) for earlier response positions, but the opposite was true for later response positions. For the same-order response conditions across presentations, there is evidence of primacy and recency. In the reverse-order response conditions, given that the last items heard are the first items responded and the first items heard are the last items responded, the best performance for the initial items may be more appropriately labeled recency. The primacy, then, is for the last items in the curve and is generally smaller than the recency. These primacy and recency effects in the reverse-order response conditions differ from the same-order response conditions likely because in the reverse-order response conditions, there is increased temporal distance between the first items heard and the response (items heard first are responded to last) and reduced temporal distance between the last items heard and the response (items heard last are responded to first).
Proportion correct recall (averaged across all listeners) as a function of response position in Experiment 2. Error bars are standard deviations. (a) Results for same-order response conditions, and (b) results for reverse-order response conditions. Each symbol depicts a different presentation order condition: circles are syntactic forwards, squares are syntactic backwards, and triangles are random presentation orders. Dashed lines mark the first clause end for the syntactic forwards presentation condition, and dotted lines mark the first clause end for the syntactic backwards presentation condition. Example word sequences in each condition are shown above each panel.
In order to examine evidence for the clause-based organization, the data in the syntactic conditions were analyzed for the presence of a U shape. In the syntactic forwards, same order and syntactic backwards, reverse order conditions, the end of the first clause corresponds to word position 5 (see line demarcations in Figure 2). In the syntactic backwards, same order and syntactic forwards, reverse order conditions, the end of the first clause corresponds to word position 7. The U shape was defined as statistically significantly poorer performance for the middle of the first clause than for the first and last word of this clause. The second clause was not evaluated because the clause end was potentially confounded with primacy or recency effects. As in Experiment 1, these comparisons were done using the emmeans function in the R statistical software and a Tukey correction for multiple comparisons. Note that the syntactic forwards, same order and random, same order conditions are the same conditions shown in Experiment 1, except now for an 11-word sequence with different participants. The middle of the first clause was significantly poorer than both the first and last words of the clause, indicating the presence of a U shape, for the syntactic forwards, same order condition (position 1 vs. 3: OR = 6.72, SE = 1.91, z ratio = 6.71, p < .001; position 3 vs. 5: OR = 0.227, SE = 0.06, z ratio = –5.96, p < .001) and the syntactic forwards, reverse order condition (position 11 vs. 9: OR = 0.497, SE = 0.08, z ratio = –4.10, p = .002; position 7 vs. 9: OR = 3.57, SE = 0.633, z ratio = 7.18, p < .001). However, this was not the case in the syntactic backwards, same order condition (position 11 vs. 9: OR = 0.114, SE = 0.02, z ratio = –11.13, p < .001; position 7 vs. 9: OR = 1.29, SE = 0.22, z ratio = 1.47, p = .93) or the syntactic backwards, reverse order condition (position 1 vs. 3: OR = 34.99, SE = 20.88, z ratio = 5.96, p < .001; position 5 vs. 3: OR = 1.28, SE = 0.24, z ratio = 1.31, p = .967). Thus, there is evidence of clause-based organization in the syntactic forwards conditions (in both same-order and reverse-order response conditions), but not in the syntactic backwards conditions.
If the random presentation condition may be considered baseline for recall of items without any syntax, then the improvement in recall with either forwards or backwards syntax may be used as an estimate of the “benefit from syntax” or, put differently, the SSE. In order to test the SSE across conditions, each individual’s improvement in each syntactic condition relative to the random condition was calculated. Specifically, the difference in average proportion correct for the syntactic condition versus the random condition was calculated for each subject. The benefits averaged across subjects are shown in Figure 3. For the same-order response conditions (filled bars), benefits are relative to the random, same order condition. For the reverse-order response conditions (open bars), benefits are relative to the random, reverse order condition. Testing that these benefits were significant was done using the emmeans function in the R statistical software with a Tukey correction for multiple comparisons. In all four conditions, mean benefits (relative to the random condition of the same response type) were significant: syntactic forwards, same order (Mean = .314, SD = .0868; OR = 0.187, SE = 0.0125, z ratio = –25.06, p < .001); syntactic forwards, reverse order (Mean = .2703, SD = .085; OR = 0.258, SE = 0.021, z ratio = –16.46, p < .001); syntactic backwards, same order (Mean = .1109, SD = .105; OR = 0.618, SE = 0.035, z ratio = –8.34, p < .001); syntactic backwards, reverse order (Mean = .1797, SD = .119; OR = 0.399, SE = 0.035, z ratio = –10.38, p < .001).
The mean proportion correct improvement in each syntactic condition relative to the random condition of the same response type in Experiment 2. The improvements for same-order response conditions (filled bars) are relative to the random, same-order condition. The improvements for the reverse-order response conditions (open bars) are relative to the random, reverse-order condition. Error bars are standard deviations.
Earlier analysis demonstrated U shapes in the syntactic, forwards conditions. However, in the syntactic, forwards (and syntactic, backwards) conditions, each serial position is defined by a particular word category. That is, the beginning of the clause is always a name and the end of a clause is always an object. To verify that the peaks observed in the syntactic conditions are due to clause boundaries and not to some inherent benefit of the specific word categories themselves, the random data in Figure 2 were analyzed by word category. It is assumed that if there is no inherent benefit of particular word categories for memory, the random condition should show equal performance across word category. A generalized linear mixed model was fitted to the random data broken down by word category and with subject specified as a random effect. As before, the analysis was conducted using the glmer function in the lme4 package of the R statistical software. The effect of word category was not significant, F (5) = 0.655, p = .6577. Thus, the word position effects in the syntactic conditions are not likely to be a result of better recall of particular word categories over others.
Experiment 2a
In Experiment 2, in all of the syntactic conditions, listeners were informed of the word order and became familiar with this order across trials in each block of these conditions. This was not the case with the referents used to calculate the SSEs—the random conditions—because a new and random ordering of word categories occurred on every trial. In the random conditions, all of the items in the corpus were equally likely alternatives at each position within the sequence of words and the predictability of the next word in the sequence was low—equal to 1/set size. In contrast, for the syntactic conditions, the a priori knowledge about the word category at each position in the sequence reduces the size of the set of possible items to only eight—the number of exemplars in each syntactic category. Therefore, conformance to a known category order can reduce observer uncertainty because the likelihood of any given item is reduced from 1/48 to 1/8. Thus, it cannot be determined if the SSEs observed reflected only a benefit due to syntax or if reduction in uncertainty also played a role. To measure potential benefits due to reduced uncertainty, an additional condition was tested in which a randomly generated order of syntactic categories was fixed across trials in a random fixed condition. Evidence that reduction in uncertainty played a role in the conditions tested in Experiment 2 would come in the form of better performance in the random fixed condition relative to the random condition.
Method
Subjects
A new set of seven young adult normal-hearing listeners participated in Experiment 2a. None of these subjects participated in the previous experiments. The mean age was 20.67 years (SD = 0.82). All listeners were native English speakers who consider English their primary language. As before, all listeners had normal hearing and no history of otologic disorders.
Stimuli
The same recorded materials used in Experiment 2 were used in Experiment 2a (see Table 1).
Procedures
The same equipment used in Experiment 2 was used for Experiment 2a. As in Experiment 2, a sequence length of 11 words was tested. There were 8 conditions: 4 presentation orders × 2 response orders. All of the conditions from Experiment 2 (see Table 2) were tested again in Experiment 2a. In addition, a random fixed condition was run. In generating the word order of the random fixed condition, the aim was to ensure that the category order was “nonsyntactic.” To accomplish this aim, the draw of the order of the word categories was restricted such that more than two sequentially adjacent categories (as depicted in Table 1) was not allowed. Even with this restriction, it is feasible that some draws of word category order would have led to partially “syntactic-seeming” sequences, or sequences better recalled for other reasons. To minimize this possibility, a different random fixed category order was generated for each subject, but this order was held fixed across trials of this condition within a subject.
These eight conditions were tested in 10-trial blocks. Each condition was tested once in a set with randomized condition order. Three such sets were tested, yielding 30 trials per condition. Listeners were informed of the condition before each block and were given a printed example of a sequence and the corresponding correct response for that condition. As in Experiments 1 and 2, subjects responded one word at a time by selecting from correct-category columns.
Results for Experiment 2a
Each individual’s improvement in the syntactic forwards, syntactic backwards, and random fixed conditions relative to the random condition was calculated. As in Experiment 2, the improvement relative to random was calculated for the condition of the same response type (e.g., syntactic forwards, same order relative to random, and same order). The cross-subject averages of these benefits relative to the true random condition are shown in Figure 4a. Note that the benefits in the syntactic conditions (syntactic, forwards and syntactic, backwards presentation) are similar in size and pattern to those with a different set of subjects in Experiment 2 (compare Figure 4a to Figure 3). As before, testing that these benefits were significant was done using the emmeans function in the R statistical software with a Tukey correction for multiple comparisons. There was a significant benefit of random fixed relative to random in the same-order response condition (Mean = 0.105, SD = 0.129, OR = 1.618, SE = 0.1961, z ratio = 3.972, p = .0018). However, in the reverse-order response condition, the random fixed and random conditions were not significantly different (Mean = 0.012, SD = 0.075, OR = 1.147, SE = 0.17, z ratio = 0.936, p = .98). The significant difference between random fixed and random in the same order condition indicates that fixing the ordering of word categories across trials yields a benefit in recall. As such, the SSEs in Experiment 2 with same order responding using the random (nonfixed category ordering) as a referent may have been influenced by a reduction in uncertainty due to a priori knowledge of category order. In order to “correct” for this effect, the SSEs were calculated using the random fixed condition of the same response type as the referent. These average benefits are shown in Figure 4b. The SSEs calculated in this way were significant for the syntactic forwards, same-order (Mean = 0.184, SD = 0.086, OR = 0.394, SE = 0.05, z ratio = –7.474, p < .001), syntactic forwards, reverse order (Mean = 0.216, SD = 0.078, OR = 0.335, SE = 0.05, z ratio = –6.83, p < .001), and syntactic backwards, reverse order (Mean = 0.104, SD = 0.09, OR = 0.611, SE = 0.09, z ratio = –3.213, p = .029) conditions. However, in contrast to Experiment 2, there was no longer a significant SSE in the syntactic backwards, same order condition (Mean = –0.025, SD = 0.103, OR = 1.145, SE = 0.134, z ratio = 1.152, p = .945).
(a) Proportion correct improvement (averaged over the seven subjects in Experiment 2a) in each condition relative to the random condition of the same response type. The improvements for same-order response conditions (filled bars) are relative to the random, same-order condition. The improvements for the reverse-order response conditions (open bars) are relative to the random, reverse-order condition. Error bars are standard deviations. (b) The mean (over the seven subjects in Experiment 2a) proportion correct improvement in each syntactic condition relative to the random fixed condition of the same response type.
Discussion
The first aim of this study was to learn whether syntactic relations among word strings (for materials with low semantic predictability and without typical prosody) would be sufficient to produce benefits for the recall of speech, either for enhancements to performance at clause boundaries or for overall sentence recall performance (SSE). Significant benefits of syntactic presentation, relative to nonsyntactic (random) presentation, were found for most word positions in Experiment 1, where the subjects reported the items in the order in which they were presented. This confirms a SSE for these methods and materials. Because the sentences used in the syntactic condition had low semantic predictability (e.g., the meaning of the current and preceding words had no bearing on the likelihood of occurrence of the next within-category word in the sequence), and there was no phrase-level prosody (i.e., the words were spoken individually with neutral inflection when recorded), the benefit observed may be attributed more directly to syntactic structure. Our results are broadly consistent with those reported by Baddeley et al. (Reference Baddeley, Hitch and Allen2009), who also found a robust SSE using closed-set materials and individually recorded words, although the length of the stimulus was limited to sentences containing only a single clause rather than extending past the clause boundary as was the case here. In addition, there were some other differences in terms of the number of sentences in the corpus and the degree of uncertainty about the talkers and presentation that may have influenced the results in the Baddeley et al. (Reference Baddeley, Hitch and Allen2009) study.
A second finding in Experiment 1 was that the pattern of word recall as a function of serial position supported the interpretation that even these semantically and prosodically impoverished sequences of words were organized in memory as clauses in the syntactic condition. Specifically, the recall functions for the first clause of the syntactic stimulus clearly conformed to the U shape that is characteristic of clause-based organization. These results are broadly consistent with past findings of clause-level bowing or U-shaped recall functions reported by Marslen-Wilson and Tyler (Reference Marslen-Wilson and Tyler1976) and Jarvella (Reference Jarvella1971; see also Jarvella, Reference Jarvella and Bower1979), although there are important differences in the methods. In Jarvella (Reference Jarvella1971), the stimuli were dependent clauses within contextually rich narratives recorded from speakers using natural inflection. In Marslen-Wilson and Tyler (Reference Marslen-Wilson and Tyler1976), a “normal” condition was examined that employed materials similar to those used by Jarvella (Reference Jarvella1971). However, Marslen-Wilson and Tyler broadened the scope of the study to include a condition in which syntactic structure was preserved while phrase- or sentence-level meaning was reduced. In order to achieve this property of the utterances, the words in the normal narrative were replaced by other words of the same semantic category yielding syntactic but meaningless materials that were uttered with natural pauses and intonation. Further, in a third condition, the words were scrambled within the test phrases yielding a fully meaningless sequence, but these words also were uttered with normal intonation and pauses. In each of these three conditions providing different degrees of word sequence structure and meaning, there was evidence of clause-level bowing in the resulting recall function. It is important to note that the nonsense materials uttered with pauses and prosody were sufficient to produce the bowed recall function indicative of clause-based organization. These findings support the view that the clause-level organization in memory (as evidenced by the U-shaped/bowed recall function) may be observed independent of the contextual richness of the speech materials. The present results complement these past studies in that the U-shaped recall function that is characteristic of clause-based organization in memory is obviously absent with no syntax or phrase-level prosody, but was shown to be present when syntax was carried only by speech produced according to grammatical English word order (with no phrase-level prosody). It is important to note that the design of the current experiment allowed direct comparison of random and syntactic conditions without confounding cues (e.g., prosody or semantic predictability of sequential items) because the exact same stimuli—at the individual word level—were used in both cases. The syntactic condition recall benefit relative to the random condition occurred at virtually all word positions, but was particularly large at word positions associated with the end of a clause (see Figure 1).
In Experiment 2, the possible benefit of syntax for overall recall (i.e., SSE) and the word position enhancements due to clause-based organization in memory were examined in conditions where normal syntax (a) was present both at presentation and at the formulation of responses (syntactic, same-order response), (b) was present only at presentation but not during response formulation (syntactic, reverse-order response), (c) was present at the formulation and act of registering the response but not during stimulus presentation (syntactic backwards, reverse-order response), or (d) was not present either at presentation or response formulation (syntactic backwards, same-order response). The goal of these manipulations was to determine if benefits of syntactic structure could be observed when mental manipulation— required in order to make a response—either disrupted or yielded syntactic structure. The results showed that the enhancement of recall at the clause boundaries indicating clause-level organization was present in syntactic forwards, same order and syntactic forwards, reverse order conditions. Thus, the clause-based organization in memory was resistant to mental manipulation of materials. However, there was no evidence of clausal organization for the syntactic backwards, same order or syntactic backwards, reverse order conditions.
In addition, in Experiment 2, there was an overall recall benefit relative to the fully random materials for the syntactic materials both when they were presented naturally in the forwards case and when they were presented in reverse order. These benefits relative to random presentation, both of which would seem to qualify as a SSE, occurred when subjects responded to materials in the order in which the materials were presented (i.e., first-in first-out) and in reverse order (i.e., last-in first-out). However, one confound in Experiment 2 was that the syntactic forwards and syntactic backwards conditions maintained a consistent category order at presentation across trials, whereas the random condition—the referent condition for calculating the SSE in that experiment—presented a different randomized category order on each trial. Thus, at least a portion of the benefits in the syntactic conditions relative to the random conditions may have been due to a reduction in uncertainty. In the random condition, during stimulus presentation, the predictability of the next word in the sequence was low—equal to 1/set size—while the uncertainty experienced by the observer about the item presumably was high. In contrast, for the syntactic conditions, the a priori knowledge about the word category at each position in the sequence reduces the size of the set of possible items to only eight—the number of exemplars in each syntactic category. Note that in all cases, the uncertainty about the response was equal because each forced-choice selection was limited to the eight items in the syntactic category of the word that was actually presented. Several past studies using open-set materials have concluded that the SSE cannot be attributed to an increase in transitional probabilities among the words or to the corresponding reductions in uncertainty, but is more plausibly attributed to chunking according syntactic or grammatical rules (e.g., Baddeley et al., Reference Baddeley, Hitch and Allen2009; Epstein, Reference Epstein1961; Marks & Miller, Reference Marks and Miller1964; Miller, Reference Miller1956). In those studies, any effects due to reductions in uncertainty were minimized because of the very large set sizes and therefore the very high uncertainty in all cases. Furthermore, Epstein (Reference Epstein1961) showed improved recall of nonsense words when put in the context of grammatical structure than for these same nonsense words without grammatical cue words, an effect that cannot be attributed to reduced uncertainty. With the current materials, however, the set size was small enough that reduction in uncertainty between the random and syntactic conditions may still have played a role in the calculated SSE.
The final experiment, Experiment 2a, provided some clarity on this issue. A random fixed condition was tested that presented a randomized ordering of word categories that remained consistent across trials. Thus, this condition did not adhere to a consistent or known syntax, but had the same word-to-word predictability as the syntactic conditions. This condition differed from the random condition only in the word-to-word predictability. Relative to the random condition, there was a benefit in the random fixed condition when subjects responded in the order in which the words were presented. However, when subjects responded to the materials in reverse order, there was no difference in performance between random and random fixed conditions. When using the random fixed condition as the referent, there was a significant SSE in the syntactic forwards conditions (both in same order and reverse-order response modes), and in the syntactic backwards, reverse order response conditions. The syntactic backwards, same order condition no longer yielded a significant SSE when the random fixed referent was used—suggesting that the significant SSE in this condition in Experiment 2 had been due to reductions in uncertainty. The different results in Experiment 2a relative to Experiment 2 highlight the importance of using the appropriate referent condition when calculating the SSE. Experiment 1 examined differences between syntactic and random conditions, where the confound of reduction in uncertainty also existed. The benefits of syntax in Experiment 1, therefore, may be overestimated. The size of this overestimation may be estimated by comparing the size of the syntactic forwards, same order benefits in Figure 4, between 4a and 4b.
The results of Experiment 2a generally indicate that the SSE is resistant to mental manipulation, which disrupts syntactic structure, and that an SSE can be obtained when mental manipulation required in order to make a response yields syntactic structure. Related to this last point, a particularly interesting finding was that there was an SSE for the syntactic backwards, reverse-order response condition but not for the syntactic backwards, same-order response condition. That is, for the syntactic materials presented backwards, performance was better relative to the random condition when the subjects responded in reverse order but not when they responded in the order of presentation. At first glance, this may seem counterintuitive given that responding in reverse order requires greater mental manipulation of materials—placing a higher load on working memory—in order to formulate a response. However, an important feature of the syntactic backwards, reverse-order condition was that the materials formed forwards syntactically correct sentences after the materials had been mentally reversed, a feature that appears to confer a benefit. This is the first evidence for this effect that we have found in the literature.
Overall, the results of the present study complement the findings reported by Baddeley et al. (Reference Baddeley, Hitch and Allen2009). That previous study found that the size of the SSE did not change when subjects also performed concurrent tasks thought to disrupt the “central executive.” According to their interpretation, the SSE is a result of automatic linguistic chunking at encoding. In Experiment 2a, mental manipulations of the materials themselves appeared not to reduce the size of the SSE of syntactic materials presented in forwards order (see Figure 4b), possibly because the materials had already been encoded. However, the finding of a significant SSE in the syntactic backwards, reverse order condition suggests that the SSE does not only come about due to linguistic chunking at encoding. Although the conditions tested here cannot definitively isolate the encoding stage, it may be assumed that materials that conform to standard syntax only after mentally reversing them would not have been syntactic during encoding. If this assumption is correct, then the SSE in the syntactic backwards, reverse order condition cannot be due to linguistic chunking at encoding. The results of Experiment 2 indicated that clause-level organization of linguistic materials in memory—at least as inferred from the U-shaped recall function—also possibly occurs automatically and exclusively during encoding. There was evidence of this U shape in the syntactic forwards conditions, and mental manipulation of the materials to respond in reverse order did not eliminate evidence of this organization. Furthermore, there was no evidence of clause-based organization in the syntactic backwards, same order or syntactic backwards, reverse order conditions. Neither of these conditions contained standard syntax during presentation, and the syntactic backwards, reverse order condition formed syntactic sequences only after mental manipulation. If it can be assumed that materials that only conform to standard syntax after mental manipulation were not syntactic at encoding, then the absence of clause organization for conditions with backwards syntax at presentation appears to support that clause organization arises during encoding. Further research is needed to support or refute these claims.
In summary, the present study provided evidence that syntactic ordering of word strings improved recall. Clear evidence of clause-based organization in memory (and the corresponding enhancements for items at clause boundaries) was only apparent when natural syntactic structure occurred during presentation. Furthermore, for syntactically presented materials, clause organization appeared to be resistant to mental manipulation of the materials to respond in reverse order. These results suggest that this clause-based chunking/organization may occur for syntactic materials at encoding. The benefit of syntactic structure for overall recall was also apparent in these conditions—for materials that were syntactic at presentation, regardless of response order. However, a benefit of syntactic structure for overall recall also was apparent for a condition in which enhancements at clause boundaries were not observed—when sentences were played backwards and then were mentally manipulated to respond in reverse order, yielding standard syntactic sequences only in the response. These findings indicate that the benefits of syntactic structure for recall are not only resistant to mental manipulation but also can be obtained when mental manipulation results in syntactic structure. Further research is needed to determine if these findings may be generalized to other types of clauses and syntactic structures. Overall, the benefit of syntactic structure, occurring even when sequential word predictability from semantic context is negated by random selection within a closed-set word category and when prosodic cues are neutralized, argues for a strong role of syntax in the organization of speech in working memory.Footnote 3
Acknowledgments
This work was supported by grants to the third author from the Air Force Office of Scientific Research, FA9550-16-10372, and the NIH National Institute on Deafness and Other Communication Disorders, R01 DC04545.
