INTRODUCTION
In face-to-face interaction, people use two very different forms of human expression: spoken language and embodied gesture. “At times they [gestures] are used in conjunction with spoken expressions, at other times as complements, supplements, substitutes or alternatives to them” (Kendon Reference Kendon2004:1). Kendon's remark suggests that there are orderly relations between talk and gesture, but this orderliness has yet to be fully delineated. In this article, I examine one kind of orderliness to the relation between gesture and spoken language involving referential practice.
Referring to physical objects (or processes) is one of the most basic and mundane actions in social interaction.Footnote 1 People refer to physical objects in the course of all manner of social activities, from a toddler's demanding a cup of juice to a homeowner's seeking door hinges at a hardware store to an astronomer's pointing out features in the heavens. While much epistemological work on the problem of reference has been done, there has been much less empirical work on the practice of reference. The problem of reference is the philosophical question of how it is possible in the first place for thoughts or verbal expressions to refer to objects in the world (Russell Reference Russell1910; Wittgenstein Reference Wittgenstein1961). Its solutions tend to take the form of abstract theories of meaning and correspondence. The practice of reference, on the other hand, consists of the systematic ways that people refer to objects in the world in and through social interaction. Understanding the practice of reference requires the empirical investigation of naturally occurring social interaction.
In this vein, there is a growing body of work examining referential practice (Kendon Reference Kendon and Key1980, Reference Kendon2004; Hindmarsh & Heath Reference Hindmarsh and Heath2000; Hanks Reference Hanks1990; Clark & Wilkes-Gibbs Reference Clark and Wilkes-Gibbs1986; Goodwin Reference Goodwin1986, Reference Goodwin1994, Reference Goodwin and Kita2003; Wootton Reference Wootton1990; Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979; Schegloff Reference Schegloff and Sudnow1972, Reference Schegloff1996). For example, Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979 examine how speakers refer to persons using names and verbal descriptors over the telephone. Goodwin Reference Goodwin1994, Reference Goodwin and Kita2003 analyzes how archaeology students are taught to see “features” in dirt in part through pointing and tracing. Hindmarsh & Heath Reference Hindmarsh and Heath2000 examine uses of pointing among workers in a telecommunications control room when referring to information objects in a complex environment of computer monitors and documents. And Kendon Reference Kendon2004 examines a variety of methods through which the inhabitants of Naples refer to mundane objects in their everyday lives.
Most studies of referential practice examine deictic or pointing gestures. For example, Hindmarsh & Heath Reference Hindmarsh and Heath2000 analyze pointing in an “operations center” and focus on the coordination work required to achieve successful ostensive or deictic reference in interaction. They analyze how co-workers must physically align their bodies in order to achieve mutual orientation toward an object on a computer screen. They write:
To enable Rob to arrive in an appropriate position before producing the actual point, Steven forestalls the gesture by simply turning the pen over in his hand, such that the nib faces the monitor…. As he nears the screen, Steven produces the thrust of the gesture, allowing his co-participant to see the actual pointing gesture at the moment it occurs and with regard to its referential domain. (Hindmarsh & Heath Reference Hindmarsh and Heath2000:1867)
We see, then, that unlike verbal resources, such as categories or names, visual resources can require complex coordination among eyes, bodies, and objects in physical space and in time. Like Hindmarsh & Heath Reference Hindmarsh and Heath2000, this study examines the coordination work involved in referential practice; however, it focuses on reference by depiction rather than reference by ostension. It examines how speakers can refer to things not by pointing to them, but by depicting them visually, using their hands as well as other props.
In addition to the use of gestures, referential practice can also, of course, involve the use of words, such as names, categories, or descriptors. Schegloff Reference Schegloff and Sudnow1972 and Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979 are among the few studies that examine the relationship among alternative verbal reference forms. They ask: Given that there are multiple, equally “true,” ways to refer to things, like “places” (Schegloff Reference Schegloff and Sudnow1972) or “persons” (Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979), how do speakers choose one verbal reference form over the others on any particular occasion? To answer this, they propose two preferencesFootnote 2 in the organization of reference (Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979:16–17). The first is a preference for minimization: “On occasions when reference is to be done, it should preferredly be done with a single reference form” (rather than a combined form). The second is a preference for recipient design: “If they are possible, prefer recognitionals” (i.e., reference forms that are dependent on the relationship between the referent and the particular speaker or recipient as well as their knowledge, such as John or John's place). “If possible” here means that recognitionals should be used only where they are “expectably recognizable” by the particular recipient (Schegloff Reference Schegloff and Sudnow1972:99). Place names and personal names are prominent examples of recognitionals, which also tend to be minimal, and therefore tend to be preferred as a verbal reference form (Schegloff Reference Schegloff and Sudnow1972:101; Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979:18).
While Schegloff Reference Schegloff and Sudnow1972 and Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979 focus on verbal references to places and persons, usually over the telephone, this article expands their notion of “minimization” by examining referential practice in face-to-face interaction and the relationship between verbal and embodied forms of reference. When face to face, speakers may refer to a thing: by categorizing or naming it (e.g., a pita sandwich), but also by pointing to it (e.g., that plus a point to a particular pita sandwich), or by depicting it visually and verbally (e.g., that kind of sandwich that's a pocket plus a visual depiction of a pita pocket by slightly cupping the hands in a praying gesture). Each of these kinds of reference has distinct affordances and constraints (Gibson Reference Gibson1979). The fact that there are these alternative ways of referring to the same objects raises the questions: Under what circumstances do speakers choose particular ways over others? And is there a larger system of reference? This article addresses these questions by analyzing referential practice in the context a particular setting, a face-to-face service encounter at a quick print shop, in which customers must refer to services in the course of requesting them.
This article contributes to the growing literature on referential practice in two ways. First, it contributes methodologically by showing that incorporating an analysis of iconic gesture is necessary in understanding referential practice overall. Most of the current literature focuses exclusively on the use of deictic gesture or pointing. Second, it makes a theoretical contribution by showing how three very different modes of reference – categorization, ostension, and depiction – are related as part of a larger system of reference. That is, they are not simply alternative ways of doing reference but rather are ordered relative to one another. This ordering is manifested in the fact that when iconic gesture, or depiction, is used for referring, it tends to be used to accomplish reparative work when names fail.
BACKGROUND AND METHODS
One kind of place where referential practices are abundant is over service counters where customers request products and services (Merritt Reference Merritt1975, Whalen et al. Reference Whalen, Whalen, Moore, Raymond, Szymanski, Vinkhuyzen, LeVine and Scoloon2004). In requesting pastries at a bakery, sandwiches at a deli, tools at a hardware store, hairstyles at a salon, or car repairs at a garage, customers must in some way refer to what they want the service organization to provide. One such organization is a quick print shop: a retail business that provides xerographic copying and printing (as opposed to offset printing). At print shop service counters, customers can place orders for document services in person. Customers provide some of the raw materials, the “originals,” and specify the services to be performed on them. Employees usually record the specifications on a standard order form, which then travels to a different employee who produces the job. The customer then receives the job back at a later time. Ordering encounters are rich in referential practices. In the course of ordering, customers must routinely refer to a variety of physical objects and processes, including types of documents, particular documents and materials on the scene, the operations that can be performed on documents, and so on.
In addition, at service counters there tends to be a particular kind of asymmetry in the relative states of the participants' knowledge. Although customers know what they want (to varying degrees) as an end product, they often lack adequate knowledge about what exactly is involved in achieving that outcome, what the required document operations are called, and what all of their options are. This can make requesting document services challenging for customers. Employees, in contrast, tend to know (to varying degrees) their institution's official terminology for its services and the ways to produce different types of documents, but not what the customer wants or needs. More importantly, employees will be assumed to possess such knowledge by virtue of their observable membership in the category “print shop employee” (Schegloff Reference Schegloff and Sudnow1972). In order to achieve mutual understanding about the order, the participants must work to manage their relative states of knowledge and arrive at mutual recognition and understanding. Often failures to achieve mutual understanding are not detected until the customer returns to pick up the order and discovers that it was done incorrectly (Whalen et al. Reference Whalen, Whalen, Moore, Raymond, Szymanski, Vinkhuyzen, LeVine and Scoloon2004).
The primary data for our study consist of video recordings of naturally occurring interactions over the service counter at a quick print shop, which we will call Eastside Reprographics. Approximately 400 hours of video were shot over a three-year period as part of a larger ethnographic study of the print shop in three different locations in northern California. For this study, 44 instances of references to documents or document services were collected for detailed transcription and analysis. The labor-intensive nature of the transcription limited the total number of cases that could be analyzed. Through iterative analysis, these 44 cases were further broken down into four sub-collections based on emergent patterns: (a) no-trouble situations in which the customer names a document service and/or points to the originals (19 cases); (b) failed attempts to use a name for a service, which are subsequently repaired using iconic gesture (6 cases); (c) the use of a name or category in combination with a gesture (15 cases); and (d) the use of the a “wrong” category in combination with an iconic gesture (4 cases).
An ethnomethodological, conversation analytic approach is taken in analyzing these data. Ethnomethodology and conversation analysis are concerned with how people organize and achieve recognizable social activities, such as “placing an order,” in and through their concrete embodied actions, and especially their talk (Garfinkel Reference Garfinkel1967, Maynard & Clayman Reference Maynard and Clayman1991, Jordan & Henderson Reference Jordan and Henderson1995). The aim of analysis is twofold: to identify the recurrent methods, practices, or devices that people use in social action, and to explain how these are uniquely accomplished on particular occasions (Schegloff Reference Schegloff1987, Zimmerman Reference Zimmerman and Anderson1988). Analysis involves close examination of audiovisual recordings and detailed transcripts of the activity of interest. Multiple instances of the same phenomenon are collected, where “sameness” is based on shared formal properties such as action type, action design, and sequential structure. By working with collections, the analyst can discover the variety of trajectories that a sequence type can take under different circumstances. Analysis focuses on how the practice works, rather than on its frequency or distribution in a setting.
The excerpts in this article were transcribed using a modified version of conversation analytic conventions.Footnote 3 Short descriptive glosses of gestures and nonverbal actions are inserted into the transcriptions of the talk in order to show how they co-occur temporally. Braces {} are used to show how the gestures overlap with the talk, similar to the way that brackets [] are traditionally used to show overlapping talk. The talk appears in boldface in order to make it easily distinguishable from the gestures.
The precise timing of gestures in relation to talk was made possible using audiovisual speech analysis software (the open source tool WaveSurfer) that provides both a video window and a graphic representation of the audio track. The synchronized video and spectrogram enable the analyst to “see” the talk and the embodied action simultaneously. The analyst can determine the exact moments at which a gesture begins and ends, using the video window, and mark these points on the spectrogram. This enables the analyst easily to measure the duration of the gesture and to see precisely how it co-occurs with the talk.
No-trouble references to originals and document services
When placing orders at Eastside Reprographics' service counters, customers routinely have two kinds of things to which they must refer: their originals and the services they want performed on their originals. Although both are physical in nature, customers use quite different resources for referring to each. Customers tend to refer to their originals ostensively, while they tend to refer to document services by name or category. The salient difference between originals and document services in this setting is that the former tend to be present on the scene, while the latter do not.
As Schegloff Reference Schegloff and Sudnow1972 argues, names are prominent forms of reference because they tend to be both minimal and recognitional. While he was talking about place names and personal names, the same pattern appears to exist for the names of document services. The customer's originals, however, cannot generally be referred to by name because employees tend not to have any prior experience with them (e.g., I need 10 copies of the XGS brief will not work with an employee who has never heard of the XGS brief, but may work with a knowledgeable co-worker). The following excerpt demonstrates the typical way that customers refer to their originals and the document services they want.
In response to the employee's offer of service (line 05), the customer approaches the counter with two large stacks of paper in manila folders (line 08) and places them on the countertop (line 10). The customer then makes a request: She places both hands flat on the stacks (line 12; Figure 1) and says, We need three copies of this (line 11). She thus uses a deictic gesture (placing her hands on the stacks) and deictic term (this) to refer ostensively to her originals for the job. In contrast, she uses a standard term (copies) used by Eastside Reprographics to refer to the service she wants performed on the originals and uses no visual resources in doing so.Footnote 4 Both references succeed in getting the employee to understand what kind of service she is ordering, as demonstrated in the employee's response (line 16). However, where the name copies is both minimal and recognitional, the ostensive reference to the originals is a slightly more expanded, embodied form.
Direct reference to originals by placing hands on stacks (line 12).
Similarly, in the following instance, the customer uses ostension to refer to her originals but naming to refer to the document service she wants.
The customer begins with an informational question, rather than a request, in which she refers to a document service by name, laminated (lines 06–07). The employee repeats part of the question (line 09) and acknowledges it (line 11), showing understanding and prompting the customer to continue. The customer then formulates a request: I need this laminated (line 12). Simultaneously with the beginning of the turn, the customer pushes her originals toward the employee (line 13). This customer thus uses ostension (pushing the originals forward plus the deictic term this) to refer to the originals and a name (laminated) to refer to the service. The employee then demonstrates understanding of the references and the request by mentioning the cost of the service (line 15).
Direct reference to original by pushing the diploma forward (line 13).
Referring to the originals ostensively and to one's desired services by name or category is the canonical way of placing orders at Eastside Reprographics. These no-trouble cases account for nearly half of the 44 collected cases and were observed many more times ethnographically. But despite this pattern, there are some systematic exceptions to it. For example, often customers' “originals” take the form of digital files on some type of portable disk. In such cases, customers cannot point directly to their documents (although sometimes they nonetheless point at the disks). Another group of exceptions involves a few document services – such as special paper, binding, and lamination – for which Eastside provides physical samples right at the service counter. Customers can inspect these in deciding which ones they want, and they can also point to them in referring to the corresponding document services. The following excerpt demonstrates how pointing can sometimes be used in referring to document services.
Both hold left index finger on paper sample (line 30).
In this case, the customer presents her originals to the employee by placing them on the countertop and pushing them slightly forward (line 07). In overlap she also describes them as a double-sided card (line 06). She thus appears to use two forms of reference, ostension and verbal description, to refer to her originals. However, later in the interaction (not shown), it becomes clear that the document the customer has presented is not in fact a “double-sided card.” She wants it printed double-sided as well as arranged four-on-a-page (or “four up”). Her description (line 06), therefore, actually refers to her desired output document, rather than to her originals in hand.
The first document service the customer indicates is the kind of paper she wants the document copied onto. In doing this, she uses a kind of pointing gesture in addition to a name. She utilizes a certain prop, the paper sample book (“PS-book” in transcript), which Eastside places at all its service counters in order to facilitate customers' choice of colored papers. As such, the samples of paper can be used by customers or employees for pointing. As early as line 10, the customer turns her gaze away from her originals toward the paper sample book. In the course of formulating the service she wants, she begins to retrieve the book (lines 12 and 15). Retrieving the book and finding the right paper sample cause significant delays in her talk, including syllable stretches (lines 11 and 17) and approximately 4.1 seconds of silence (lines 19–22), as her lips wait for her hands to catch up. Eventually after she finds the sample of the paper she wants, she points to it with her finger and holds the point (line 23). She then reads the name of that paper color, lunar blue (line 24). The employee displays understanding by putting his finger on the sample (line 29) and saying okay (line 28).
In referring to the document service in this case, we see that when a physical proxy of the service is present and the name is not readily available, the customer can point to the proxy. The customer most likely did not know the name of the color lunar blue until she found it printed next to the sample. However, she does mention the name instead of relying entirely on pointing. The name lunar blue helps disambiguate exactly to which paper sample she is referring.
Another notable feature of the customer's pointing to the paper sample book is that it takes quite a bit of work to bring the paper sample into play in the interaction. So in this sense, her pointing at it is not very minimal. In excerpts (1) and (2), the customers have their originals ready for referring before they produce the reference. However, in excerpt (3), the customer does not do this: When she begins her verbal reference, And I wanted it uhm printed onto (lines 11 and 17), the paper sample is not ready for pointing. Thus, pointing requires that the object, the speaker's body, and the recipient's gaze be brought into a configuration that enables the recipient to see both the hand and the object at the appropriate moments. Sometimes this is unproblematic, but at other times it can require elaborate interactional work.
We see, then, that as alternatives, categorical reference and ostensive reference have very different affordances and constraints (Gibson Reference Gibson1979). Ostension can accomplish recognition of an object with a simple gesture and a deictic word (e.g., that). At the same time, it requires that the object be available on the scene and that eyes, hands, and objects be coordinated. Categorical reference, in contrast, can accomplish recognition of an object with a single word. It does not require the presence of the object on the scene, nor the interactional management of the recipient's gaze. Instead it requires that the recipient bring certain knowledge to bear on it. Both the speaker and recipient must possess adequate knowledge of the standard names and categories – for example, photocopies, double-sided, lamination – for the objects in question. As we will see in the next section, this is not always the case.
Using depiction to repair failed attempts at naming
Although names for document services usually succeed in securing recognition, they also sometimes fail. As stated above, the successful use of names and categories requires certain knowledge on the part of both the speaker and recipient. Thus, on any given occasion, speakers must make judgments or analyses regarding these relative states of knowledge. Where “trouble” in recognition occurs, it may turn out to be due either to an error on the part of the speaker or to ignorance on the part of the recipient (Schegloff Reference Schegloff and Sudnow1972:93).
At Eastside Reprographics' service counters, it is not uncommon for customers to use vernacular terms for document services instead of the print shop's standard terminology. In such cases, employees are in a position to correct the customer's usage. The following two excerpts show failed attempts at using the names and categories by customers. We will see that in the face of such troubles in recognition, customers employ alternative reference forms to perform reparative work. In particular, they use depiction, both verbal and visual, on subsequent attempts at recognition.
This is not to say that customers always use depiction for the purpose of referring. In many cases in my data, customers use depiction for other kinds of actions, such as explaining, for example, indicating precisely where they want an image placed in a document or just how they want a document cropped. I do not analyze such cases in this article, but instead focus on instances in which depiction is used to refer to an object or service as part of a request. Furthermore, we will see that in using depiction in this way, participants nonetheless orient to a preference for the official term.
In the following excerpt, the customer attempts to refer to a document service by using a vernacular term rather than the institution's standard term. When the vernacular term meets with trouble in recognition, he switches to multi-modal depiction to repair the trouble.
“Laminating” gesture (line 26).
In requesting a document service, this customer refers to it using the term coated (line 18). He uses the term with confidence; that is, he does not hesitate or qualify it. In other words, he orients to the term as a perfectly adequate or correct choice of words. However, it nonetheless encounters “trouble” (Schegloff Reference Schegloff and Sudnow1972:93). It is met with a request for clarification by the employee (line 20), suggesting that she does not know it and therefore it is likely not the correct term (although “coated” in some printing contexts can refer to a particular type of paper). In response to the employee's repair initiator, the customer abandons categorical reference altogether and switches to an alternative reference form. He uses a multi-modal depiction consisting of a verbal description and an iconic hand gesture (lines 22–26). The gesture component appears to depict the action of a lamination machine applying a sheet of plastic to both sides of an invisible sheet of paper. These verbal and visual components appear to be designed as one piece. That is, both the verbal description, you know ya put ‘em between the two plastic deals or the hand gesture, thrusting two flattened palms outward and bringing them together, alone do not appear to be very intelligible. But together they elaborate each other and form a rather clear reference.
This multi-modal depiction then proves adequate for enabling this employee to guess the service in question. She proposes a candidate term, laminated (line 25), which the customer confirms (line 28). In other words, this customer lacks the standard term for the service he wants. He nonetheless tries a vernacular term, coated (line 18), but when the employee fails to recognize it, he tries a multi-modal depiction (lines 22–26). The recipient employee then displays recognition by providing the correct term for the service (line 25). The customer and employee thus display a preference for the official term for the service: First, the customer produces a term by itself; second, the employee replaces the incorrect term with the correct term; and third, the customer repeats the correct term.
While the employee in excerpt (4) initiated repair on the term coated explicitly, employees may also display trouble with customers' vernacular terms in less direct ways. In excerpt (5), the customer initially uses a term, spiral bound (line 6), in requesting a particular type of document binding. Although “spiral binding” may appear to be a standard term, Eastside Reprographics does not actually use it because it is ambiguous: It is sometimes used to refer to “coil” bindings and other times to “comb” bindings.
In response to this customer's use of spiral, the employee indicates no trouble with the term (line 8) and initiates a side sequence (Jefferson Reference Jefferson1972) regarding the turn-around time for the job (sequence omitted). However, when the employee returns to the topic of binding, she repeats the customer's earlier specification, but does so using a different term, comb binding (line 13). When the customer fails to confirm it, as indicated by a brief pause (line 14), she offers a third term, coil (line 15).
By proposing comb and coil, the employee suggests that spiral is not precise enough. Rather than choosing one of the alternatives, the customer abandons categorical reference altogether and opts for alternative resources. He could have fetched the binding sample rack, which is several feet away from him at the end of the counter to his right. However, he may not have known it was there, and at the very least, it is not ready to hand at the right moment. Instead, like the customer in the previous excerpt, he utters a verbal description, something that we can wrap around, and simultaneously performs an iconic hand gesture (lines 17–18). He raises both hands and makes two fists (Figure 5a, left frame) and then rotates his right fist outward so that it is palm-side up (Figure 5a, right frame). The gesture thus appears to depict the opening of the back cover of a booklet 180° such that it can lie flat. This can be done with either a comb or a coil binding. In the absence of immediate uptake by the employee, the customer elaborates. He again raises his fists with the left slightly higher (line 21; Figure 5b, left frame) and then slides his left fist down around the side of his right fist, cups the left hand, and cradles the right fist (line 22; Figure 5b, right frame). He then holds this gesture as he begins to say turn the back all the way over to lie flat (line 23). This second gesture thus appears to depict the wrapping of the front cover all the way over 360° so that it is flat against the back cover. This can only be done with a coil binding, not with a comb binding. In overlap with this second attempt at depiction, the employee proposes recognition with okay and supplies the correct term, coil (line 24). Again the participants display a preference for the standard term by initially attempting to produce it, although this fails, but then finding the correct term in the end. Multi-modal depiction is thus used to do reparative work on troubles in mutual recognition, but after the trouble is repaired, the participants revert to verbal categorization.
Figure 5a: “Opening book to lie flat”: Customer makes two fists and rotates fist (line 18).
Figure 5b: “Wrapping the cover all the way back”: Customer makes two fists and cups his right fist with his left hand (line 22).
Anticipating trouble with a name by accompanying it with a gesture
In the previous section, we saw customers attempting to refer to document services using categorical reference but then switching to depiction at the sign of troubles in recognition. These customers did not appear to know that they were using a problematic term until they saw the employees' responses. However, in other situations, customers display uncertainty in the use of a term from the start. They do this by producing names or categories in a “non-minimal form” (Schegloff Reference Schegloff and Sudnow1972) – that is, combined with hand gestures on their first attempts at referring. Employing two different kinds of resources in referring to a thing gives the recipient more opportunities to identify the referent.Footnote 5 It is therefore a way to manage situations in which either the speaker or recipient may have inadequate knowledge of the correct terminology. If either the speaker uses the wrong term or the recipient fails to recognize it, the inclusion of a second form of reference, a gesture, can help to compensate. In this section we will see situations in which customers compensate for their potentially inadequate knowledge up front, before any signs of trouble, by accompanying terms with iconic hand gestures.
In excerpt (6), the customer displays trouble with the pronunciation of the term for a document service but accompanies the term with an iconic hand gesture.
Figure 6a: Three finger spirals (line 14).
Figure 6b: Grabbing coil sample (line 21).
The customer, who speaks with a German accent, uses a kind of deictic gesture – the removal of pages from her box (line 12) plus the verbal deictic those (line 13) – to refer to her originals ostensively. She then says that she wants them to be speeral (line 13) and produces eight spirals with her index finger in overlap (lines 14, 17, and 19). She marks her use of the term speeral as possibly problematic in the way she produces it. She delays the production of the term with eh and a syllable stretch on be (line 13), during which time she completes three of the finger spirals (line 14), thus visibly doing a word search (Goodwin & Goodwin Reference Goodwin and Goodwin1986). She continues the gesture throughout her stretched production and nonnative English pronunciation of the term s:peeral: (line 13), and well into the employee's turn (line 18). Thus the “spiraling” iconic hand gesture provides an additional context for recognizing the term. Of such combinations of word and gesture (in ordinary conversation in Naples), Kendon (Reference Kendon2004:180) writes, “By using the gesture, however, the speaker is able to present that concept in another way, in a way that is not fleeting, but can be held before the audience.” Thus, by repeating the spiral finger gestures, the customer offers an alternative, longer-lasting reference to spiral binding than her use of the term alone and thereby orients to its possible inadequacy.
As we saw in the previous excerpt, “spiral binding” can be ambiguous in this setting, and this employee seeks to resolve the ambiguity by using an alternative form of reference. He shows recognition of the customer's intended word by uttering it and thereby correcting her nonnative pronunciation (line 18). Because the customer marked her use of the term as problematic, the correction comes as an anticipated or invited correction. The employee then proceeds to handle the ambiguity problem by ostension.
He asks, This one here? (line 18) as he reaches out and steps toward the binding sample rack (lines 20 and 22) at the end of the counter to his right. He quickly lifts and drops the sample of coil binding (line 23) immediately before saying, This one (line 24) and then lifts the sample of comb binding (line 25) as he says Or this one (line 24). In response, the customer does not confirm the comb sample that the employee is holding, but raises her right hand to do a gesture (line 27). As she does this, the employee drops the comb sample and reaches back for the coil (lines 28 and 30). With her arm raised, the customer does two quick downward points, somewhat as if she were tapping a key on an invisible keyboard (line 31). The downward taps appear to be directing the employee's hand downward, toward the lower sample, the coil, rather than pointing at it directly. A direct point from the distance she is standing from the rack (approximately six feet) would probably not adequately differentiate between the two samples, which are only inches apart. As she does the two “keyboard taps,” the customer also says (That'll be the one.) this one is (line 24). To confirm that it is the lower, coil sample to which the customer is referring, the employee also uses a deictic gesture of shaking the coil sample up and down (lines 30 and 33) while saying This one? (line 32). The customer then confirms this by nodding (line 35) and saying, Mhm? (line 34). Hence, the customer initially uses an iconic gesture to supplement her problematic categorical reference, speeral, and the employee and customer then use a series of deictic gestures to disambiguate the customer's use of the term.
The following case, excerpt (7), can be contrasted with excerpt (5). In that case, the customer used the term spiral bound with confidence, and the employee proposed alternatives, comb and coil, to disambiguate it. In contrast, in excerpt (6), the customer marked the term, speeral, as problematic from the start, and the employee then abandoned categorical reference altogether and switched to ostensive reference to solve the problem. Both employees oriented to the customers' displayed confidence with the institution's vocabulary: Where the customer displayed confidence, the employee continued using the institution's vocabulary, but where the customer displayed uncertainty, the employee switched to an alternative form of reference.
In excerpt (6), the customer included an iconic hand gesture (finger spirals) in combination with a term (speeral) in a way that displayed uncertainty with that term. In the following excerpt, the customer likewise produces an iconic hand gesture in combination with a term. However, in this case, she does it in a way that displays more confidence with the term. She thus provides for the possible lack of knowledge on the part of the employee.
“Cutting” gesture: first chop (line 13).
When the customer begins her initial request (line 05), she is already gazing at her originals on the countertop, and she maintains this gaze until she gets to the deictic word these at the end of the request (line 05). In other words, she “points” with her eye gaze toward her originals. She then prepares for a hand gesture (line 07) well before her production of the term score (line 12), so that the stroke or thrust of the gesture occurs simultaneously with the term (Kendon Reference Kendon and Key1980). She thereby illustrates the referent of the term score (line 12) with a chopping hand gesture (line 13). In the context of requesting document services, this can be seen as indicating machine cutting. Now had this customer delayed elements of her talk, produced the term with questioning intonation (or “try-marked” it; Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979), and/or looked up at the employee for a sign of recognition, her hand gesture would be seen as displaying her own uncertainty about the correct use of the term, like the customer in excerpt (6). However, she does none of these things. She uses the term score without delays (line 12), produces it with continuing intonation (line 12), prepares for a next gesture (line 17), does not look up at the employee for a sign of recognition, nor does she wait for such a go-ahead signal (line 16). In other words, she uses the term score with confidence. As a result, her hand gesture appears to be oriented not to the possibility that she may be using the wrong term, but rather to the possibility that the employee may not be able to recognize it.
But despite her displayed confidence, the customer in fact uses an incorrect term. The employee detects an inconsistency between the customer's visible iconic gesture, which depicts “cutting,” and her use of the term score, which denotes a kind of scratching that enables a hard crease. From this, she diagnoses the trouble source as the customer's use of the term, rather than her gesture (and thus the customer's knowledge). She then proposes an alternative term, cut (line 15), which the customer confirms (lines 19 and 22). Thus, it is the customer's knowledge that emerges as the source of trouble, and it is the iconic hand gesture that enables the employee to guess what the customer really wants despite what she says.
However, rather than simply accepting the employee's substituted term, the customer then probes further. She asks the employee to define “scoring.” When the employee does so, we see her rely on the same resources that customers use when they do not know the correct term for a service: multi-modal depiction.
“Folding” gesture (line 28), and tracing the “crease” (lines 28 and 31).
This employee uses a verbal definition, Scoring is like on a card, a folding card and it's a nice crease fold that sorta- (lines 25, 27, and 29), which is complemented with iconic hand gestures. The hand gestures involve two flattened hands depicting a folding card (lines 26 and 28) and then a pointing index finger tracing back and forth along the flat palm (lines 28 and 31) depicting the etching of the card that enables a nice crease. The employee thereby enables the customer to understand, and literally to see, to what scoring refers. From this multi-modal depiction, the customer again confirms the employee's proposal that she does not want scoring (excerpt 7a, line 19) but now does so in an informed way. In this case, as in others in our data, employees use multi-modal depiction to explain document services, although these are not the focus of this study.
By producing a term along with an iconic hand gesture, these customers project a potential trouble with the recognizability of the categorical reference alone. By producing the term in a tentative way, they imply that it is their own knowledge that is likely the source of the trouble. On the other hand, by producing it with confidence, they project the possibility that the recipient's knowledge might be a problem. Either way, the fact that these customers attempt to use categorical reference even though they anticipate that it will be problematic further demonstrates a preference for using the official term.
Compensating for a wrong term using hand gestures and props
In the previous section, we saw how customers can use gestures in combination with verbal categories to anticipate possible trouble in recognition. In this section, we examine cases in which customers anticipate definite trouble with a term, or rather, mark a term as definitely the wrong one. The customers compensate for their ignorance by using a combination of elaborate verbal and visual depiction. These depictions resemble those used by customers in the second section above to try to compensate for a failed use of a term; however, in this case they are produced up front in the initial attempt at reference.
Furthermore, these multi-modal depictions not only involve verbal descriptors and iconic hand gestures, but they also heavily involve the use of physical props (i.e., the customer's originals). The hands are of course also a kind of physical prop, but other physical objects can also be used by the hands. For example, Streeck (Reference Streeck1996:367) analyzes how mundane objects – a cookie, a cookie wrapper, and box – are used by speakers in creative ways as props or “situated symbols.” At the quick print shop service counter, customers routinely use their originals as components of visual depictions of documents and document services. Originals are ideally suited for such work because they are usually ready to hand and they share certain structural similarities to that which they are used to depict. While customers use their originals as props in a wide variety of ways, we focus in this section on a particular one: referring to particular document types by depicting them verbally and visually.
Like the gestures we saw in the prior section, the prop demonstrations in this section are used by customers as alternatives to the terms for document services. The customer in excerpt (8a) uses a term for a document type, tablet, but produces it as an inadequate one. She does this by immediately modifying the term with an elaborate multi-modal depiction involving her originals.
Figure 8a.1 Tracing the bound edge of a “pad” (line 4).
Figure 8a.2 “Tearing off” gesture (line 6).
The customer requests copies of this (line 01) and rotates a stack of originals to face the employee (line 02). She then requests an additional service by saying put it into a tablet where it's attached up here an’ ya tear ‘em off (lines 03 and 05). While saying this she does two different hand gestures. First, she makes her hand into a pinching shape, indicating a width, and then she traces her fingers along the top edge of the stack twice (line 04) as she says a tablet where it's attached up here (line 03). The verbal description and the hand gesture in relation to the originals elaborate each other. They indicate a kind of binding of the pages along one edge. There is also a deictic component to this multi-modal depiction, here plus the hand traces, both of which refer to the location of the binding along the top edge. Second, she verbally describes an action one performs with the kind of binding she wants: ya tear ‘em off (line 5). At the same time she mimes the tearing off of three invisible pages (lines 6–7), thereby depicting how the document is used. The customer has thus depicted verbally and with her hands and originals what Eastside Reprographics officially calls “padding.” The employee proposes recognition merely with sure (line 8).
However, somewhat later in the conversation when explaining the costs, the employee does an “embedded correction” (Jefferson Reference Jefferson, Button and Lee1987) of the customer's reference.
Employee mirrors customer's tracing (line 20).
In explaining the cost for the job, the employee introduces the standard term for the service, “padding,” into the conversation for the first time. After mentioning the set up fee (line 19), she gives the price per pad (line 25). However, well before she gets to the term pad, she mirrors the customer's earlier tracing hand gestures (8a, line 04). She reaches for the same corner of the stack (line 18) where the customer's traces originated. She then performs a series of three slow back-and-forth traces along the top edge of the stack (lines 20, 22, 24, 26, 28, and 30). She reaches the term pad in her talk just before she begins the third trace. The customer then acknowledges the fee for the service (line 23), and the employee asks about how many pads the customer wants (line 31). In answering this question, the customer now uses the term herself, two pads (line 32), in referring to the service. The employee thus links the customer's depictive reference with the standard term and in so doing, she teaches the customer the standard terminology.
Like the customer in the previous case, the customer in excerpt (9) tries to request a particular type of document. Rather than wanting her originals assembled into a pad, this customer wants hers assembled into a particular kind of booklet. This type of layout consists of a “tabloid-size” (11 by 17-inch) sheet of paper which, when folded in half, forms an 8.5 by 11-inch booklet. Two 8.5 by 11-inch impressions can then be printed on the outside of the booklet and two on the inside. Although this is a common type of document layout, surprisingly Eastside Reprographics itself lacks a specific term for it. (It could perhaps be called a “tabloid booklet” or the like.) Instead, employees tend to refer to it as being “on an 11 by 17” or “laid out as an 11 by 17” – that is, they refer to it with a short verbal description.
Figure 9a: Assembling the transient mockup (lines 10 & 12).
Figure 9b: Tracing the center line (line 18).
Although this customer uses a term in referring to the document service she wants, she does so in a different way than previous customers. She uses the phrase sort of like a book (line 11) and thereby indicates that she does not in fact want a “book” but something similar. In other words, she displays that she does not know the correct term. Because the term she uses is only an approximation, the customer must employ another form of reference. In the absence of something to point to, she performs a visual demonstration with props (lines 15–18). The customer begins preparing for the demonstration (line 8) well in advance. She opens a manila folder to reveal a stack of three single-sided 8.5 by 11-inch sheets that have text and small photographs on them (line 8) and arranges them into a configuration that resembles a double-sided 11 by 17-inch document (lines 8 and 10). As she announces that she wants it t' be sort of like a book (line 11), she is still preparing for the demonstration; she tries to grip the right edge of the right sheet but cannot get it at first. She then says Where it's: and pauses for 0.9 seconds while she grips the left corner of the left page with her left hand. Now she is finally ready to perform the demonstration. As she says front ‘n back (line 15), she briefly lifts both edges of the assemblage of sheets to indicate the printing on the underside. She then adds, an' it's connected hereso it's one (lines 15 and 17) and at the same time does one trace with her left hand down the border between the two stacks (line 18) indicating that it should be one large sheet.
The customer thus produces a kind of transient mock-up of her desired document. Through her real-time performance, she animates her static originals in a way that makes a tabloid booklet layout visible.Footnote 6 However, this mock-up is transient in that it dissolves by the end of the performance: Only her hands keep all the pieces together. This contrasts with static mock-ups, which are made to persist over time and are routinely handed over to the production staff in order to visually communicate requirements. Performing such a visual demonstration requires an expansive turn-at-talk as the props are maneuvered (lines 8, 10, 16 and 18) and some troubles encountered in doing so (lines 12 and 14). This contrasts sharply with a compact turn that could be achieved if there were a jointly recognizable term like “tabloid booklet” or a sample document that could be pointed to.
In response to the customer's prop demonstration, the employee does not provide a term for the particular document layout, but a candidate verbal description: So it's on an eleven by seventeen (line 21). The customer says Is that the size (line 25) and then “closes” the mock-up document, most likely as an aid in figuring out the size of the page that must be printed on. The employee answers Yeh (line 29), and the customer confirms Yeah, it is (line 31). Thus, in this case, both the customer and the employee appear to lack a standard term for this type of document. The customer uses a kind of place-holder term, book, that is marked as the wrong term from the start, and supplements it with an elaborate multi-modal depiction. Unlike in the previous excerpts, this employee substitutes not a term, but instead a minimal verbal description (line 21). Thus, if there is no official institutional term for a document service, a preference for such a term appears to be relaxed. Of course, it is the employee who is in a better position than the customer to know whether or not there is such a standard term. In this case, the customer displays an attempt to furnish the term while employee does not.
We see, then, that if terms are not known and referable objects are not available for pointing, multi-modal depiction is always available as a resource. However, this flexibility of depiction is counterbalanced by the fact that it is less minimal (i.e., requires more interactional work) than categorizing or pointing. While categorical reference can be achieved with only a word or two and ostensive reference can be achieved with a deictic word and a coordinated pointing gesture, depiction requires longer, more complex phrases and gestures.
Even though the customers in the two preceding cases appear to lack standard terms for the services they want and use multi-modal depiction to refer to them, they nonetheless display an orientation to the appropriateness of an official term. The customers do not simply depict the services: they first produce a term – tablet and book – but produce it as the wrong term or as an inadequate term. Customers thereby demonstrate their attempt to furnish the standard term even when they know they do not know it.
CONCLUSION
I have examined several ways in which gesture is used as a complement, supplement, and alternative to units of talk in the context of referring to objects at a face-to-face service counter. While most studies of referential practice and gesture focus on how deictic gestures are used to refer to things, I show that iconic gestures are also used in achieving reference. Furthermore, I have shown that categorization, ostension, and depiction are not simply alternatives for reference; they are ordered alternatives. At the print shop service counter, we saw that customers first try to use the standard terms for document services, but if these attempts fail or encounter troubles, customers tend to fall back on depicting the document services using iconic gestures with verbal descriptors and sometimes even props. Depiction thus appears to be primarily a reparative resource in referential practice.
We saw that whatever form of reference customers employed, they and the employees nonetheless displayed a preference for the print shop's official term for the service. This is consistent with Schegloff's (Reference Schegloff and Sudnow1972) and Sacks & Schegloff's (Reference Sacks, Schegloff and Psathas1979) argument that “names” are frequently a preferred reference form because they tend to be both “minimal” and “recognitional.” In examining referential practice in face-to-face service encounters, we found that nonverbal reference forms also appear to be part of this system of reference. However, in making this claim, the notion of “minimization” must be expanded to include not only utterance length, as Sacks & Schegloff Reference Sacks, Schegloff and Psathas1979 define it, but also the duration, complexity, and coordination work involved in using embodied forms of reference. Hence, for face-to-face interaction, Sacks & Schegloff's (Reference Sacks, Schegloff and Psathas1979:16) “preference for minimization” may be refined as follows: On occasions when reference is to be done, it should preferredly be done with the most minimal form, that is, the one that requires the least amount of interactional work.
Although we cannot predict which reference forms will work best in a situation without first considering its particular features – the configuration of objects and bodies in the physical setting, the location of the referable object, the presumed knowledge of the participants, the level of noise – we can nonetheless compare the relative affordances and constraints (Gibson Reference Gibson1979) of the various referential resources more generally.
The main affordance of categorical reference, including naming, is that it can be accomplished with little interactional work – often by uttering a single word. However, the success of a category or name is dependent on both parties knowing its meaning in advance, on the ability of the speaker to presume the recipient's state of knowledge, and of course on their ability to hear the utterance in the setting.
Ostensive reference, including pointing, can be achieved with slightly more work than categorizing: uttering a single deictic word together with a deictic gesture. However, ostension requires somewhat more interactional coordination than categorizing because it requires the recipient not only to hear the deictic term in the talk but also to see the gesturing limb and see the referable object all at the right times in the turn. Reference can fail if the recipient does not see the direction of the point or cannot find the object. This work is simpler if the object is ready to hand because then the gesturing limb and referable object are virtually in the same place and coupled. But an advantage of pointing over categorizing is that it can be used even when one or both parties do not know (or cannot say) the name or category. Pointing is especially useful when the participants do not share a common institutional vocabulary or even a common language. Pointing does, however, necessitate the presence of the referent, or a proxy for it, on the scene.
Finally, depiction can be used on any occasion when the official term is not known in common and the referent is not present. Therefore, depiction is well suited for repairing failed attempts at referring through categorization or ostension. But despite its flexibility, depicting will tend to require the greatest amount of interactional work. The verbal component may be significantly longer and more complex than that with categorizing or pointing. Similarly, iconic gestures may be more complex than deictic gestures and thus require greater coordination of hands and eyes. In addition, when physical props are involved along with the hands, depictions may require even more time to perform.
Whether at a service counter or any other face-to-face setting, speakers have very different methods at their disposal for getting a recipient to recognize a thing. If the most minimal resources are not possible given the features of the situation, then less minimal methods are always available. Furthermore, by combining resources, speakers can anticipate or repair possible troubles and ambiguities revealed by the recipient. Taken together, these kinds of referential resources constitute an extremely powerful and adaptable system for achieving mutual recognition in face-to-face interaction.
