Mobile Computing and Archaeological Research

The rapid growth of mobile computing in archaeology has been driven by the development of both specific hardware and software capacities and the expansion of the consumer market for tablet PCs and smartphones. Mobile digital devices have been commercially available since the late 1980s, but archaeological applications of mobile computing were relatively limited until the 2010 releases of the Apple iPad and the Samsung Galaxy Tab (Ceruzzi Reference Ceruzzi2012; Wallrodt Reference Wallrodt, Averett, Gordon and Counts2016). Powerful lithium-ion batteries, large screens, integrated cameras and GPS antennae, and an ever-expanding range of software options have all made such devices increasingly appealing for recording archaeological field data (Wallrodt Reference Wallrodt, Averett, Gordon and Counts2016:43). Many early adopters developed “bespoke” systems modeled on paper forms, particularly using Filemaker Touch and Filemaker Go (e.g., Ellis and Wallrodt Reference Ellis and Wallrodt2012; Gordon et al. Reference Gordon, Averett, Counts, Averett, Gordon and Counts2016; Motz Reference Motz, Averett, Gordon and Counts2016; Spigelman et al. Reference Spigelman, Roberts, Fehrenbach, Averett, Gordon and Counts2016; Wallrodt et al. 2015). Others have created forms in customizable applications such as Open Data Kit (e.g., Austin Reference Austin2014; Cascalheira et al. Reference Cascalheira, Bicho and Gonçalves2017). More recently, several teams have developed generalized, open-sourced platforms designed for recording and aggregating archaeological data (e.g., Dufton Reference Dufton, Averett, Gordon and Counts2016; Ross et al. Reference Ross, Sobotkova, Ballsun-Stanton and Crook2013; Ross et al. Reference Ross, Ballsun-Stanton, Sobotkova, Crook, Wilson and Edwards2015; Sobotkova et al. Reference Sobotkova, Ross, Ballsun-Stanton, Fairbairn, Thompson, VanValkenburgh, Averett and Gordon2016).

Studies of the effects of mobile computing on archaeological fieldwork have reported largely positive results. Numerous projects have reported that using tablets to record field data has allowed them to save time due to the fact that “born-digital” data do not require postfield transcription (Ellis Reference Ellis, Averett, Gordon and Counts2016:55–56; Gordon et al. Reference Gordon, Averett, Counts, Averett, Gordon and Counts2016; Motz Reference Motz, Averett, Gordon and Counts2016:82–83). Others report time savings and improvements in data integrity facilitated by the fact that using Mobile digital devices allows data standards to be applied at the “trowel's edge”—features such as drop-down menus and required fields that limit how much time is needed to correct typos and eliminate inconsistent terminology (Berggren et al. Reference Berggren, Dell'Unto, Forte, Haddow, Hodder, Issavi, Lercari, Mazzucato, Mickel and Taylor2015; Poehler Reference Poehler, Averett, Gordon and Counts2016; Rabinowitz Reference Rabinowitz, Averett, Gordon and Counts2016). Finally, teams that sync tablets to either on-site or lab-based servers have reported improvements in project management, through real-time monitoring of data entry (Morgan and Eve Reference Morgan and Eve2012; Opitz Reference Opitz2015).

While mobile devices have had their greatest impacts in the field, archaeologists have also employed them for recording data in laboratory settings, sometimes making use of hardware and software previously purchased and developed for use in the field (e.g., Austin Reference Austin2014; Bria and DeTore Reference Bria, DeTore, Averett, Gordon and Counts2016; Prins et al. Reference Prins, Adams, Homsher and Ashley2014). Published results suggest that mobile devices may also facilitate similar enhancements in laboratory data quality to those observed in field deployments. For example, Austin (Reference Austin2014) reports that OsteoSurvey, a collection of mobile forms developed in Open Data Kit for recording osteological data from burials at Deir el-Medina, Egypt, significantly reduced numbers of transcription errors and incorrectly skipped fields that were typical of paper forms.

While the primary focus of literature on mobile devices in archaeology has been on their conferred mechanical advantages, some researchers have also argued that the transformations they facilitate in the mechanics of data collection directly influence other elements of archaeological methodology, interpretation, and presentation. The ability to monitor data entry in real time has allowed some team leaders to chart patterns while they are emerging and adjust excavation methodologies to better capture them (Opitz Reference Opitz2015). Several scholars have also suggested that the use of mobile devices to streamline data management might “free up” archaeologists to devote more time to conceptual work (Ellis Reference Ellis, Averett, Gordon and Counts2016:55–56; Motz Reference Motz, Averett, Gordon and Counts2016:104). Others have demonstrated how digital media can enhance collaboration with local communities by opening up spaces for the inclusion of new types of knowledge in digital forms (Jackson et al. Reference Jackson, Motz and Brown2016; Sayre Reference Sayre, Averett, Gordon and Counts2016). However, there have been few fine-grained analyses of the effects of digital mediation on archaeological knowledge production.

Field Acquired Information Management Systems and the Proyecto Arqueológico Zaña Colonial

The Proyecto Arqueológico Zaña Colonial (PAZC) began in 2008, with the goal of examining the impacts of Spanish colonial forced resettlement (reducción) on the native populations of the lower Zaña Valley, in northern coastal Peru. The project adopted a tablet-based system for recording field data in 2014, seeking to achieve many of the same mechanical advantages that have been reported in previous literature. We partnered with the Field Acquired Information Management Systems (FAIMS) Project, which produces FAIMS Mobile—an open-sourced, off-line, multi-user data-collection platform that runs on Android devices (Ross et al. Reference Ross, Sobotkova, Ballsun-Stanton and Crook2013; Sobotkova Reference Sobotkova2013; Sobotkova et al. Reference Sobotkova, Ross, Ballsun-Stanton, Fairbairn, Thompson, VanValkenburgh, Averett and Gordon2016). Its core code handles the synchronization of data among multiple users, versioning, backup, export, flow control, and other automation. In turn, research projects tailor FAIMS core code using XML “definition packets” that create customized “modules” to accommodate specific data and workflow requirements (see Ballsun-Stanton et al. Reference Ballsun-Stanton, Ross, Sobotkova and Crook2018; Sobotkova et al. Reference Sobotkova, Ross, Ballsun-Stanton, Fairbairn, Thompson, VanValkenburgh, Averett and Gordon2016). Because the required programming skills and time commitments necessary to write such XML definition documents are much more modest than those for developing fully operational digital recording systems from scratch, archaeological research teams may be able to handle development without hiring third parties. FAIMS therefore provides many of the advantages of bespoke systems, at costs closer to those of off-the-shelf applications.

The FAIMS team developed the PAZC's first XML recording module (PAZC Excavation) in 2014. Following the release of FAIMS 2.0 in 2015, PAZC Excavation was updated, and it is currently available for download both through the FAIMS Mobile application and on GitHub. We ran PAZC Excavation on seven tablets—three Samsung Galaxy Tab 10.1’s, one Samsung Galaxy Tab 10.1 (2014 model), and three Samsung Galaxy Tab S8.4’s—all of which synced to an on-site server, running the FAIMS server platform. The latter consisted of an Intel NUC1i3RYH low-power computer with a 500 GB hard drive and 4 GB RAM, a 1 TB external drive for backup, and a wireless router and antenna to create a local area network for syncing in our field laboratory each evening. We employed the same hardware in the laboratory experiment described below.

Deployment yielded many positive results, as reported by Sobotkova and colleagues (Reference Sobotkova, Ross, Ballsun-Stanton, Fairbairn, Thompson, VanValkenburgh, Averett and Gordon2016)—particularly, a dramatic enhancement in data consistency and richness, which we attribute to the use of drop-down menus, required fields, and the ability to quickly review entered data each evening after tablets had synced to the FAIMS Mobile server. Following the completion of fieldwork, we pursued independent development of the PAZC Ceramics module for our 2016 laboratory season.

Ceramic Classification as Mediated Task

The macroscopic classification of ceramics is among the most long-standing elements of archaeological laboratory research, and it continues to be a vital component of many research programs even amidst the proliferation of archaeometric methods of characterization. Debates over its aims have figured prominently in the development of archaeological epistemologies (e.g., Ford Reference Ford1954; Ford and Steward Reference Ford and Steward1954; Gifford Reference Gifford1960; Hegmon Reference Hegmon1992; Hodder Reference Hodder1982; Lyman et al. Reference Lyman, O'Brien and Dunnell1997; Spaulding Reference Spaulding1953, Reference Spaulding1954). But the practical work of classification—the way in which archaeologists observe, measure, and label ceramics based on regularities of form, decoration, and other indices of production and consumption—has received little critical attention.

The media we employ for recording and aggregating ceramic data participate in concealing these processes. Methodological manuals (e.g., Orton and Hughes Reference Orton and Hughes2013; Rice Reference Fee, Pettegrew and Caraher2015; Sinopoli Reference Sinopoli1991; Velde and Druc Reference Velde and Druc1999) provide many useful suggestions about how typologies might be constructed and applied. In practice, however, much like the organization of labor in archaeological excavations (Leighton Reference Leighton2015, Reference Leighton2016), our classification procedures are generally treated as matters of common sense—steps that are so mundane and unvaried that they do not bear mention in publications and formal presentations of research results. Within the scope of projects themselves, spreadsheets and databases tend to be presented to users in a finished format, the design decisions behind them hidden from critique.

Unlike either the pioneers of archaeological seriation in the nineteenth century (e.g., Petrie Reference Petrie1899; Thomsen Reference Thomsen1848) or the great regional archaeological projects of the twentieth (e.g., Griffin Reference Griffin1949; Strong and Evans Reference Strong and Evans1952), today's archaeologists typically do not face the challenge of having to build classifications from scratch. In many cases, our first steps involve examining existing systems published in academic venues, gray literature, and online catalogs and deciding which elements to adopt and which to modify based on our research questions or obligations to clients and stakeholders. In this context, drawing on existing classification systems is more than simply an expedient way of building a typology: it is also the mechanism by which we compare data from distinct projects and mitigate their provincialism.

Adapting typologies entails steps that are seldom described in print: (1) deciding what variables should be recorded, (2) distinguishing between the values of different attributes (e.g., specific rim forms and decorative motifs), and (3) applying standards of measurement (e.g., deciding precisely where on the body of a vessel to use as a baseline for measuring neck height). Such distinctions are not self-evident in materials themselves (Orton and Hughes 2013: 81–92; Sinopoli Reference Sinopoli1991:50; Velde and Druc Reference Velde and Druc1999:262). They entail value judgments that must be objectified so that they can be applied in a reasonably standardized fashion. To achieve standardization, archaeologists employ representations of attribute values and measurement standards that include written descriptions, illustrations, and artifact photographs. Typically, experienced researchers then help new team members learn established standards and guide them as they adapt project typologies to label highly variable artifact forms.

Ceramic classification is mediated by both written and pictorial representations not only when typologies are first developed and learned but also as they are applied. Because researchers must choose between hundreds of distinct attribute values that are challenging to commit to memory and may be difficult to recognize, we often employ “codebooks” in which attribute vocabularies are defined and illustrated. Particularly in large teams where individual classifications are difficult to monitor, these representations become keystone arbiters of meaning before data are entered onto paper forms or directly into digital spreadsheets.

For new users, learning how to work through a codebook can be a time-consuming process. Thumbing through pages of attribute codes and entering corresponding values into a spreadsheet makes the user exceedingly aware of the codebook's functioning as a tool—what Heidegger (Reference Heidegger and Stambaugh2010) would call its presence-at-hand (Vorhandenheit). Through repeated use, tools may become more “ready-to-hand” (Zuhanden), functioning as extensions of the user's intentions and becoming less “objectively” present (Turner Reference Turner2005, following Heidegger Reference Heidegger and Stambaugh2010:69). Yet codebooks resist this tendency because their length and their physical separation from spreadsheets continue to demand a great deal of “knowledge-in-the-head” to operate (Norman Reference Norman2013).

Ceramic Classification in the Proyecto Arqueológico Zaña Colonial

The PAZC began ceramic analysis after the project's first extensive survey and excavation season in 2009–2010. The materials collected include precolumbian, Spanish colonial, and Republican period ceramics, dating from 1800 BC through the early twentieth century AD, with a heavy concentration in the late prehispanic (ca. AD 1100–1532) and early colonial (AD 1532–1650) periods. Our typological system drew heavily on Tschauner's (Reference Tschauner2001) well-illustrated ceramic guide from the neighboring Lambayeque and La Leche valleys. From Tschauner's list of 102 variables, we focused on 62 that were directly relevant to project research questions. To accommodate variables and attributes of Spanish colonial ceramics, we also drew on the ceramic database key of the Proyecto Arqueológico Tuti Antiguo (Wernke Reference Wernke2013) and the Florida Museum of Natural History's digital type collections. We employed the resulting system during the project's first laboratory season in 2010, as well as during subsequent analysis of materials excavated at the site of Carrizales in 2012, 2014, and 2015. Further description of the PAZC's ceramic typology appears in Torres Mora (Reference Torres and de María2011).

Prior to analysis, all sherds were labeled with alphanumeric strings consisting of the last two digits of the year of excavation, an inventory (bag) number, and an arbitrary, sequential artifact number—for example, 14.2321.3 (2014, bag 2321, artifact 3). Labels were drawn on ceramics using fountain pens, in either white or black ink, on top of small strips of nail polish. Attribute values were listed in a printed codebook where codes were paired with written descriptions and several variables (rim type, lip form, rim thickening, and base form) were accompanied by illustrations. To speed up data entry, each attribute was given a short alphanumeric code—for example, “brimmed plate” = BPL, “Late Horizon” = LHO, “serving vessel” = SER. When users encountered artifacts with new attribute values, they set them aside so that their distinguishing characteristics could be photographed, illustrated, and added to the codebook.

Each new project member received formal training in the coding system, beginning with a detailed explanation of each attribute and a full day working side by side with an experienced team member, who demonstrated proper procedures for taking measurements and recognizing attribute values. All team members were Peruvian archaeologists who had previous experience conducting archaeological fieldwork and ceramic analysis. During our first field season (in 2009–2010), the project director worked alongside project members during our training session to ensure that they correctly learned procedures for labeling and measuring artifacts. In practice, because the project had only three laptop computers at our disposal for data entry, team members worked in pairs, typically dividing their work by having one researcher focus on taking metric measurements of fragments while the other focused on labeling attributes related to surface treatments. This configuration enabled us to accelerate data recording using the hardware available to us at the time, but it also had unintended affective consequences, which we discuss below.

The resulting data were entered into digital spreadsheets. Each team used a laptop, a 500 g digital scale, calipers, and printed paper gauges for judging ceramic rim diameter. Work proceeded slowly at first, as teams worked with one sherd at a time, compared them with attributes listed in the codebook, and talked through labels with the team director. As they worked independently, team members continued to communicate verbally, with one reading out measurements and the other entering them into the spreadsheet. Though they had not been instructed to do so, team members also frequently discussed attributes with each other before entering data. In some cases, they taped images from the codebook to the laboratory wall, making it easier to view illustrations of types and their corresponding alphanumeric codes at the same time, = adding illustrations of new types to the wall as they appeared. As team members’ vision became increasingly trained to the categories and illustrations in the codebook, work accelerated, and they spoke less frequently. However, they still tended to consult with one another about tricky or ambiguous categorizations.

Our implementation of this system produced valuable data but also met with some common challenges, including missing fields, transcription errors, and inconsistent results between users. The prospects of addressing these shortcomings and scaling up our data collection through the multiplication of digital devices led us to pursue the development of the PAZC Ceramics FAIMS module in 2016.

Designing PAZC Ceramics

Our primary design goals for PAZC Ceramics were to create forms that limited data entry errors and interobserver variation while creating a more engaging (visual and haptic) experience for users that would ultimately make data collection more efficient. We sought to use two specific elements of the FAIMS interface to achieve these objectives: (1) drop-down menus, to restrict vocabularies and eliminate typographic errors, and (2) “Hierarchical Picture Galleries,” (scrollable “buttons” that incorporate visual representations of attribute values directly into the mechanism of data entry), to eliminate the ceramic codebook and add new illustrations of key attributes, reducing the number of steps necessary in the coding process and reducing ambiguity. Bria and DeTore (Reference Bria, DeTore, Averett, Gordon and Counts2016) report similar goals and advantages in their deployment of a FileMaker Go database for ceramic classification. In addition, we used tabs and nested menus to simplify the visual interface and group similar classification tasks, based in part on observations of how our teams practically divided measuring and classification tasks when using the two-person, laptop/codebook configuration. No elements of database structure were altered, to allow data collected using PAZC Ceramics to be seamlessly integrated into tables containing data recorded in previous field seasons.

During development, we pursued two tasks—producing new illustrations and images of attribute values to include in the module and the XML coding of the module itself. Image curation began with cropping and saving individual images of rim forms and base forms from a digital copy of our codebook. We created images of wares and surface conditions were based on photographs of artifacts taken in previous seasons and others contained in the Florida Museum of Natural History digital type collections (Figure 1). We created new illustrations of general form categories and firing patterns in Adobe Illustrator, and added descriptions of all code values to drop-down lists and scrollable menus.

FIGURE 1.

User selection by drop-down menu (upper left), followed by the first three tabs of the PAZC Ceramics module—Main Page, General Characteristics, and Firing. Images are clipped, and not all fields shown.

We then uploaded these images and referenced them with our XML code, which we developed using two templates—the FAIMS “User to Developer” Guide and existing code in the PAZC Excavation module. Following development, we tested the module and edited it with the help of the FAIMS development team.

Like all FAIMS modules, PAZC Ceramics is accessed through the FAIMS Mobile application, where it is selected from a list of modules loaded onto the individual tablet and/or available from download from a connected server. After it is opened, the user must select a name from the list of approved users managed on the FAIMS server (Figure 1, upper left panel). We chose to divide data entry between five tabs—“Main,” “General,” “Firing,” “Surface,” and “Measurements” (Figures 1 and 2), with all required fields highlighted in red. To simplify data entry for the two fields with the greatest number of possible attribute values (“General Form Category”—69 options; “Rim Form”—57 options), we organized drawings into nested menus that users accessed by selecting an overarching category (e.g., “Bowls” or “Olive Jars”), which would then open up submenus with more detailed options (e.g., “Cumbrous Bowl with Slight Constriction at Rim,” “Olive Jar Type 1”; Figure 3).

FIGURE 2.

Additional tabs of the PAZC Ceramics module—Surface, Measurements, Attachments, and Validation.

FIGURE 3.

Illustration of nested menus of images—General Form.

Above each field, users can access key features of FAIMS modules—annotation fields (symbolized by a pencil icon), certainty fields (symbolized by a “?” which allow users to quantize their perception of the reliability of observations, much in the way someone could scribble a question mark in the margins of a paper form), an “information” function that provides basic descriptions of each field instead of a longer and more entailed manual, and visual guides for measurements (Figure 4). In the “Files” tab, users can directly attach files to records, including photographs from tablets’ integrated cameras or images captured with other devices. The “Validation” tab provides a tool that checks for empty values in required fields and returns a list of those that are missing. Finally, the “Search” tab provides access to an open text search by which users can access all previously saved records. A complete version of PAZC Ceramics is available for download on GitHub, as described in the Data Availability Statement.

FIGURE 4.

Illustrations of measuring guides incorporated into the help section of the PAZC Ceramics module.

Overall, the structure of tabs and menus in PAZC Ceramics maintained continuity in the organization of classification, grouping together affectively similar tasks (e.g., measuring different vessel dimensions with calipers, judging color and surface decoration) in the same sections of the module. While some previous applications (e.g., McPherron and Dibble's [Reference McPherron and Dibble2002] E4 software) have found that linear structures of data entry employing conditional statements can be highly effective for improving speed and consistency, we believed that the great number of nonhierarchically organized attributes in our ceramic coding system would make such an option less practical than maintaining a tab-based structure and using drop-down menus to improve data integrity and validation to limit numbers of missing fields.

Evaluating Module Performance

We evaluated module performance using three methods—direct observation of participants using both recording systems, quantitative comparison of “test” data recorded using both systems, and anonymous surveys administered to users at the end of ceramic analysis. To allow for direct comparison of both systems, we began by having team members follow the spreadsheet/codebook method of data entry for one week, before they employed the new tablet-based system. All of these researchers were Peruvian archaeologists holding BA degrees, with extensive experience working in archaeological laboratory and field settings. Following our training protocol from previous seasons, each new team member was given a full-day, one-on-one training session by a more experienced team member. Then, before the tablet-based system was implemented, members of the design team worked one-on-one for one half day with project members to illustrate how to operate the tablets, the FAIMS application, and the PAZC Ceramics interface while also explaining correspondences between the laptop/codebook system and PAZC Ceramics.

During data entry sessions, we observed team members at work and kept time-on-task logs to chart their efficiency. After the trials were complete, all users worked with three “test” bags of material, containing a total of 109 sherds. Users analyzed test materials in two sessions—one using the spreadsheet/codebook system, the other using PAZC Ceramics. To reduce the chance that researchers would remember details about individual artifacts from one session to the next, we had them wait one week between analyzing the test bags using each different recording system. To guard against the possibility that memories of specific sherds might exaggerate the precision and accuracy of data collected with tablet-based recording, we had team members conduct the tablet-based trial first. We evaluated all results against a single “answer key” that contained accepted values for each artifact. It is worth underscoring that, despite these controls, the results we present here do not represent a strict, two-by-two product trial that isolates the impacts of the PAZC Ceramics module. Rather, they reflect the performance of PAZC Ceramics using the specific hardware and labor configurations listed above, in comparison with data recorded using our laptop-and-codebook system.

Efficiency of Data Entry

We observed notable improvement in the efficiency of data entry using PAZC Ceramics and one-person teams (Figure 5). To evaluate the efficiency of data entry using each configuration, we aggregated per-sherd time-on-task data and calculated an average number of sherds analyzed per person, per hour. Data entry using the two-person spreadsheet-and-codebook-based system was “achingly slow” on the first day, according to one user—averaging 2.58 sherds/person/hour across seven groups. However, speed nearly doubled to 4.93 sherds/person/hour on day 2 and improved again to 6.79 and 6.70 sherds/person/hour on days 3 and 4, respectively. In contrast, data entry using the one-person team and tablet-based system began at 9.79 sherds/person/hour on day 1, remained virtually the same (9.68) on day 2, and dropped modestly to 8.76 and 8.57 sherds/person/hour on days 3 and 4, before improving in the final three days of data collection.

FIGURE 5.

Team members at work using the spreadsheet and codebook system (left) and the PAZC Ceramics FAIMS Mobile module (right).

In addition to an overall improvement in the efficiency of data collection and differences in the shape of the learning curve, we also observed greater deviation in data entry speed using the tablet-based system and one-person teams configuration. Indeed, by days 3 and 4 of our trials, some of the least efficient users of PAZC Ceramics were entering data at rates equivalent to (and in some cases slower than) teams employing the spreadsheet and codebook system.

We attribute these trends to both the simplified interface of PAZC Ceramics and the reorganization of our research team. In their survey responses, users attributed difficulties using the laptop/codebook system in the first few days to the challenges of distinguishing between attribute values, which were often described only in writing in the codebook and required them to spend considerable amounts of time thumbing through pages and deciding how codes corresponded to artifact features. Users attributed improving efficiency on days 2, 3, and 4 to their increasing familiarity with these attributes, memorization of key codes, and the fact that they began to divide the work of classification and data entry into more repetitive tasks between team members. The flatness of the learning curve among tablet users may reflect the fact that all had some prior experience with the codebook before using the PAZC Ceramics module. However, survey responses (discussed below) also indicate that users thought the tablet-based system of data entry was more efficient, and they attributed this improvement to the more intuitive character of the PAZC Ceramics module.

The greater variation of data entry rates we observed among tablet users in comparison to the codebook/laptop teams was unexpected. We interpret it as an indirect product of isolating team members from one another. In the two-person teams, differences in individuals’ efficiencies were somewhat averaged out. In contrast, solitary data entry on tablets exposed variation between individuals’ data entry speeds (Figure 6). In turn, this shift transformed the atmosphere of the laboratory. Before, it was filled with the chatter of teams discussing individual sherds, music playing over the radio, and the occasional sing-along. During the tablet trials, the lab was much quieter. Users put in their earphones and turned on music or podcasts on their personal mobile devices or on the tablets themselves. Two users in particular seemed more likely to lose focus under these conditions, entering data less and less efficiently as the days wore on. Yet, the cumulative effects of multiplying devices, dividing up our research teams, and using PAZC Ceramics yielded a modest improvement in the efficiency of data entry.

FIGURE 6.

Rates of data entry by sherds/person/hour using the two systems.

Because the tablet models we used are devices that are scaled and equipped for the immersive media experiences of individual consumers, it is unsurprising that they tended to isolate users from each other—particularly people who were already habituated to using tablets for personal entertainment. However, our decision to have individuals rather than two-person teams operate the tablets also contributed to these effects.

Error and Data Consistency

To evaluate the impact of our new recording system on data consistency, we focused on data recorded in 13 fields—“Type,” “Period,” “Function,” “Style,” “General Ware,” “General Form,” “Firing,” “Surface Condition,” “Paste Color,” “Rim Form,” “Exterior Decoration,” “Interior Decoration,” and “Slip Color.” All of these fields were filled using free text entry in the spreadsheet/codebook system and managed with drop-down menus in PAZC Ceramics. We excluded fields with highly repetitive values (e.g., inventory numbers, researcher names) and fields with low rates of usage (e.g., handle form and base form). To evaluate data consistency, we compared user-entered values to the answer key.

The results demonstrate that our deployment of PAZC Ceramics yielded a significant reduction in the number of missing fields—42 (or 0.175% of 11,445 entries) versus 106 (2.16% of 4,905 entries) using the codebook-based system (Table 1). The new system's effects on data consistency were more variable. As illustrated in Table 1, among 10 of the 1 fields, rates of divergence from the answer key were either comparable or modestly higher using the tablet-based system. Our deployment of PAZC Ceramics yielded more consistent data in three fields—Firing, Interior Decoration, and Exterior Decoration. While this improvement may be partially attributable to the addition of photographs and illustrations to represent corresponding field values in PAZC Ceramics, it is worth noting that users entered inconsistent data at similar — and in some cases, even higher — rates in PAZC Ceramics for other fields whose values were also accompanied by photographs and illustrations (specifically General Ware, General Form, Surface Condition, and Rim Form). Standard deviations of metric measurements recorded on the tablets were also moderately higher. The average standard deviation for sherd weights entered via spreadsheet/codebook was 0.22 g, and for sherd thickness, 0.44 mm; while corresponding SD rates for PAZC Ceramics were 0.24 g and 0.60 mm.

TABLE 1.

Errors and Missing Fields, by Method of Data Entry.

Some of these inconsistencies in data entry point to underlying issues of ceramic classification, rather than unique features of the PAZC Ceramics interface or project labor organization. Across both systems, data consistency tended to be higher for fields—such as Type, Period, Style, and Exterior and Interior Decoration—with relatively few options and high frequencies of “unknown/other” or “unobservable” values. In the General Ware field, certain types of highly distinctive vessels—olive jars and majolica, in particular—tended to be correctly identified at higher rates, while other categories such as “slipped blackware” and “slipped brownware” were frequently confused with one another, perhaps highlighting the imprecision of the ware concept itself (Rice Reference Rice1976) and/or users’ difficulties judging color.

In general, we observed high rates of inconsistent data among fields requiring color judgments—Firing, Paste Color, and Slip Color. The addition of a color guide for the Firing field in PAZC Ceramics was associated with a modest reduction in inconsistent Firing entries in our tablet-collected data. However, 42% of Firing values were still distinct from those listed in the answer key, indicating that users found it quite challenging to match firing characteristics observed in our diverse collection of domestic ceramics to those represented in the coding system.

User Survey

Questionnaires provided another valuable source of information about our deployment of PAZC Ceramics. We administered them anonymously, with respondents filling in their answers directly into Google Forms on a laptop in a private room. We asked users to directly compare and rate PAZC Ceramics and the laptop/codebook configuration, as well as to provide comments on several aspects of the digital module's performance. To reduce bias in favor of the tablet-based system, we did not survey team members who had been directly involved in developing PAZC Ceramics. We collected six questionnaires in total—a sample size too small to allow for statistical analysis but one that provides complementary insights to our quantitative analysis.

In the questionnaires, users gave the tablet-based program generally favorable ratings. Five out of six agreed that the new system was easier to learn than the old one, reinforcing the results of our time-on-task records. They also consistently rated PAZC Ceramics as more engaging than the spreadsheet-and-codebook-based system of data entry. When solicited for comments about their favorite elements of PAZC Ceramics, users mentioned that they appreciated the visual guides embedded in the module—particularly those for firing and rim types—as well as the ability to tag all entered data with notes and the “validate fields” feature, which allowed users to check their forms for missing data.

While all six users claimed that they preferred the tablet-based system, their comments also highlighted deficiencies and suggestions for improvement. Some users requested that images be enlarged and/or clarified, while others noted that they would have appreciated access to Bluetooth keyboards, because they found it cumbersome to enter data on a touch screen. Many found it comparatively challenging to edit data and to make comparisons with previously entered values in PAZC Ceramics. This is an unsurprising result, given the fact that the PAZC Ceramics interface keeps the user narrowly focused on the individual artifact and does not allow data to be visualized in spreadsheet format without the writing of custom SQL queries. Users also found it challenging to make comparisons between sherds because the search function performed poorly after several hundred records had been generated, making it difficult for them to look up previously entered data.

Many users also expressed a desire to have more control over the process of declaring new attributes and adding them to the system. In our implementation, we restricted users’ ability to add attribute values that emerged in the course of analysis (e.g., new rim types, decorative patterns, etc.)—in part because we believed that updating the module in real time might create problems for data management. Instead, we instructed users to tag artifacts that did not seem to fit the existing typology using the “uncertainty” bars, take photographs of them using the tablets, and write notes about their distinguishing characteristics. The result was a classificatory system that was in practice more rigid than the former method of data entry but also a greatly expanded database of artifact photographs documenting the collection's diversity. On a positive note, this rigidity helped us avoid the unmitigated proliferation of types and the collapse of data integrity that can result from allowing users to select “other” as an attribute value.