Primary data collection is an integral part of clinical research, and its quality is fundamental to the quality of any results obtained. Traditionally, it has been performed on paper, but this has limitations as the data size grows. In particular, because research data are often stored electronically to facilitate analysis, using paper necessitates copying that data to the electronic system with methods that tend to be expensive, slow, and/or error-prone (Reference Goldberg, Niemierko and Turchin1;Reference Kawado, Hinotsu and Matsuyama2).
Electronic data collection (EDC) is a viable alternative that does not require copying between different media, thereby solving both the speed and accuracy issues. The EDC data collection process shares many similarities with data collection on electronic health record systems (EHR), and the latter have been studied extensively (Reference Car, Black and Anandan3–Reference Black, Car and Pagliari5). While their benefits to secondary data analysis such as research and audits are evident, EDC and EHR have been studied in various contexts using different devices (laptop, tablet, PDA) and have been found to impact clinical workflows in primary data collection (Reference Haller, Haller and Courvoisier6;Reference Walther, Hossin and Townend7). Plenty of evidence shows that EDC decreases data entry errors, especially because data collection software can hint toward missing or incorrectly entered data (Reference Häyrinen, Saranto and Nykänen8;Reference Coffey, Wurster and Groner9), but there are conflicting results regarding its effects on time required by clinicians to enter data compared with paper methods (Reference Walther, Hossin and Townend7;Reference Coffey, Wurster and Groner9;Reference Sanders, Read-Brown and Tu10). Some authors have discussed the possibility of a new type of error arising from EDC, such as loss of overall picture and inadvertently clicking a nearby but undesired option (Reference Ash, Sittig and Dykstra11;Reference Campbell, Sittig and Ash12).
The technological landscape changes rapidly, and this necessitates re-examining EDC using next-generation devices. For example, one study that looked into the use of tablet PCs for data collection have praised the tablets’ mobility and ease of use, but ran into problems that they were too bulky, that the screen sometimes froze and that text recognition did not always work (Reference Carlson, Catrambone and Oder13). Since then, a new generation of tablets, such as the Apple iPad®, have been made available that improve on many of these usability aspects, and significantly change the human-computer interaction. They also have some drawbacks, such the potential for loss or theft which could lead to data compromise, and the fact that they may be better suited to data consumption than entry, especially for free-text typing (Reference Archibald, Macdonald and Plante14). Few studies examine such tablets as data entry methods for research, despite the fact that surveys reveal a significant number of clinicians who use them and are familiar with them (Reference Boruff and Storie15).
At the Psoriatic Arthritis Clinic at the Toronto Western Hospital, patient care data are collected and stored on paper charts; parallel to that, clinical research data are collected on paper forms and subsequently copied onto a detailed electronic database, based on Oracle software, that includes demographic information, clinical history, medications, physical examination, laboratory investigations and radiographic assessments (Reference Gladman and Chandran16). The demographic, clinical history, medications, and physical examination portions are completed by the physician during the patient encounter and amount to >400 unique variables. When an error or omission is discovered, the form is sent back to the physician, who by then might not be able to remember the correct value. This leads to significant delays in the data entry of forms that have omissions, which can be in the order of months, and may compromise data quality. To solve this issue, we developed an electronic application for data collection on iPads, and in this paper we describe the application and a method for its evaluation.
METHODS
The Application
We designed and developed the Web application in-house using free and open source software (Java 1.6, Apache Tomcat 7, MySQL 5.5). The period from conception and design to introduction in the clinic lasted six months and required 350 developer-hours. To speed up development and allow expandability, several user interface components were standardized and re-used repeatedly, and as a result, new forms can be created and added with relative ease provided a programmer is present. This is particularly relevant for expanding the scope of the application to other diseases and specialties. Once completed, it was run on a server hosted by the information technology department and was accessed by iPads over Wi-Fi in a connection secured using the Transport Layer Security protocol. With this design we ensured that all patient data was stored on the central server, and consequently the risk for data compromise would be minimal in case of device theft or loss.
The application divided visits into two types: a baseline visit form assumed no previous data for the patient, whereas a follow-up visit form meant that previous data, such as the history of pharmacotherapeutic drugs prescribed to the patient, had to be fetched and made available. Forms are in turn divided into pages by categories (e.g. demographics, medical history, physical examination, etc.), which physicians can go through in any order. Once the physician finishes a page, they click “save”; the data is then saved immediately, and any missing fields are highlighted. The physician can then choose to complete those fields or move on to other pages. However, all missing fields on all pages must be filled before a form can be submitted.
Evaluation
The study was approved by the University Health Network research ethics board. Psoriasis and psoriatic arthritis patients who consented were enrolled in the study. They were randomly assigned to two groups, the paper arm whose data were collected on paper forms, and the iPad® arm whose data were collected on iPads. The patients were then assessed by seven physicians who were divided into “experienced” and “new” according to their familiarity with the paper protocol assessment form. Three physicians had more than 12 months of prior experience and were classified as “experienced.”
The time physicians spent with patients was used as an indicator of how the electronic system affected the clinical workflow. Little difference in patient-encounter time may indicate that the electronic system is just as efficient as paper, and thus this variable was used as the primary indicator. Visits were timed starting when the physician entered the examination room with the patient and ending when the physician left the room. In addition, the physicians were asked to record the time taken for data entry alone.
To quantify the incidence of missing data in the paper arm, the number of missing fields was counted for every visit after the clinic. A missing field is one that is left empty when a response should have been entered. The missing fields were categorized into detectable/nondetectable depending on whether they can be detected if they had occurred on an iPad®.
Finally, an important aspect of using iPads for data collection is that clinical data would be immediately available on the database. To measure the gain from this feature, the number of days between a clinical visit and its data entry was recorded.
RESULTS
A total of 106 patients participated in the study, with a mean age of 52 years and mean disease duration of 26 years for psoriasis and 16 years for psoriatic arthritis (Table 1). Fifty-three of them were enrolled the iPad® arm, and fifty-three in the paper arm.
Demographic and Disease Characteristics of Study Subjects (n = 106)
*No. of patients with at least unilateral sacroiliitis.
PASI, Psoriasis Area and Severity Index.
Only follow-up visits were used in the time comparisons, because baseline visits were relatively rare, and generally take longer than follow-up visits (Table 2). Five follow-up visits were missing encounter timing data and were excluded from that analysis, and six were missing data entry time and were excluded from data entry time analysis. One experienced physician elected not to take notes directly on iPads; rather he jotted down a summary of each visit and entered it on the iPad® afterward, and, therefore, his patient encounter time in the iPad® arm was also excluded. Taking those into account, the number of visits used for each type of time analysis is listed along with the results in Table 3.
Distribution of Follow-up Data Collected on Paper and iPads
Duration of Patient–Physician Encounter and Time Spent Entering Data (Minutes).
The average patient-physician encounter time for iPads was 46.5 minutes compared with 37.2 minutes for paper, and this difference was significant with p = .0035 using two-tailed t-test (Table 3). However, when taken separately, the experienced physicians group showed only a trend toward statistically significant difference between the two data collection methods (p = .059) reflecting relatively small sample size.
Experienced physicians reported that data entry on iPads took on average 2.6 minutes longer than on paper, and this was statistically significant (Table 3). New physicians, however, reported no significant time difference.
With regards to incomplete entries, all forms, baseline and follow-up, were used and we found that 3.34 (SD 3.24) missing fields occurred on average per paper form. Of those, 2.19 (SD 2.32) would have been detected if they had occurred on an iPad®. These included missing text fields, missing choices and inconsistent heading-subheading entries, and their numbers were similar across both physician groups.
Of the fifty-three paper forms collected, thirty-four were copied to the psoriatic arthritis database within an average of 7.3 days (SD 8.0) after the clinic. Ten protocols were still pending entry an average of 33 days (SD 16.5) after the visit. The remaining nine forms with missing data on time required for completion were excluded.
DISCUSSION
Methods to improve the efficiency and quality of data collection in a clinic setting are an area of considerable interest to clinicians and researchers. Here we evaluated the traditional paper-based data collection to EDC using an iPad® application. We found that the iPad® application extended the length of patient encounters, but collected better quality data and eliminated the delay between a clinic visit and its data entry. Other studies on touch-screen devices reported similar findings for error reduction, but had mixed results regarding documentation time (Reference Walther, Hossin and Townend7–Reference Sanders, Read-Brown and Tu10;Reference VanDenKerkhof, Goldstein and Rimmer17–Reference Cole, Pisano and Clary19).
In our study, the increase in quality was due to not only the elimination of manual transcription errors, but also feedback on missing fields that is provided by the application to physicians in real time; the latter ensures that data collection is complete before the patient leaves the clinic. We also found that by eliminating the delay between collecting data on paper and copying it onto the database, the application significantly improved the efficiency of data handling in the clinical workflow. This is especially true because 25 percent of the forms in the paper arm were still pending entry into the database more than a month after the clinic visit.
The application was able to detect more than half of the missing fields that were made on paper. While this greatly benefits the quality of research data, it may have also inadvertently contributed to the increase in patient–physician encounter time because it forces physicians to answer all fields correctly during the visit. Moreover, paper follow-up forms, unlike the application, contained information for four visits side by side and this gave physicians the opportunity to simply copy data from prior visits when short on time. Because this was not possible on the iPads, it may have further contributed to the observed difference in encounter time.
Physicians reported little difference in the time needed to enter data on iPads compared with on paper, and this was especially true for the new physicians group, who are younger and more likely to have used an iPad® in the past. This finding comes in contrast to the larger overall increase in patient encounter time, and may indicate that physicians perceive data entry to be faster than it really is. It could also mean that part of the difference in time is due to measurements often skipped on paper.
The patients who participated in this study varied in their severity of illness and presented cases of different complexity. Thus the difference in encounter time between the iPad® group and the paper group may be affected by the assignment of patients to groups. However, because the assignment was random, we believe that the variation among patients had little effect, if any, on the observed difference in time.
To conclude, we have created an iPad® application that allows for the collection of more accurate data by reducing the incidence of missing fields and eliminating errors made during data entry. It also eliminates the delay between a clinical visit and its data being available electronically. However, it increases patient–physician encounter time.
CONFLICTS OF INTEREST
The authors have no conflicts of interest to report.
