Tools in the Virtual Psychological Test Library

A wide range of psychological instruments and screening tools are now available in electronic formats, including files to be downloaded and completed by hand or automatic scoring and reporting facilities. Many of the tools commonly used by school psychologists and counsellors in Australia have been adapted in some shape or form so that practitioners can access online testing formats.

Some questionnaires that can be accessed free of charge on the internet directly from those who developed these instruments include the following:

A Trial in Progress: Phase 1

Currently, a group of school psychologists in Canberra is implementing a trial of online assessment tools. The trial does not attempt to investigate the effectiveness of online versus traditional formats of assessment tools in an empirical sense. Rather, its purpose is to help determine whether digital formats of assessment tools could complement existing test resources by providing qualitative feedback about usability, identifying issues and challenges of their use in school settings, and offering suggestions for staff training and support needed for successful implementation.

School psychologists recruited for the trial work across a range of school settings and are self-reported to be at various levels of computer proficiency. Tools already selected by the school psychology service and already being used in pencil-and-paper versions were chosen for the trial. Phase 1 started in 2014 and involves access to the Conners 3rd Edition and BRIEF assessments described above. Each of these requires a log-in on the individual publisher's portal. For both tools, a multi-user account was created that enables an administrator to set up individual users, set permissions to maintain privacy and purchase forms, and allows all users in that account to access the pool of tests purchased.

School psychologists volunteering in the trial were invited to sessions where individual accounts could be set up. Basic mechanisms of the platform were explained on a Smartboard, and iPads allowed the volunteers to experiment and practise navigating the portals. Trouble-shooting sessions also allowed for brainstorming and problem-solving. The volunteers reported that they found this process very helpful, and opportunities for frequent support helped them to familiarise themselves with the interface and feel confident in using the tools when they returned to their schools.

Reflections Thus Far and Ethical Considerations

In the first week, one school psychologist wrote: ‘I did my first online Conners with a teacher — so good . . . easy to use from the link and it then generated a 22 page report online . . . that just saved me at least 1–2 hours!’ Another said: ‘the feedback I got [from the teacher] was that it was so much better than using the pencil-and-paper versions . . . she appreciated that she could go back and change an answer easily if a mistake was made, and that it was a lot quicker than the paper version’. While only anecdotal, these comments provide some insight into the potential benefits for both school psychologists and counsellors and respondents. It is acknowledged that these computerised reports need to be used cautiously with respect to ethical and practice guidelines. Indeed, the recent Practice guide for the assessment of school-age students in educational contexts (Australian Psychological Society & Speech Pathology Australia, 2014) specifically cautions that ‘computer generated reports that provide test scores and basic interpretations of results for the practitioner are not [to be] incorporated directly into reports' (p. 19).

In addition to the notions of increasing efficiency and productivity in school psychology practice previously mentioned, such online assessments administration and scoring promise to offer greater integrity and accountability by maintaining a greater degree of standardisation and reducing human error in transposing scores. The authors of the BRIEF assessment tool estimate that hand scoring takes 15 to 20 minutes to complete, and with multiple informants (e.g., parent/teacher), hand scoring could take up to an hour for just one instrument. If this aspect of school psychologists' and counsellors' work is more automated, it allows practitioners to spend more time researching, conceptualising the case, conducting more comprehensive assessments and assisting more students.

In addition, discussion among the group of volunteers has also suggested the benefits of developing deeper understanding of the instruments by increasing learning in situ and hence building professional capacity. The comments provided in computerised reports can be useful and informative, providing practitioners with explanations and often helpful strategies. However, it is important to be aware of conclusions such as those drawn by Butcher, Perry, and Hahn (Reference Butcher, Perry and Hahn2004) that for the majority of computer-based reports, only 60% of the report narratives are appropriate. Interpretations that are more concise as opposed to lengthy interpretations are generally thought to have more valid interpretations. With the possibility that up to 40% of comments are deemed inappropriate for particular clients, school psychologists and counsellors must continue to maintain responsibility for interpretation of computer-based reports to avoid misdiagnosis. This emphasises the importance of the statement made in the Practice guide for the assessment of school-age students in educational contexts (Australian Psychological Society & Speech Pathology Australia, 2014) about directly incorporating computerised interpretations into reports. Clearly, these reports require analysis and scrutiny and the application of clinical judgment to individual cases in the context of the client's history and other assessments; however, the convenience of having relevant information immediately is a bonus for a time-poor workforce.

Furthermore, with the adoption of online testing modes, interesting ethical questions arise — if school psychologists and counsellors have a greater selection of assessment instruments in their virtual psychological test library, could they make better clinical decisions about the best tools to use for individual clients? That is, rather than be limited by the tools they can access or the time they have to administer and score tests, perhaps decision-making could entirely focus on best clinical practice and eliminate consideration of more practical aspects such as available resources (Pfohl & Jellins, in press).

The Trial Continues — Phase 2

The next phase will involve trialling Pearson's interactive administration of the Wechsler Intelligence Scale for Children — Fourth Edition (WISC-IV; Wechsler, Reference Wechsler2003), known as Q-interactive (available through the Australian version of the website http://www.helloq.com.au), as well as the Wechsler Preschool and Primary Scale of Intelligence — Fourth Edition (WPPSI-IV; Wechsler, Reference Wechsler2012) reporting software available through Q-global. The Q-interactive system has revolutionised the way that psychological assessments can be administered via tablets such as iPads. Briefly, the examiner is equipped with an iPad that serves multiple functions and contains the manual, proforma embedded with starting points, basals, reversals, ceiling rules and response options, a stopwatch, a microphone to temporarily capture verbal responses, and mechanisms to write answers and observations. The student's iPad functions as a stimulus book and is connected to the examiner's iPad via Bluetooth technology to facilitate the scoring of some items. The only additional materials used in the assessment process are the blocks and response booklets for paper-and-pencil tasks.

Despite the more automated nature of some aspects of the assessment, clinical judgment continues to play a critical role in these digital formats, and this is evident in the options available when testing a student suspected of having a delay or when ‘testing the limits’. Wagner (Reference Wagner2013), a school psychologist based in the United States who was involved in testing this product at the development stage, claims that digital formats allow the examiner ‘to shave up to about 30% of [the] set-up, scoring, and collating time . . . [and] . . . a little more freedom to pay attention to the child, not the test’. In his newsletter article, Wagner (Reference Wagner2013) summarises some advantages and disadvantages of interactive assessments:

In relation to this new mode of administration, Pfohl and Jellins (in press) caution that while the publisher has provided equivalency data on digital adaptations, it will be important for the integrity of the instruments and the confidence of practitioners for this to be independently evaluated.

Apps to Gather Additional Data

Apps are one of the fastest growth areas of technology (Pfohl & Jarmuz-Smith, Reference Pfohl, Jarmuz-Smith, Harrison and Thomas2014). Although most of these apps are yet to be subjected to empirical research, many have evolved from a strong theoretical base. A handful of apps to support school psychologists and counsellors in their observations of students have emerged, with some developed by school psychologists in the United States. While behavioural observations may not be a significant part of school psychologists' and counsellors' roles in Australia, as is commonly regulated in the United States, many of these techniques and the information gathered may be used in the context of comprehensive assessments. In principle, observations are seen as a necessary part of assessment in Australia (Australian Psychological Society & Speech Pathology Australia, 2014) and these apps may help to gather information. Some examples include:

Further Ethical Implications and Conclusions

While general principles and professional codes of ethics apply to all aspects of practice, explicit guidelines and recommendations with reference to school settings will help school psychologists and counsellors navigate the integration of recent developments in technology. The APS Ethical Guidelines for Psychological Assessment and the Use of Psychological Tests (2013b), which accompany the Code of Ethics, address the management of online assessment, with caveats on a number of potential issues that might arise in using online assessment tools, including tests' psychometric properties, informed consent in an online context, data security, and interpretive issues.

A number of issues that have important clinical implications have been raised in this article, including the potential for technology to help school psychologists and counsellors meet service needs, and the possibility that access to a greater variety of tools could enable practitioners to make better clinical decisions. These implications highlight the responsibility of practitioners to consider electronic options.

An additional example of an ethical consideration that emerged while preparing for the trial concerned the principle of confidentiality. The team extensively investigated responsibilities related to the ability to enter client information into online tests. These tests are stored on servers outside the computer known as a ‘cloud’ (Pfohl & Jarmuz-Smith, Reference Pfohl, Jarmuz-Smith, Harrison and Thomas2014). Publishing companies frequently reference the American laws of Health Insurance Portability and Accountability Act (HIPAA) and Family Educational Rights and Privacy Act (FERPA) to advertise their level of security, with password protection and encryption in place. The team preparing for the trial consulted with the Australian test distributors, the Australian Psychological Society and the employer. While it was deemed safe to use names during testing with informed consent, it was decided that an additional safety precaution would be taken and student initials rather than full names would be used for the trial. This was considered to reduce associated risks provided practitioners agreed to print completed reports and add them to student files immediately so as to avoid any confusion of student identity.

Bush, Naugle, and Johnson-Greene (Reference Bush, Naugle and Johnson-Greene2002) warned that due to cognitive and emotional limitations of many individuals seeking psychological assessments, when psychologists adopt new technologies, they ‘have a responsibility to anticipate and negotiate . . . unique ethical challenges’ (p. 536). But with careful consideration of ethical implications, and importantly with support from colleagues and professional associations, school psychologists and counsellors can enter the digital age with confidence and implement online assessment tools to enhance and extend the delivery of psychological services in Australian schools.