1 Introduction
Over the past decade, there have been substantial advances in the field of computer-aided decision support for the early detection of cancer (Haque et al., Reference Haque, Mital and Srinivasan2002). At the same time, advanced biological characterization and innovative imaging modalities have provided novel approaches to determining the diagnosis and prognosis of brain tumours (Howe & Opstad, Reference Howe and Opstad2003; Armstrong et al., Reference Armstrong, Cohen, Weinberg and Gilbert2004). Early efforts in the implementation of interactive decision support systems (DSSs; Underwood et al., Reference Underwood, Tate, Luckin, Majós, Capdevila, Howe, Griffiths and Arús2001) for brain tumour diagnosis have identified the need to combine biomedical automated pattern recognition techniques (Tate et al., Reference Tate, Underwood, Acosta, Julia-Sape, Majos, Moreno-Torres, Howe, van der Graaf, Lefournier, Murphy, Loosemore, Ladroue, Wesseling, Bosson, Cabanas, Simonetti, Gajewicz, Calvar, Capdevilla, Wilkins, Bell, Remy, Heerschap, Watson, Griffiths and Arus2006) and data from magnetic resonance imaging (MRI) and advanced imaging techniques, such as spectroscopy (MRS), which provides information on the biochemical profile of the tumour (Luts et al., Reference Luts, Heerschap, Suykens and Van Huffel2007). Although it has shown much promise, MRS has not been widely used in clinical practice partly because of the difficulty of developing robust classifiers from small patient numbers and the challenge of providing DSSs acceptable to clinicians. Most studies have concentrated on adult cases and there is an unmet need for the application of such approaches to children and young adults, a group where brain tumours are the most common solid tumours (Packer, Reference Packer1999). However, the problems of small patient numbers and a large number of diagnostic classes are particularly severe in this patient group.
The design and implementation of such an interactive DSS should reflect important elements and steps of the decision-making process performed by a clinician. Such a process is usually accomplished at different levels of abstraction and is supported by the clinical evidence available to the clinician at the time. In the case of brain tumour diagnosis, the diagnostic process initially involves the structured combination of clinical information, tumour biomarkers and imaging modality indicators to formulate a list of potential diagnoses, often termed a differential diagnosis. In this context, elements of this process should be correctly represented in the level of systems requirements for the design and implementation of such DSSs. This is best achieved with the support and involvement of actual users in the processes of requirements elicitation and design of such a system. As such, Participatory Design (Schuler & Namioka, Reference Schuler and Namioka1993) is an approach to interactive systems design that seeks to actively involve end users in the various stages of the design life-cycle of a system, ensuring that the final system meets users’ needs. Therefore, Participatory Design is a very attractive way of successfully capturing the requirements of diagnostic DSSs, as it encourages ownership of the development process by users (clinicians), while they are addressing their own needs (Dredger et al., Reference Dredger, Kothari, Morrison, Sawada, Crighton and Graham2007).
A distributed DSS for the diagnosis and prognosis of brain tumours, termed HealthAgents, has been previously described (González-Vélez et al., Reference González-Vélez, Mier, Julià-Sapé, Arvanitis, García-Gómez, Robles, Lewis, Dasmahapatra, Dupplaw, Peet, Arús, Celda, Van Huffel and Lluch-Ariet2009). In this paper, we employ a participatory design approach to develop an interactive clinical user interface for childhood brain tumours within HealthAgents. In particular, we discuss the clinical need and context of developing interactive elements for an interface that facilitates the classification of childhood brain tumours, for diagnostic purposes, as part of the HealthAgents European Union project. We provide the argument for the clinical need for such a system and the constraints that should be imposed upon the building of classifiers for childhood brain tumours. The constraints are based on tumour type, patient age and tumour location. To illustrate the strategy and demonstrate its potential, classification results are presented from a small cohort of children with cerebellar tumours. The HealthAgents project (The HealthAgents Consortium, 2009) has been developing a distributed DSS, based on software agent technologies, in order to provide a set of automated classifiers for the diagnosis and prognosis of brain tumours.
2 The need for decision support in the diagnosis of childhood brain tumours
Brain tumours are the most common solid tumours in children and young adults. They have a poorer outcome than many other forms of cancer in this age group and the survivors often live with a significant burden of neurological disability (Pizzo & Poplack, Reference Pizzo and Poplack2002). The diagnosis and treatment of brain tumours has considerable ongoing challenges for clinical medicine. Magnetic resonance imaging is the mainstay of non-invasive investigation for these patients, yielding important information on location, size and heterogeneity of the tumour. However, MRI has a limited ability to distinguish between different tumour types and provides little information on the likelihood of treatment being successful (Barkovich et al., Reference Barkovich, Moore, Grant and Jones2007). These limitations have led to interest in developing alternative imaging methods, which can probe biological properties of tissue to give improved information on tumour type and prognosis. One method of particular interest is magnetic resonance spectroscopy, which provides information on the biochemical composition of selected volumes of tissue (Mukherji, Reference Mukherji1998). There are many publications presenting non-invasive diagnostic classifiers based on MRS for adults with brain tumours (Preul et al. Reference Preul, Caramanos, Collins, Villemure, Leblanc, Oliver, Pokrupa and Arnold1996; Tate et al., Reference Tate, Majós, Moreno, Howe, Griffiths and Arús2003), as well as MRS's ability to predict and monitor response to treatment (Preul et al., Reference Preul, Caramanos, Leblanc, Villemure and Arnold1998; Nelson et al., Reference Nelson, Vigneron and Dillon1999).
Childhood tumours are generally of different types to adult tumours, a fact which dictates that specific diagnostic classifiers need to be built in the context of the disease in children. Even for tumours which occur in all age groups, such as high grade gliomas, there is emerging evidence that such tumours are considered to be different in children compared to adults (Barnett, Reference Barnett2007). Therefore, data from tumour cases in adults should be used with extreme caution when developing diagnostic classifiers for childhood tumours. Furthermore, though brain tumours are less common in children than in adults, there exist a large number of different types of childhood brain tumours (Pizzo & Poplack, Reference Pizzo and Poplack2002). The latter presents one of the considerable challenges for obtaining sufficient cases to build robust classifiers for the disease in children.
The low incidence of each tumour type is compounded by an increasing subcategorization of tumours (Louis et al., Reference Louis, Ohgaki, Wiestler, Cavenee, Burger, Jouvet, Scheithauer and Kleihues2007), which is likely to be escalated with the discovery of new tumour biomarkers. There is also increasing evidence that tumours of the same type have a different biological profile when they arise in different locations of the brain (Pomeroy et al., Reference Pomeroy, Tomayo, Gaasenbeek, Sturla, Angelo, McLaughlin, Kim, Goumnervoa, Black, Lau, Allen, Zagzag, Olsson, Curran, Wetmore, Biegel, Poggio, Muckherjee, Rifkin, Califano, Stolovitzky, Louis, Mesirov, Lander and Golub2000; Taylor et al., Reference Taylor, Poppleton, Fuller, Su, Liu, Jensen, Magdaleno, Dalton, Calabrese and Board2005; Sharma et al., Reference Sharma, Mansur, Reifenburger, Perry, Leonard, Aldape, Albin, Emnett, Loeser, Watson, Nagarajan and Gutmann2007). Given the large number of potential diagnostic categories, the development of classifiers capable of giving a specific diagnosis from the complete list of tumour types, based purely on MRS, is unlikely to be possible in the near future.
Although such development and implementation of classifiers to distinguish between all childhood brain tumours is currently intractable, MRS classifiers can still play an important role in clinical decision-making. The decision-making process undertaken by the clinicians involves the increasing refinement of a list of potential diagnoses by the use of clinical information, tumour markers in the blood and appearances on conventional Computerised Tomography and MRI. Combining the multitude of information mentioned above, commonly allows the clinician to narrow the diagnosis to a short list of two or three candidate diagnoses. Indeed, in a significant proportion of cases there is deemed sufficient information to make a non-invasive diagnosis without the need for new methods such as MRS (Pizzo & Poplack, Reference Pizzo and Poplack2002). An attractive approach for clinicians, in support of the decision process, would be to select or build a classifier, which specifically answers the particular clinical question posed.
Although narrowing the list of potential diagnoses can help the classification process by reducing the number of diagnostic classes, the problem of small numbers in each class remains. One approach to resolve this problem is to combine tumours, with related diagnoses, into the same class. Brain tumours are currently diagnosed using the WHO 2007 classification (Louis et al., Reference Louis, Ohgaki, Wiestler, Cavenee, Burger, Jouvet, Scheithauer and Kleihues2007) and this is structured into specific diagnostic categories and subcategories. This allows tumour groups to be combined together in a well-accepted manner, based on the similarity of their histopathology. In addition to these groupings, clinicians often combine some of these groups together for treatment purposes, since some tumours are known to behave similarly. A particular example where tumours may be grouped to form a large class, is their grouping by tumour grade, a marker of their aggressiveness. One can differentiate between low and high grade classes, since most tumours in the WHO 2007 classification are typified by grade.
Building a DSS, which is sufficiently flexible to allow clinicians to select appropriate classifiers or build a classifier from appropriate cases, has the potential to promote the clinical use of MRS in paediatrics. Since diagnostic classification schemes are regularly updated by groups like WHO, this information needs to be incorporated in a manner which allows straightforward revision and maintenance. In addition, the system needs to have access to extensive MRS data sets and associated, precise, tumour diagnoses in order to design, implement and train classifiers. Therefore, high levels of adaptability and usability are also required for the DSS to be used efficiently in a clinical environment.
3 A participatory design approach in the development of a decision support system for childhood brain tumours
There certainly exist many technical challenges in the development of clinical DSSs, mainly relating to the complexity of the clinical decision process in the context of answering specific clinical questions. However, over the past 20 years, we have experienced an increased and continuous proliferation of new examples of clinical DSSs for a large number of sub-specialities of clinical medicine. Although existing clinical DSSs have proven their accuracy and reliability in several cases (Shortliffe, Reference Shortliffe1987), many of such systems have failed to be adopted in routine clinical use due to a variety of complex socio-technical issues (Barahona & Christensen, Reference Barahona and Christensen1994; Hutton & Prys-Roberts, Reference Hutton and Prys-Roberts1994). In particular, the limited availability of appropriate data, the poor interface design and the lack of appropriate considerations in the natural integration of such systems in the context of clinical practice, are the major issues that have impeded the routine clinical use of DSSs. In particular, clinicians are reluctant to adopt such systems, if their use does not fit naturally into the process of healthcare provision (Barahona & Christensen, Reference Barahona and Christensen1994).
To address the aforementioned socio-technical issues in the development of clinical DSSs and accommodate the specific needs of the clinical decision support process and challenges that exist in the development of optimal diagnostic classifiers for childhood tumours, based on MRS and other associated clinical evidence, we have followed a participatory design approach in the development of the paediatric-related decision support interactive functional elements for the HealthAgents DSS.
The concept of participatory design has enabled us to involve clinicians as potential users of the HealthAgents DSS to direct the processes of requirements analysis, design and interface implementation, while it has facilitated the initial clinical validation of the system within the context of specific clinical questions. We conducted three requirements analysis workshops and 10 design sessions, each of 1 hour, where clinical users provided input in designing the interface elements corresponding to the paediatric childhood tumour decision support components of the HealthAgents DSS. The identification of functional and non-functional requirements for these elements, together with the design of workflow of the clinical decision process in the context of diagnosis of childhood brain tumours from MRS and other associated clinical data, has originated directly from a group of 30 clinicians. This group consisted of clinical paediatric oncologists and neuroradiologists, all members of the functional imaging group of the UK's Children's Cancer and Leukaemia Group (CCLG). This group has an inclusive membership of both clinicians and scientists, with an overall aim to encourage the development of new technologies and techniques to support the care of children with cancer (CCLG, 2009).
We have chosen the participatory approach to encourage the clinical users’ ownership of the developed elements of the HealthAgents DSS, rather than a simply user-driven methodology. The later identifies the users as simply being the passive sources of initial system requirements identification (Shneiderman, Reference Shneiderman2003), an approach that usually results in alienating the potential users of the resulting system and as a consequence impedes the process of broader acceptance and adoption of the system in the future (Magnusson et al., Reference Magnusson, Matthing and Kristensson2003, Sharp et al., Reference Sharp, Rogers and Preece2007).
The web-based interface screens presented below represent the childhood tumour diagnosis part of the interactive functional elements of the overall HealthAgents DSS, as derived through a participatory design approach. The clinical user is able to initially upload the index case for which he wishes to perform a diagnostic classification, by displaying MRS spectra in conjunction with associated supporting elements of spectroscopy and imaging—position of voxel for single-voxel MRS, magnetic resonance (MR) images associated with the case, etc.—under the ‘index case’ interface element of the system. Associated patient and clinical information for the index case, in terms of gender, age of patient and anatomical location of the malignancy, are recorded under the ‘clinical information’ interface element of the system.
The clinician can then construct a set of diagnostic groups pertinent to the index case through an interface screen labelled ‘differential diagnosis’ (Figure 1). Each diagnostic group is constructed by selecting one or more entries from the 2007 WHO classification list of tumours of the central nervous system (brain and spine; Louis et al., Reference Louis, Ohgaki, Wiestler, Cavenee, Burger, Jouvet, Scheithauer and Kleihues2007; Figure 1). The choices from the WHO 2007 list constitute the query which will identify the existence of appropriate classifiers, in the HealthAgents network of distributed nodes, for the clinician to use and run on their index case. The ability to allow more than one WHO 2007 diagnosis to be combined in the same diagnostic group addresses one of the important challenges that the CCLG group has identified: the low incidence of each tumour type in children that is compounded by an increasing subcategorization of tumours. Combining related diagnoses into the same group allows the identification of classifiers that overcome the issue of low incidence of cases in each tumour type. In the following section, we demonstrate this flexibility by considering the process of combining two specific tumour types, Pilocytic Astrocytomas and Ependymomas into a single diagnostic group and comparing this against another tumour type, Medulloblastomas, as a two-class classifier.
Interface screen element for identifying potential differential diagnoses for the index case for which classifiers are required
The design of the interface screens for both the ‘clinical information’ and ‘differential diagnosis’ interface elements of the system, has adopted the approach of dynamic tree-based presentation of concepts that are encoded in the HaDOM (the HealthAgents Domain Ontology) which is discussed in detail by another paper within this special issue (Hu et al., in press). This ontology-based approach has allowed the presentation of hierarchical trees of concepts that are presented as list of choices in the interface, an interaction style that is particularly favoured by clinicians in the way they prefer to identify information in computer-based information and DSSs (Boyes et al., Reference Boyes, Eberholst, Farliec, Sørensend and Lynge2007). The ontology-based approach is particularly useful in this setting since diagnoses, which are closely related, are located close to each other simplifying their co-selection.
The default query selects cases from all ages and tumour locations and both genders. However, the system can refine this query for appropriate identification of classifiers by allowing the clinician to re-determine information relating to age, gender and tumour location. To achieve this, a user (clinician) can select specific age ranges, include one or both classes of gender and specify all or specific locations for the malignancy, in an expandable/collapsible tree menu structure, made available in the ‘refined differential diagnosis’ screen element. This interaction style assists the clinician in refining his query and therefore identifying more appropriate candidate classifiers. This refinement in the query for classifiers (Figure 2) addresses important challenges in the diagnosis of childhood tumours, as identified in current evidence in the literature and by the experience provided by the participating clinicians. For example, based on the input provided from various clinicians in the participatory group, even for tumours which occur in all age groups, such tumours may be considered to be different in children compared to adults, and therefore the refinement of age ranges helps to identify the appropriate classifiers. This design decision provides an approach which avoids using classifiers developed for adult cases being used inappropriately to classify an index case of a childhood brain tumour. Equally, as elicited in the requirements capture sessions, the choice of more specific anatomical locations can accommodate the identification of specific classifiers that might exist in the HealthAgents DSS, which will differentiate some histopathological types of tumour that have different biological profiles due to their growth in specific anatomical locations in the brain.
Gender, age and location refinement of classification query
Figure 3 shows the final part of the workflow, where the system will display any available classifiers derived for the classification of the index case and allow the clinical user to run the classifiers in order to support their clinical decision support process in establishing a diagnosis for the index case. In this example, we display an instance, where two specific classifiers are provided for discriminating between three different childhood tumour types, namely PNET (primitive neuroectodermal tumour) a category which includes medulloblastoma, ependymoma and grade 1 astrocytoma, which is equivalent to pilocytic astrocytoma. This example will be further discussed in the following section.
Classifier identification and classification performance screen – an instance for differentiating three different childhood tumour types, namely primitive neuroectodermal tumour (PNET), ependymoma and grade 1 astrocytoma
4 The healthagents childhood tumour clinical decision support interface: an illustration of diagnosis for cerebellar tumours
In order to illustrate the functionality and flexibility of the childhood tumour DSS, we consider the diagnosis of cerebellar tumours, a common clinical scenario in paediatric oncology. The differential diagnosis is usually pilocytic astrocytoma, medulloblastoma and ependymoma. Ependymomas are of low incidence, making it difficult to attain sufficient numbers of this tumour type to build a robust classifier with this as a single class. The problems are compounded by evidence that both ependymomas and pilocytic astrocytomas in the cerebellum differ from their counterparts in different locations of the brain and the lack of these tumours in adults. This section investigates the discrimination of the three tumour types medulloblastoma, ependymoma and pilocytic astrocytoma using both a three-class classifier and a two-class classifier where the pilocytic astrocytomas and ependymomas are combined into a single class of glial tumours. All cases are taken from children with tumours in the cerebellum.
4.1 Training the classifiers
Data for training the classifiers was obtained from 33 children each with a suspected cerebellar tumour. As each patient has a conventional MR image, this was used to delineate the margins of the primary tumour and enabled the placement of the MRS voxel within the solid-appearing component of the tumour. A single-voxel MRS (1.5 Tesla, PRESS, TE 30 ms, TR 1500 ms) was performed. The diagnosis was obtained by surgical resection and histopathology yielding 11 pilocytic astrocytomas, four ependymomas and 18 medulloblastomas (the acquisition bandwidth per point was 1.024 Hz, cubic voxels with 2 cm or 1.5 cm length were used and 128 or 256 repetitions were acquired, respectively).
The MRS data were phased and 496 data points extracted using equal ppm spacings between 0.2 and 4.0 ppm. Each spectrum was then normalized by dividing each point by the sum of the intensity values. A principal component analysis was performed for dimension reduction and feature extraction. The principal components (PCs) were ordered according to the amount of variance which they explained. The number of PCs used in the subsequent classification was then chosen to be that which gave the optimal classification with an upper limit being the number which explained 95% of the cumulative variance.
Each MRS was classified using LDA (linear discriminant analysis) of the PC scores. Optimized feature selection was performed by minimizing the classification error in a leave-one-out (LOO) analysis cross-validation loop, while increasing the number of PCs used.
4.2 Results of the MRS classification
The results of a three-class classifier are shown in Figure 4. A plot of the discriminant function scores for each case based on eight PCs is shown in Figure 4(a). Discriminant function 1 separates the pilocytic astrocytomas and the ependymomas from the medulloblastomas. Discriminant function 2 separates the ependyomomas from the pilocytic astrocytomas with the exception of one ependymoma case that clusters more closely with the pilocytic astrocytomas. In Figure 4(b), it is identified that the first 11 PCs of the spectra account for approximately 95% of the variance, with almost 50% being accounted for by the first PC (Figure 4(b)). Optimum performance of the classifier was reached using the first eight PCs. This resulted in a 91% (31/34) classification success rate in a LOO analysis (with one case in each class being incorrectly classified). Figure 4(c) and (d) shows plots of the corresponding spectral coefficients for the two discriminant functions. The LOO external cross validation error was 21.2%.
Classification of cerebellar tumour type using pattern recognition of the normalised spectra (method 1) showing (a) scatter plot of LDA of the first eight principal components (pilocytic astrocytoma class represented by squares, ependymoma class represented by diamonds and medulloblastoma class represented by circles), (b) distribution of variance being accounted for by the principle components, (c) and (d) plot of the mean spectral coefficients for the discriminant functions 1 and 2 determined by the LOO cross-validation procedure with 95% confidence intervals indicated by shaded area
A two-class classifier, in which the pilocytic astrocytomas and ependymomas were combined to create a class of glial tumours, gave a classification success rate of 94% (32/34) using a LOO analysis. One pilocytic astrocytoma and one medulloblastoma were incorrectly classified, all ependymomas were classified correctly. The LOO external cross validation error was 14.7%.
4.3 Discussion
Although there are only a small number of cases in this study, the improved classification success rate of the two-class classifier when compared with the three-class classifier supports the strategy of combining diagnoses to create larger classes. The lower external cross validation error for the two-class classifier further supports this approach. The finding that none of the incorrectly classified cases, in the two-class classifier, were ependymomas demonstrates that this tumour group can be successfully combined with pilocytic astrocytomas to form a larger class of glial tumours. Where clinical and imaging findings restrict the likely diagnoses for the index case to either a medulloblastoma or an ependymoma, a two-class classifier, such as that constructed in this study, may be the most appropriate to use clinically.
There are over 200 childhood cases with MRS in the HealthAgents data warehouse at present with approximately equal numbers of brain tumours and other cases. This has allowed the construction of a number of childhood-specific classifiers of clinical importance, including tumour cases versus non-tumour, aggressive versus non-aggressive tumours and glial tumours versus primitive neuroectodermal tumours. The classes in the latter two classifiers contain cases with a variety of diagnoses demonstrating the approach advocated in this paper. There are some cases where these classifiers will not answer the clinical question posed. For these scenarios, the clinician can interrogate the database to find how many cases are available in the classes they wish to define. Mean spectra or specific cases can then be used to aid diagnosis, where the number of available cases is too few for a reliable classifier to be made available. As more cases are added to the data warehouse, more specific classifiers will be developed allowing the clinician to select these when appropriate.
5 Conclusion
We have presented a user participatory approach to the design, implementation and initial clinical validation of a set of specialized interface elements to the HealthAgents DSS. The purpose of such interactive functional elements is to support the clinical process of diagnosing childhood brain tumours using MRS, by giving the user increased control over the selection of appropriate pattern classification techniques. The participatory design approach has provided a modular and flexible web-based interface that addresses some of the challenges of childhood tumour diagnostic process. The design of interface screens has been supported by the systems’ domain ontology, allowing clinicians to input clinical and differential diagnosis information in order to classify an index case to specific diagnostic class. The participatory design approach has involved clinical users in directing the design and implementation of the functional elements of the system. In addition, a initial clinical validation of the usefulness of the current design was demonstrated by addressing two important challenges in childhood tumour classification; the low incidence of certain tumour types and appropriate restriction of cases by patient age and tumour location. To further benefit from the participatory design approach, the interface and interactive functional elements could be further evaluated by our clinical group from a viewpoint of usability and more additional user acceptance studies in the clinical environment, to determine future deployment options of such a DSS in clinical practice.
Acknowledgements
The authors thank all of the clinicians who took part in the participatory design approach and in particular the Functional Imaging Group of the UK's Children's Cancer and Leukaemia Group. They also thank members of the Radiology Department at Birmingham Children's Hospital, in particular Shaheen Lateef and Rachel Grazier, for their help in collecting MRS data. The work was funded as part of the HealthAgents project by the European Union IST-2004-27214. AP was part funded by a Department of Health Clinician Scientist Award.
