We are delighted that these responses to our editorial expand on issues that we could not explore more fully. Tulloch gives a cogent account of the typical methodology – not now being followed in England – for deriving case-mix groupings and finds the present plan wanting. He suggests a slow, careful change to commissioning based on activity and case-mix. Kingdon et al make the case for a system in which both diagnosis and care pathways are central in costing and thus purchasing, only en passant asking the crucial question of how (not whether) cost can be firmly linked to the quality of services delivered.

Both letters focus on which type of data should be chosen. The intended benefit of case-mix systems is to improve the direction of resources towards the greatest local need. If that were the only eventual use of the data items under discussion then mental health units should collect whichever (activity counts, clusters, diagnoses, pathways, etc.) best satisfy criteria such as Fetter's, as Tulloch implies. But data, once collected, have many other uses and misuses.

Kingdon et al argue on theoretical grounds that diagnostic categories should be better indicators than clusters of the type and quantity of care that is required by patients. Yet as Tulloch points out, findings from international analysis of variance studies of actual resource consumption within diagnostic groupings have tended to lead to their abandonment. We can add that similar methodology was used in mental health services in England from the early 1990s by the National Health Service Information Authority, testing both diagnostic and multidomain descriptors of patients’ problems, in national and multi-site trial data-sets. ^{Reference Carthew, Elphick and Page1} Diagnostically defined healthcare resource groups were abandoned by the Department of Health, not only because of the modest reduction in variance achieved, but also because of resistance by non-psychiatrists to the collection of diagnostic data. There was also resistance to informatics in general by a substantial proportion of clinicians, including senior Royal College of Psychiatrists’ leaders at the time, although that is no longer the case. ² Clusters were seen by policy makers as more likely to be acceptable. The fact that they become mandatory on 31 December 2011 with only this discussion in the Journal suggests that this approach is working.

As Kingdon et al point out, diagnostic categories enable us to use therapeutic research findings to decide which type of drug or psychosocial approach is chosen, but that does not much affect overall costs, and people often retain the same diagnosis throughout many life changes. By contrast, multidomain scores include more factors that indicate whether someone currently needs admission or frequent contact with paid professionals, which are the main financial determinants. And since the mental health clustering tool (MHCT) includes symptomatology ratings, and separates clusters into broad diagnostic groups anyway, the statistical benefits of diagnosis have not been entirely lost. Until there is more empirical evidence from costing studies, the relative merits of diagnostic versus multidomain data will remain debatable. Of course their value in outcomes and other quality monitoring, and predicting prognosis, must also be considered in developing mental health informatics generally. Prognosis is important because there is more 'value’ in resolving a situation that would otherwise become chronic.

We do not support the automatic assignment of patients to any form of treatment, pathway or package of care on the basis of MHCT scores alone. The data may raise retrospective questions about clinical judgements, but should not replace them.

So what should we be doing about commissioning? Tulloch suggests in effect returning to the 1993 position and starting again. We do not think this is possible; while we looked away, boats were burnt. Kingdon et al propose the combination of diagnoses with pathway data for costing purposes, but do not say quite how. The strong argument against using intervention counts, pathway data or other activity measures on their own for remuneration is that there is no safeguard against unnecessary, ineffective or inefficient interventions or pathways. Tariff ‘matrices’ in which prices are applied to cells containing both broad diagnoses and clinical management data have been proposed in the past, ^{Reference Carthew, Elphick and Page1} but as we said above, they were abandoned. The large number of resulting categories should theoretically reduce costing variance, but it may be that commissioners would not in practice be able to use them effectively.

Yeomans concentrates on Routine Clinical Outcomes Measurement (RCOM), arguing strongly for its development and enhancement, while wisely refraining from almost suggesting ‘Payment by Outcomes’, which would violate Goodhart's law, succinctly put by Strathern: ‘When a measure becomes a target, it ceases to be a good measure’. ^{Reference Strathern3} We agree with nearly all his points, especially on the importance of feedback, which are, notwithstanding the dated survey he quotes, already coming to pass in some parts of England, as are developments in Patient-Reported Outcomes Measures (PROMs). Efforts to usefully involve HoNOS in clinical work itself are being reported, although from the other end of the earth. ^{Reference Stewart4} As he says, HoNOS are a start but not the last word in outcomes measures, and we would caution against using them for thresholds for referral or discharge. Validity in groups is no guarantee of validity in individual cases.

With exceptions, we have been slow to grasp the twin nettles of outcomes and costing of services, and if we are to regain the initiative, we have to think widely and deeply about what systems we think will work best for service users, even while change in these very systems is accelerating. A start would be made when trusts have clinical, outcomes, intervention, costing, human resource and finance data on the same spreadsheets for themselves.