Dr Killaspy and colleagues make some important points about the Mental Health Recovery Star, but they adopt a surprisingly dismissive tone about this innovative user-led tool and about our study. With careful caveats, we have argued from naturalistic data that the tool is measuring an underlying construct, and that it has the potential to record reported change. Killaspy et al criticise claims that we simply have not made. Our analysis was not intended to put the psychometric properties of the Mental Health Recovery Star beyond doubt. The development of the tool has employed a user-based approach and, as such, has lacked some of the formal and restrictive academic rigour associated with traditional psychometric testing. We would welcome further research and development to address this.

It is our understanding that the interrater reliability testing cited by the authors is largely based on staff-only ratings of service users’ recovery journey. This is not how the tool is intended to be used. It is surprising that Dr Killaspy and colleagues would choose to evaluate a tool in a way which goes against the directions for its use. That intraclass correlation coefficient results fall short of the required 0.7 could reflect the inherent inaccuracy and instability of having sensitive personal recovery dimensions estimated by professionals without discussion with the service user. It is unclear how this fits with recovery as a construct built on individual service users’ own priorities. Surely user involvement in the measurement of recovery should be central to the definition of their outcomes.

In relation to sensitivity, it is true that there is a lack so far of proven external validity for the Recovery Star. Again, our paper makes no claims about external validity but merely comments on the fact of change between readings and the promise that this holds. We agree that reported changes are small and that the underlying ‘ladder of change’ model remains untested. However, it is useful to provide a clear and accessible model of change, which is supported by training and the Recovery Star Organisational Guide. Importantly, this instructs that second readings are taken without reference to the first.

We would like to see future versions of the Recovery Star and other recovery tools that are both psychometrically robust and, crucially, of practical use and relevance to mental health service users and their carers. There is little point in adopting a scientific gold standard for tracking recovery outcomes if it eschews the involvement of people in appraising their own recovery.