Introduction
Globally, public mental health systems struggle with scarce resources while the demand for services is increasing (1). Moreover, there are significant disparities in access to care (Reference Cummings, Allen, Clennon, Ji and Druss2–Reference Lowther-Payne, Ushakova and Beckwith4) and widespread challenges with recruitment and retainment of clinicians (Reference Alkan, Cushen-Brewster and Anyanwu5–Reference Owolabi, Olatoye, Elufioye and Okunade7). There is a growing need for more sustainable models of organizing and delivering services (Reference Berry8).
Digitalization is a strategy to reshape care models within current resource constraints (9–Reference Schmidt13). Existing literature indicates great promise for the integration of accessible digital tools in mental health, such as video consultations to deliver therapy (Reference Hagi, Kurokawa and Takamiya14–Reference Giovanetti, Punt, E-L and Ilardi16) and mobile apps to support therapeutic interventions or monitor clinically relevant variables (Reference Currey and Torous17–Reference Romm, Skoge and Barrett20).
The potential of digitalization extends to services targeting severe mental disorders, such as psychotic and bipolar disorders, but there is still a lack of knowledge concerning these patient populations (Reference Ehrt-Schäfer, Rusmir and Vetter21–Reference Bell and Alvarez-Jimenez24). Findings from previous work suggest that no service users should be excluded from the target groups of digitalized care based on diagnosis alone (Reference Skoge, Aminoff and Barrett25;Reference Skoge, Aminoff and Ihler26). Insights gained from studying this population are often applicable to individuals with less severe conditions, making this group a valuable reference point for advancing digitalized mental health care overall.
However, public health care is characterized by slow implementation processes, (Reference Nembhard, Alexander, Hoff and Ramanujam27) and adoption of digitalized tools in routine care remains low (Reference Lattie, Stiles-Shields and Graham28–Reference Mohr, Weingardt, Reddy and Schueller31). Many health care innovation projects are costly, and there is a high rate of failure (Reference Stanimirovic32–Reference Kaló, Imre, Tatár and Nagy34). This indicates great losses related to the waste of resources spent on research, development, and implementation efforts (Reference Berry8;Reference Chalmers, Bracken and Djulbegovic35). Moreover, digitalized care is commonly perceived as a deviation from in-person treatment and triggers resistance toward adoption, especially among clinicians, who commonly hold more critical views than service-users (Reference Drissi, Ouhbi, Serhani, Marques and de la Torre Díez36;Reference Cowan, McKean, Gentry and Hilty37).
Hybrid care models combine digital and traditional formats. Since they offer benefits from both types of care delivery, digital elements may be perceived as more feasible and acceptable (Reference Drissi, Ouhbi, Serhani, Marques and de la Torre Díez36). Hybrid care models facilitate more flexible and need-based treatment and new ways of allocating resources and structuring workflows (Reference Steidtmann, McBride, Mishkind and Shore38–Reference Macrynikola, Nguyen, Lane, Yen and Torous41). Nevertheless, they are uncommon in Norwegian routine mental health care and remain at a conceptual stage of service innovation. Services must make informed decisions about further research and implementation to reduce the risk of failed investments, but conclusive evidence regarding the value of different hybrid care models and how to successfully implement them is currently lacking (Reference Gujral, Illarmo, Jacobs and Wagner42).
Adaptations of traditional research approaches are needed to efficiently guide the next steps for such service innovations early on. There is a rapidly growing academic interest in early health technology assessment (eHTA) (Reference Støme43;Reference Grutters, Bouttell and Abrishami44), a forward-looking methodology that enables analyses while existing evidence is scarce. eHTA allows flexible compositions of various methods and generates formative decision support, which can help advance the further innovation processes of new concepts, such as hybrid care models.
The objective of this study is to explore unmet needs of service users, clinicians, and other actors affected by the way health care services are organized, estimate the potential value of hybrid care models in treatment of severe mental disorders through eHTA methodology, and guide further research and inform practical implementation strategies.
Methods
The structure of this manuscript adheres to the main elements of the SQUIRE guidelines (Reference Chalmers, Bracken and Djulbegovic35), a framework adapted to reporting research on systemic health care improvements.
Context
The general context of the study is the Norwegian public specialist health care for severe mental disorders. Norway is a high-income country with a comprehensive welfare system and low private health expenditure (Reference Saunes, Durvy and Litvinova45). The specialist psychiatric services have a mandate to serve the population equally across social strata and geography. To access specialist health care, service users must experience moderate to severe symptoms and receive a referral from their general practitioner or another primary health actor (46). The sector struggles with limited availability of clinicians, and health authorities call for new sustainable strategies of organizing and delivering care (Reference Sør-Øst47;48).
The background of this assessment is a series of three pilot studies conducted by the research group to investigate the integration of a mobile app into treatment for severe mental disorders (Reference Romm, Skoge and Barrett20;Reference Skoge, Aminoff and Barrett25). The pilots revealed a sharp contrast between the readiness expressed at research sites prior to the research project and the low levels of activity during the study period. This was also observed at the third site, which was characterized by beneficial practical circumstances and a professional environment explicitly supportive of digitalization. There were reports of concrete problems that partly explained challenges with coherent participation, such as technical bugs and unforeseen changes in the health condition of patients. More importantly, however, there seemed to be fundamental systemic barriers to integrating technology into routine services. Although these observations were made during pilots, representing much earlier phases than implementation, they were interpreted as symptoms of the recurrent implementation problems in the digital mental health field and termed an implementation paradox. See Supplementary Material 1 for a description of this phenomenon and details about the pilot study procedures.
The research group’s paradoxical experiences with the unsuccessful integration of technology highlighted the need to revisit the innovation process. Consequently, we chose to proceed with eHTA to gain a better understanding of how to address these challenges and effectively implement hybrid care models.
Hybrid care models
We define hybrid care models as routine services that integrate digital tools in their delivery of care based on clinician evaluation and current needs of the service user. The concept focuses on creating a new digitalized service that enables more flexible communication and treatment through the purposeful use of accessible technologies. We selected the integration of video consultations and mobile apps in treatment for severe mental disorders as a case for the evaluation to explicitly demonstrate and discuss potential value and implementation challenges. Although this case includes two digital tools, this assessment explores hybrid care models in general, rather than evaluating video consultations and mobile apps specifically.
Methodological framework
eHTA is a methodology used to evaluate the potential value of health technologies from early phases of development, while scientific evidence is scarce. eHTAs also support simultaneous iterative testing with a heavy emphasis on aligning the innovation with user needs. The output of eHTAs is formative decision support, guiding the next steps in the development, research, implementation, or discontinuation of health technology based on validated estimates. This process reduces risk related to research waste and implementation failure. eHTAs can combine available data sources and methods that are suitable for specific knowledge gaps (Reference Grutters, Kluytmans, van der Wilt and Tummers49;Reference Rodriguez Llorian, Waliji, Dragojlovic, Michaux, Nagase and Lynd50). Stakeholder involvement is typically a core element and can comprise a range of different data collection activities, such as workshops, surveys, and key informant interviews. In addition, methods such as scenario building and analysis, economic simulations/sensitivity analyses, and usability tests can comprise elements in an eHTA process.
We applied the eHTA framework Step Up (Reference Kværner, Støme, Kværner and Thomas51) in this study. Step Up is an evidence-based method that assesses innovations across four domains: user, clinical, organizational, and economic. The framework includes tools designed to gather insights from a diverse range of stakeholders affected by the innovation while also considering potential risks and barriers. Step Up is designed to be applied in a proactive and iterative manner, developing formative knowledge that advances the innovation and clarifies what is needed for successful implementation and realization of intended value. Step Up is flexible and can be used in various processes, ranging from early need-mapping and problem-defining, to informing procurement strategy. The Step Up handbook and material supporting workshop activities are publicly available from the Norwegian Centre for E-health Research’s website (52).
Co-creation is integral in this framework and may be defined as the process in which end-users of a service are included as active partners in defining challenges and designing the service, based on their experience and competences (Reference Stier and Smit53). Co-creation challenges the traditional top-down governance models in health care, including service-users in the design, delivery, and maintenance of health innovations (Reference Voorberg, Bekkers and Tummers54). In this eHTA, we used scenario drafting to construct realistic and tangible examples of hybrid care models prior to the main data collection. Scenarios were co-created with two experts by experience recruited through the Bipolar Association Norway, who applied insights concerning unmet service user needs in current care models. The process resulted in several hypothetical scenarios illustrating enhanced user journeys based on digitalized care models. See Supplementary Material 2 for a description of the scenario drafting process and examples of complete scenarios. The scenarios were validated by a broad range of other stakeholders during the main workshop and applied throughout the assessment as a tool to evaluate hybrid care models as a concept. In addition to contributing to the construction of scenarios, the experts by experience also participated in a panel that provided feedback during planning and writing of the manuscript.
Data collection and analysis
The eHTA is based on two data sources (Figure 1).
Flowchart of data sources. Note: Insights developed from our previous research comprise the context for the stakeholder workshops. Output from the workshops fed variables to the exploratory economic simulation model.
Figure 1. Long description
The flowchart is organized into three vertical columns that feed into a final horizontal box at the bottom.
1. Left Column (Context for the assessment): Titled ‘Pilot series investigating the clinical integration of a mobile app’. It contains three dashed boxes connected by ‘Iteration’ arrows: ‘Proof of concept study’ leads to ‘Feasibility study’, which leads to ‘Pilot study’. This sequence feeds into a final dashed box: ‘Ecologically valid insights into the implementation paradox of digitalized innovations and critical knowledge gaps’.
2. Middle Column (Main data source): Titled ‘Broad stakeholder involvement and co-creation’. An arrow from the left column's pilot study feeds into a dashed box: ‘Scenario drafting workshop’. This leads down to a solid box: ‘Stakeholder workshop’, which then leads to a solid box: ‘Stakeholder-validated unmet needs, hybrid care scenarios, and anticipated value of hybrid care models’.
3. Right Column (Secondary data source): Titled ‘Hybrid care scenarios translated into quantitative value’. An arrow from the middle column's stakeholder workshop feeds into a dashed box: ‘Model construction based on validated scenarios, local registry data, and reports’. This leads down to a solid box: ‘Economic simulations’, which leads to a final solid box: ‘Estimated economic, environmental, and service-related impact of increased use of video consultations’.
4. Bottom Horizontal Box: Arrows from the final boxes of the middle and right columns point to a large grey box at the base titled: ‘Early health technology assessment of hybrid care models for severe mental health disorders’.
Stakeholder involvement
We involved a broad range of stakeholders to map and validate potential areas of application, anticipated effects, and implementation issues related to hybrid care models. Stakeholders included researchers and clinicians with experience from various specialist services for severe mental disorders, a leader of a specialist health service, experts by experience as next-of-kin and through personal experience with severe mental disorders, a specialized nurse with mercantile competencies, a leader of economics in specialist health services, and clinicians working in municipal services. A participant with experience with occupational therapy for severe mental disorders was prevented from attending due to technical issues and provided their viewpoints at a later point in written form. The stakeholder workshop (n = 11) was conducted on 28 February 2025 and led by KJK, LNS, and MS. It was conducted using Microsoft Teams and lasted two and a half hours. The workshop resulted in an audio recording, a text chat log with stakeholder comments, and Step Up worksheets in Microsoft PowerPoint. All participants received a summary of the workshop material for review.
The workshop included two Step Up activities (Reference Kværner, Støme, Kværner and Thomas51), an impact analysis, and a benefit planning. During the impact analysis, participants identified potential beneficial effects and risks of implementing hybrid care models. They assessed the impact of each effect across different domains. They also reported current unmet needs, discussed how hybrid care models could address these needs more efficiently, and identified conditions relevant for implementation. We performed a thematic analysis of these data using King and colleagues’ template analysis techniques (Reference Brooks, McCluskey, Turley and King55), adapted to the Step Up framework. See Supplementary Material 3 for a detailed description. During the benefit planning, stakeholders discussed which potential effects should be prioritized and identified suitable methods for investigating these in real-world settings. These data were later used to construct quantitative scenarios for the exploratory economic simulations.
Economic simulations
We performed economic simulations to quantitatively demonstrate the potential value of implementing video consultations as part of hybrid care models. Importantly, the simulations of this study were performed in an early conceptual phase characterized by a scarce data foundation and a considerable degree of uncertainty. The simulations are formative and exploratory, not a full cost-effectiveness analysis.
We collected and processed data in two phases to build the economic simulation model: First, we utilized the results from the benefit planning, extracting potential short- and longer-term beneficial effects that were identified by stakeholders. The selection of potential effects (i.e., scenarios) to be included in our model was based on a) stakeholder prioritization: perceived relevance and impact, b) quantitative operationalizability, c) accessibility of the necessary data, and d) that scenarios could be used as proxies to study potential impact on different pillars of sustainability (economic, environmental, and service-related, i.e., clinical capacity). We selected the following scenarios: 1) increased outpatient capacity through increasing the number of video consultations, 2) reduction in cancelled consultations, and thus, increased clinical activity levels, 3) increased revenue through enhanced activity, and 4) reduced travel costs for patients and clinicians and environmental impact in terms of reduced CO2 emissions. A prerequisite for the scenarios was that video consultations are used either as a supplement to in-person consultations (as clinically required) or as an integral component of a predefined service-user pathway.
Second, we collected publicly available registry data, internal cost data, and relevant public reports and guidelines, supplying the model with real data from local settings. Assumptions concerning anticipated effects of the integration of video consultations into routine services on increased clinician capacity for hybrid employees, reduced cancellations due to more accessible services for service-users, and following changes in travelling patterns, were based on expert opinion from the local clinical and research environments and stakeholder insights. The local context for the simulations was the Division of Mental Health and Addiction at Oslo University Hospital. Oslo University Hospital is located in the capital and is the largest hospital in the country. The hospital is highly specialized and delivers secondary and tertiary services, serving several districts of Oslo as a local hospital while also having multiregional and nation-wide responsibilities (56). Supplementary Material 4 contains information about other preconditions and assumptions underpinning the scenarios, as well as descriptive sensitivity analyses demonstrating more radical and moderate alternative scenarios.
The economic model can be validated after actual implementation at a later point through a socioeconomic analysis based on a randomized controlled trial or case study data, testing whether the selected assumptions align with empirically observed sustainability outcomes.
Ethical considerations
The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on research involving human subjects and with the Helsinki Declaration of 1975, as revised in 2013. All stakeholder workshop participants signed a written consent form. The procedure was approved by the data protection officer at Oslo University Hospital (24/14691 - 34477779).
Results
Potential effects of implementing hybrid care models
Combined data analysis demonstrated potential effects across the user, clinical, organizational, and economic domains of the Step Up framework (Table 1).
Stakeholders’ anticipated effects of implementing hybrid care models
Table 1. Long description
The table is divided into four primary domains.
1. User domain
* Benefits: Service users and next-of-kin report higher satisfaction, safety, and time savings. It facilitates balancing treatment with everyday life and offers alternatives to traditional settings. Clinicians benefit from flexible working conditions and home-based work.
* Risks: Clinician fatigue from digital communication, loss of the physical therapy room context, and unclear role definitions. Prerequisites like technical skills and language demands are noted as barriers.
2. Clinical domain
* Benefits: Improved prognosis through early intervention and need-based care. Increased engagement and access to ecologically valid clinical parameters. Allows for new therapy delivery methods like short check-ins.
* Risks: Loss of face-to-face information leading to masked symptoms. Service users must manage emotional reactions alone. Potential for increased social withdrawal and unsuitability for certain therapy orientations.
3. Organizational domain
* Benefits: Reduced cancellations and smarter resource allocation. Supports new workflows like patient-led booking and task shifting. Clinical competencies are no longer limited by geography, aiding recruitment.
* Risks: Lack of infrastructure leading to added workload and pressure on clinicians, which may threaten autonomy.
4. Economic domain
* Benefits: Potential for increased hospital income and socioeconomic benefits from higher workforce participation. Reduced sick leave, travel costs, environmental impact, and office space requirements.
* Risks: Significant initial investment required for digital care equipment.
Note: Anticipated beneficial effects (+) and risks (−) of implementation of hybrid care models, extracted from the first and second level thematic levels of the template analysis.
The exploratory economic simulations estimated that increased use of video consultations as part of hybrid care model implementation had a positive impact on sustainability pillars across the four scenarios (Table 2). In a descriptive sensitivity analysis, we explored the effect of moving from a moderate estimate assuming an increase of one outpatient consultation per clinician per week to the increase of two and three consultations per clinician per week. This resulted in a 12, 25, and 37 percent increase in clinical activity, respectively. Regarding capacity loss due to service users not showing up to scheduled sessions, the estimated reduction in cancellations ranged from 3 to 13 percent. Strengthened clinical capacity and resource allocation combined were estimated to result in a 19 percent increase in production, representing 1.9 million euro in hospital income. Finally, the sensitivity analysis showed that the simulated shift towards more digitalized patient pathways and hybrid therapist workflows resulted in annual travel cost savings ranging from 326 000 to 978 000 euro, as well as a reduction of CO2 emissions ranging from 82 to 247 tons. See Supplementary Material 4 for more details.
Results summary from the exploratory economic simulations
Table 2. Long description
The table consists of two columns: Outcome and Description.
* Row 1: Outcome is Increased outpatient capacity by 12 percent. Description states video consultations may save time. One additional consultation per week per therapist yields 22,500 more consultations annually, based on 500 practitioners performing one more consultation per week for 45 weeks.
* Row 2: Outcome is Reduction in cancellations by 7 percent. Description states video consultation may offer a more flexible therapy form. Among 4,000 monthly consultations not carried out in the clinic, a 25 percent reduction in no-shows and cancellations could result in 12,000 additional consultations annually.
* Row 3: Outcome is Increased revenue for the hospital of 19 percent through enhanced capacity. Description states total capacity may increase by 19 percent based on previous scenarios. This results in increased revenue of 1.9 million euro, based on a unit cost of 329 euro with a D R G of 0.157 to 0.185 and 34,500 additional consultations.
* Row 4: Outcome is Annual travel cost savings of 652,000 euro and annual reduction of C O sub 2 emissions of 165 tons. Description states 500 therapists and their patients save trips. This results in 22,500 therapist trips and 112,500 patient trips saved annually. Calculations are based on travel distances of 30 kilometers for therapists and 20 kilometers for patients, with Oslo travel habits of 40 percent car and 60 percent public transport, totaling 651,708 euro in savings and 165 tons of C O sub 2 per kilometer.
Note: Descriptions of the simulated outcomes of four scenarios. The exploratory simulation models are based on local registry data from Oslo University Hospital and data from publicly available reports.
Stakeholder recommendations
Change through incentives
The thematic template’s first integrative theme Change Through Incentives highlighted the necessity to closely adhere to not only needs but also preferences and ambitions of clinicians, from early developmental phases to succeed in the implementation of new care models. Stakeholders suggested that care models should result in improved workflows and working conditions for clinicians. The possibility to work from home, increased everyday autonomy, and greater variation in tasks are examples of stakeholder-validated incentives that can trigger a change in practices and thus facilitate organizational shifts toward hybrid care models. Stakeholders also emphasized the relevance of incentives in creating attractive workplaces to ensure sustainable recruitment and retainment.
Nuancing novel care models
The second integrative theme, Nuancing Novel Care Models, illustrated how a nuanced understanding of digitalized care is needed for users to accurately evaluate the feasibility and value of reshaping existing care models into hybrid ones. Stakeholders expressed that digitalization is commonly connoted with fully digital services and a permanent opt-out from in-person encounters. Such all-or-nothing conceptualizations may result in doubtful attitudes, heightened attention toward value that is lost in digitalized alternatives, and thus, maintain resistance. Conversely, understanding the concept of hybrid care models as a collection of varied digitalized care model designs with flexibility at their core may result in more optimistic attitudes and spark creativity concerning local adaptation and implementation.
Discussion
This eHTA explored the potential value of hybrid care models in health care for severe mental disorders while addressing current unmet user needs and implementation challenges. Estimations based on stakeholder involvement and exploratory economic simulations demonstrated potential beneficial consequences from implementing hybrid care models across all domains included in the assessment, indicating great promise of hybrid care delivery in strengthening the sustainability of mental health care.
Although the optimistic findings of this assessment are mirrored in the literature (Reference Rebello, Marques, Gureje and Pike10–Reference Stanic, Saygin Avsar and Gomes12;Reference Steidtmann, McBride, Mishkind and Shore38), realization of anticipated value depends on successful implementation, which is not a given outcome. The starting point of this assessment, the implementation paradox observed in our previous work, aligns with well-documented implementation gaps in the digital mental health field (Reference Appleton, Barnett and Vera San Juan29;Reference Berardi, Antonini and Jordan30) and in the public health sector more generally (Reference Nembhard, Alexander, Hoff and Ramanujam27). Digitalized innovations repeatedly gain scientific support from technical trials, feasibility studies, and controlled randomized trials; however, their part in reshaping care models remains modest. Research that contributes to bridging the persistent gap between the theoretical value of digitalization and its realization by producing actionable and contextualized insights is increasingly requested (Reference Kaló, Imre, Tatár and Nagy34;Reference Chalmers, Bracken and Djulbegovic35;Reference Turner, Goulding, Denis, McDonald, Fulop, Raine, Fitzpatrick, Barratt, Bevan, Black and Boaden57). eHTA provides the opportunity to identify future potential benefits early in the implementation process, allowing for stakeholder-validated adjustments to be made along the way to enhance the chances of successful implementation. Moreover, the methodology allows a broad scope, studying services, rather than technology in isolation, and including the overarching systems that the service innovation is part of. To our knowledge, this study is the first eHTA that assesses a mental health service and the first study that uses eHTA methodology to address digital mental health implementation. The demonstration of eHTA application in this study may comprise an academic contribution, as it can inspire the methodological thinking in future work across both fields of research.
The assessment highlights the clinicians as a user category to consider from conceptual to implementation-related stages of innovative care models. The patient perspective has traditionally been prioritized in the design of care models. However, service providers comprise a key user category and often play the role of the gatekeeper in the implementation of digitalized service innovations (Reference Cowan, McKean, Gentry and Hilty37). The service design approach is an integral part of the Step Up framework, which incorporates service users, next-of-kin, and clinicians within the user domain (Reference Stickdorn and Schneider58). Accordingly, the stakeholders of this assessment indicated that desired changes in the practices of clinicians toward delivering digitalized care should be linked with clinician incentives. In the pilot studies conducted prior to this assessment, which were characterized by implementation failure, the clinicians did not experience any direct incentives related to the digital innovation. Instead, across the three sites, integrating the app seemed to represent an additional task, and few clinicians gained much experience with its beneficial clinical effects. In the stakeholder-validated clinician scenario, however, incentives were a core prerequisite for the new, enhanced workflow. The strategy of boosting practice change through incentives is supported by existing literature, which underlines that the effects of incentives vary between contexts and must be adapted according to local needs, preferences, and culture (Reference Gneezy, Meier and Rey-Biel59–Reference Parkinson, Meacock and Sutton61). Moreover, research supports investments in clinician incentives to enhance the attractiveness of mental health services as workplaces and improve recruitment and retention rates (Reference Alkan, Cushen-Brewster and Anyanwu5;Reference Barriball, Bremner and Buchan6;Reference Adams, Ryan and Wood62).
The assessment also underlines the relevance of subjective perceptions of digitalized care models in willingness to change practices and contribute to implementation. Developing a nuanced and pragmatic view on digitalization has been suggested as a precondition for implementation in the literature (Reference Drissi, Ouhbi, Serhani, Marques and de la Torre Díez36;Reference Cowan, McKean, Gentry and Hilty37). Importantly, more positive attitudes are associated with direct experience with a digitalized method of care delivery (Reference Connolly, Hogan, Shimada and Miller63;Reference Kaphzan, Sarfati Noiman and Negev64). However, there is a tendency to prefer the current situation rather than engage in change, that is the status quo bias (Reference Kim and Kankanhalli65). This represents a threshold for gaining initial experiences and maintains all-or-nothing thinking for both clinicians and decision-makers and may be an important target in the planning of future implementation processes. There is a need to provide inspiration, augment credibility, and fuel willingness to gain initial experiences as hybrid service providers. An enhanced focus on how even low-threshold changes can lead to substantial systemic and quantitative changes may represent a core message in this communication. Demonstrating different types of evidence for this notion may be effective for this purpose. Evidence can comprise material such as the early evidence produced in this study. Moreover, it should include direct accounts from clinicians with hybrid care experience from real clinical settings, such as those identified in our previous work and in other existing literature (Reference Skoge, Aminoff and Ihler26;Reference Shalev, Khazen, Rose, Lubin and Eitan66).
eHTA methodology is also concerned with unintended effects (Reference Grutters, Bouttell and Abrishami44). Through the stakeholder analysis, we identified risks related to all user categories, including technical barriers and loss of therapeutic context, clinicians’ digital fatigue, service users’ maintenance of avoidance strategies, as well as unclear roles for next-of-kin. Some of these risks are well documented in existing literature (Reference Natsukari and Ozaki67–Reference Whitehead, Alrashdi, Bin and Drini69), underlining their relevance. Risks experienced from the next-of-kin perspective, however, remain a less explored research area. In accordance with service design principles and the Step Up method (Reference Kværner, Støme, Kværner and Thomas51;Reference Rösch, Tiberius and Kraus70), we argue that searching for hidden risks, developing a comprehensive understanding, and then addressing them, is at least equally important as direct efforts to realize potential benefits.
We encourage applying the empirical and methodological insights developed through this study in multiple small-scale case studies across clinical contexts, testing various hybrid care models and implementation strategies. The clinician user perspective must be emphasized in each distinct context. We also encourage research on direct impact and ripple effects of the potential changes in user experiences, organizational setup, and clinical practice related to hybrid care implementation, both negative and beneficial ones. This includes consequences of establishing clinician incentives, e.g., working from home setups; expansions of current treatment alternatives, e.g., service-user-led booking of consultations; and shifts in relational dynamics for different user categories. As hybrid care models remain in a conceptual phase, eHTA methodology will continue to play a useful role in advancing the innovation process. Moreover, eHTA methodology is still evolving, and there is a need to develop and test additional methodological features, such as including sustainability parameters (Reference Holtorf, Pegg, Mueller and McMeekin71) or adapting frameworks to the specific stage of the innovation subject to assessment (Reference Støme, Moger, Kidholm and Kværner72).
Limitations
This eHTA is a formative evaluation and represents only one of several steps in the process of evaluating, further developing, and implementing hybrid care models. The study design is explorative and produces validated estimates, rather than aiming to draw firm conclusions. Accordingly, we refer to value as potential, rather than empirically documented. A limitation of the study is its conceptual level of analysis and its lack of focus on specific clinical environments. Another important limitation that is important to acknowledge is the authors’ starting point, as well as their active, interpretive approach to conducting the study, which also shape the angle of the assessment.
Conclusions
This assessment provides early decision support concerning hybrid care models. Hybrid care models hold potential in meeting needs of stakeholders to a greater extent than current models and in improving system sustainability. The eHTA estimates that conservative changes toward digitalized practices result in a substantial impact on quantitative sustainability parameters and create new opportunities for organizing services. The eHTA emphasizes that implementation problems should be expected across mental health contexts and must be addressed early. Successful implementation requires continuous consideration of the user perspective of clinicians, starting from the conceptual phases of innovation. Promoting nuanced perceptions of digitalized care and developing clinician incentives that prompt praxis change are stakeholder-validated recommendations. eHTA methodology can be applied to align further research with critical knowledge gaps and to ensure that care models are reshaped according to stakeholder needs.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/S0266462326103985.
Acknowledgments
The authors wish to thank the stakeholders who participated in this study.
This work was supported by the Research Council of Norway through grant 309264 (Forhelse Research Centre for Digital Health Services).
Competing interests
The authors declare none.
Open access
