Hostname: page-component-6898cfcb55-zkb6p Total loading time: 0 Render date: 2026-11-11T03:49:20.332Z Has data issue: false hasContentIssue false
  • English
  • Français

Associations between keystroke and stylus metadata and depressive symptoms in adolescents

Published online by Cambridge University Press:  05 September 2024

Moonyoung Jang,
Youngeun Cho,
Do Hyung Kim,
Sunghyun Park,
Seonghyeon Park,
Ji-Won Hur,
Minah KimOpen the ORCID record for Minah Kim[Opens in a new window],
Kwangsu Cho,
Chang-Gun Lee and
Jun Soo Kwon
Moonyoung Jang
Affiliation:
Department of Psychiatry, Seoul National University College of Medicine, Seoul, Republic of KoreaDepartment of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea
Youngeun Cho
Affiliation:
Department of Brain and Cognitive Sciences, Seoul National University College of Natural Sciences, Seoul, Republic of Korea
Do Hyung Kim
Affiliation:
Department of Computer Science and Engineering, Seoul National University College of Engineering, Seoul, Republic of Korea
Sunghyun Park
Affiliation:
Department of Psychiatry, Seoul National University College of Medicine, Seoul, Republic of Korea
Seonghyeon Park
Affiliation:
3R Innovation Research Center, Seoul, Republic of Korea
Ji-Won Hur
Affiliation:
School of Psychology, Korea University, Seoul, Republic of Korea
Minah Kim*
Affiliation:
Department of Psychiatry, Seoul National University College of Medicine, Seoul, Republic of KoreaDepartment of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of Korea
Kwangsu Cho
Affiliation:
3R Innovation Research Center, Seoul, Republic of Korea
Chang-Gun Lee
Affiliation:
Department of Computer Science and Engineering, Seoul National University College of Engineering, Seoul, Republic of Korea
Jun Soo Kwon
Affiliation:
Department of Psychiatry, Seoul National University College of Medicine, Seoul, Republic of KoreaDepartment of Neuropsychiatry, Seoul National University Hospital, Seoul, Republic of KoreaDepartment of Brain and Cognitive Sciences, Seoul National University College of Natural Sciences, Seoul, Republic of KoreaInstitute of Human Behavioral Medicine, SNU-MRC, Seoul, Republic of Korea
*
Corresponding author: Minah Kim; Email: verte82@snu.ac.kr
Rights & Permissions [Opens in a new window]

Abstract

Background

Adolescents often experience a heightened incidence of depressive symptoms, which can persist without early intervention. However, adolescents often struggle to identify depressive symptoms, and even when they are aware of these symptoms, seeking help is not always their immediate response. This study aimed to explore the relationship between passively collected digital data, specifically keystroke and stylus data collected via mobile devices, and the manifestation of depressive symptoms.

Methods

A total of 927 first-year middle school students from schools in Seoul solved Korean language and math problems. Throughout this study, 77 types of keystroke and stylus data were collected, including parameters such as the number of key presses, tap pressure, stroke speed, and stroke acceleration. Depressive symptoms were measured using the self-rated Patient Health Questionnaire-9 (PHQ-9).

Results

Multiple regression analysis highlighted the significance of stroke length, speed, and acceleration, the average y-coordinate, the tap pressure, and the number of incorrect answers in relation to PHQ-9 scores. The keystroke and stylus metadata were able to reflect mood, energy, cognitive abilities, and psychomotor symptoms among adolescents with depressive symptoms.

Conclusions

This study demonstrates the potential of automatically collected data during school exams or classes for the early screening of clinical depressive symptoms in students. This study has the potential to serve as a cornerstone in the development of digital data frameworks for the early detection of depressive symptoms in adolescents.

Keywords

adolescentdepressiondigital phenotypingkeystroke metadatastylus metadata

Information

Type
Original Article
Information
Psychological Medicine , Volume 54 , Issue 11 , August 2024, pp. 3109 - 3114
Check for updates
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
Copyright © The Author(s), 2024. Published by Cambridge University Press

Introduction

Depressive symptoms have a high prevalence and can be accompanied by many mental illnesses, including schizophrenia, bipolar disorder, and major depressive disorder. The lifetime risk of experiencing depressive disorders is estimated to be between 15% and 18%, and the probability of experiencing depressive symptoms is even greater (Malhi & Mann, Reference Malhi and Mann2018; Shorey, Ng, & Wong, Reference Shorey, Ng and Wong2022; Substance Abuse and Mental Health Services Administration, 2021). Depressive symptoms are correlated with adverse clinical outcomes, including hopelessness, suicidal behavior, and demoralization, highlighting the critical need for early identification and treatment of these symptoms (Berardelli et al., Reference Berardelli, Serafini, Cortese, Fiasche, O'Connor and Pompili2020; Costanza et al., Reference Costanza, Vasileios, Ambrosetti, Shah, Amerio, Aguglia and Berardelli2022). Furthermore, in patients with other mental disorders not specifically diagnosed as depressive disorders, the presence of depressive symptoms can lead to greater functional impairment (Berardelli et al., Reference Berardelli, Rogante, Sarubbi, Erbuto, Lester and Pompili2021b; Upthegrove, Marwaha, & Birchwood, Reference Upthegrove, Marwaha and Birchwood2017). Children and adolescents are particularly vulnerable to depressive symptoms, and the prevalence of these symptoms in children and adolescents is reportedly greater than that in adults (Miller & Campo, Reference Miller and Campo2021). A significant number of adults who experience depressive symptoms trace their symptoms back to adolescence, and those with symptoms that emerge in adolescence often have poorer prognoses (Berardelli et al., Reference Berardelli, Innamorati, Sarubbi, Rogante, Erbuto, Lester and Pompili2021a; Berardelli et al., Reference Berardelli, Sarubbi, Rogante, Erbuto, Cifrodelli, Giuliani and Pompili2022; Weavers et al., Reference Weavers, Heron, Thapar, Stephens, Lennon, Bevan Jones and Rice2021). Therefore, the early identification and treatment of depressive symptoms, especially in younger populations, is critically important (Cuijpers, van Straten, Smit, Mihalopoulos, & Beekman, Reference Cuijpers, van Straten, Smit, Mihalopoulos and Beekman2008; Mendelson & Tandon, Reference Mendelson and Tandon2016).

Digital phenotyping is the process of measuring and analyzing individuals' behavioral and physiological characteristics using digital tools and technologies. This process typically involves the use of mobile devices such as smartphones to collect information on users' smartphone usage patterns, location data, social media activity, keyboard usage, and other sensor data. The collected data are subsequently utilized to monitor behaviors and diseases (Huckvale, Venkatesh, & Christensen, Reference Huckvale, Venkatesh and Christensen2019). Digital phenotyping is gaining significant attention as a valuable tool in mental health. Subtle changes can be detected early through digital phenotyping, allowing these minor alterations to be continuously monitored. Moreover, an advantage of this method is that it reflects individual characteristics, making it a powerful approach for personalized health monitoring and assessment (Brietzke et al., Reference Brietzke, Hawken, Idzikowski, Pong, Kennedy and Soares2019; Kamath, Leon Barriera, Jain, Keisari, & Wang, Reference Kamath, Leon Barriera, Jain, Keisari and Wang2022; Sequeira et al., Reference Sequeira, Perrotta, LaGrassa, Merikangas, Kreindler, Kundur and Strauss2020; Torous et al., Reference Torous, Bucci, Bell, Kessing, Faurholt-Jepsen, Whelan and Firth2021).

In digital phenotyping, two primary types of data are utilized. Active data are consciously provided or input by users based on their subjective perceptions and experiences. However, potentially low compliance is a disadvantage of utilizing this type of data because it requires active participation from users, and its accuracy can vary based on users' memories and perceptions. In contrast, passive data are automatically collected without the user's conscious involvement and include smartphone sensor data, location tracking data, and online activity logs. Devices or applications can automatically collect these passive data without direct user participation. They are not influenced by users' subjective evaluations or inputs and therefore reflect objective situations or activities. Additionally, passive data are collected continuously, yielding a real-time data stream (Abbas et al., Reference Abbas, Sauder, Yadav, Koesmahargyo, Aghjayan, Marecki and Galatzer-Levy2021; Torous et al., Reference Torous, Bucci, Bell, Kessing, Faurholt-Jepsen, Whelan and Firth2021). Due to these advantages, our research focused on exploring and utilizing passive data.

There are numerous types of passive data, including smartphone usage data, location data, biometric signals, environmental data, and keyboard and mouse usage patterns (Mohr, Zhang, & Schueller, Reference Mohr, Zhang and Schueller2017). Among these types of data, we were especially interested in keyboard and stylus data. Keyboard data allow a user's emotional state and cognitive functions to be inferred by analyzing keystroke speed, length, and duration. Stylus data offer insights into fine motor skills and emotional states by enabling the analysis of hand movements and pressure during stylus use as well as usage patterns (Mastoras et al., Reference Mastoras, Iakovakis, Hadjidimitriou, Charisis, Kassie, Alsaadi and Hadjileontiadis2019; Vesel et al., Reference Vesel, Rashidisabet, Zulueta, Stange, Duffecy, Hussain and Leow2020). While other types of passive data, such as location and app usage data, can reflect a user's behavioral patterns and activity levels, they are less capable of capturing the subtle cognitive and emotional states that can be captured using keyboard or stylus data. Given these characteristics, multiple reports suggest that keyboard and stylus data have the potential to reflect depressive symptoms in real time and capture fine distinctions, highlighting their utility in monitoring nuanced mental health states (Brietzke et al., Reference Brietzke, Hawken, Idzikowski, Pong, Kennedy and Soares2019; Kamath et al., Reference Kamath, Leon Barriera, Jain, Keisari and Wang2022; Sequeira et al., Reference Sequeira, Perrotta, LaGrassa, Merikangas, Kreindler, Kundur and Strauss2020; Torous et al., Reference Torous, Bucci, Bell, Kessing, Faurholt-Jepsen, Whelan and Firth2021).

Previous studies have explored the correlation between keystroke metadata and depression (Bennett, Ross, Baek, Kim, & Leow, Reference Bennett, Ross, Baek, Kim and Leow2022; Cao et al., Reference Cao, Zheng, Zhang, Yu, Piscitello, Zulueta and Leow2017; Huang, Cao, Yu, Wang, & Leow, Reference Huang, Cao, Yu, Wang and Leow2018; Mastoras et al., Reference Mastoras, Iakovakis, Hadjidimitriou, Charisis, Kassie, Alsaadi and Hadjileontiadis2019; Stange et al., Reference Stange, Zulueta, Langenecker, Ryan, Piscitello, Duffecy and Leow2018; Vesel et al., Reference Vesel, Rashidisabet, Zulueta, Stange, Duffecy, Hussain and Leow2020; Zulueta et al., Reference Zulueta, Piscitello, Rasic, Easter, Babu, Langenecker and Leow2018). However, few studies have evaluated the specific relationships between the individual components of keystroke data and depressive symptoms (Bennett et al., Reference Bennett, Ross, Baek, Kim and Leow2022; Mastoras et al., Reference Mastoras, Iakovakis, Hadjidimitriou, Charisis, Kassie, Alsaadi and Hadjileontiadis2019). Moreover, only one paper has investigated the correlation between depressive symptoms and keystroke metadata in children and adolescents (Braund et al., Reference Braund, O'Dea, Bal, Maston, Larsen, Werner-Seidler and Christensen2023). This paper demonstrated a positive correlation between mental symptoms, such as depression, anxiety, distress, and insomnia, and typing speed. However, no studies that specifically targeted adolescents were found. Based on these previous studies, we hypothesized that keystroke and stylus metadata would be correlated with depressive symptoms in adolescents. In particular, this study aimed to examine how specific components of keystroke and stylus metadata correlate with depressive symptoms in adolescents.

Methods

Participants

The study involved 927 first-year middle school students with an average age of 13 years. Of these participants, 483 (52.2%) were female. In Seoul, South Korea, there are 281 middle schools with a total of 66,358 first-year middle school students. To identify individuals who were willing to participate in the research, official letters were sent to each school to solicit individual applications. Enrollment in the study was contingent upon voluntary consent to participate from both the students and their parents. Consent forms were obtained from 935 students and their parents simultaneously. Individuals with intellectual disabilities or autism spectrum disorders, individuals with central nervous system conditions such as epilepsy or significant head injuries, individuals experiencing severe health problems that might result in psychological symptoms, and individuals who were unable to use study-specific tablets were excluded. Before participation, students and their parents or legal guardians were informed of and consented to the study's purpose, methodology, and type of data to be collected. Information on the participants is summarized in Table 1.

Table 1.

Demographic and clinical characteristics of the participants

Clinical and digital data collection

On the day of data collection, the students arrived at Seoul National University in Seoul, South Korea. The students were randomly assigned to classrooms in groups of approximately 30, replicating their regular classroom study environment. The desks and chairs were set up like those used in a typical school setting, with randomly assigned seating. A 30 minute session before data collection was provided to familiarize the students with the system interface and check for technical errors. Subsequently, the students completed the Patient Health Questionnaire-9 (PHQ-9), a widely used screening tool that is highly effective in screening for depressive symptoms among adolescents (Richardson et al., Reference Richardson, McCauley, Grossman, McCarty, Richards, Russo and Katon2010).

The students then spent 45 minute solving 20 Korean language problems aligned with the first-year middle school curriculum. All questions were subjective and required the students to type their responses using a keyboard. After a 15 minute break, the students spent another 45 minute solving 20 mathematics problems suited to their grade level. These math questions were subjective and were answered using a digital pen. During the Korean language and mathematics problem-solving sessions, the students were allowed to freely use both a keyboard and a digital stylus. While there was an overall time limit, there were no time constraints for individual questions. The students could also perform actions such as underlining or taking notes alongside the problems. An example of the screen on which the students solved the problems is shown in Fig. 1.

Figure 1.

Examples of problems solved by students during the process of digital data collection. Although the problems and options were presented in Korean, they have been translated to English for this paper. (a) An example of a Korean language problem for which responses were typed. (b) Example of a math problem for which responses were provided using a stylus pen.

Digital data were collected using two passive sensors built into the tablet computers while the students worked on the tasks. These sensors included an on-screen keyboard sensor and a touchscreen sensor. The tablets used were Samsung Galaxy Tab S7 FE models with Android 11, the same as those used by the students for their regular schoolwork. The students positioned their tablets on the desktop to perform typing and keyboard activities, intentionally reducing any possible sounds from the movement of the devices. The data were recorded unobtrusively at 24 Hz (or every 42.67 milliseconds) during the learning activities via a preinstalled module called Dr. Simon on the tablets. This process captured 77 distinct types of interactions with the keyboard and stylus, such as keystroke count, speed, acceleration, length, duration, trajectory, and pressure. The missing data accounted for 0.9% of the variance and were attributable to device or network errors.

In accordance with the guidelines set forth in the Declaration of Helsinki, the students and their parents were thoroughly informed about the study, after which written informed consent was obtained. The study's methodology was formally approved by the Institutional Review Board of Korea University, as indicated by authorization number IRB-2023-0144. All procedures were carried out following review and approval by the Seoul Metropolitan Office of Education to ensure ethical compliance.

Statistical analysis

In this study, we standardized the measurements of various indicators, including self-reported clinical scores, frequencies, durations (measured in milliseconds), lengths (measured in pixels), pressure intensities, and ratios. This was achieved through z-score normalization by setting the mean of each indicator to zero and the standard deviation to one. This normalization allowed for comparisons across indicators with different distribution characteristics.

Traditionally, multiple regression with stepwise selection is a popular technique for identifying crucial indicators or measurements. Instead, due to reliability and adequacy issues, we applied a bootstrap-based multiple regression approach. For this purpose, we generated 10,000 sample sets using the Mersenne Twister algorithm for random number generation. This approach enhanced the stability of the model and was useful for handling outliers.

Results

Multiple regression

Regression analysis of the PHQ-9 scores was performed. In this model, the average keystroke acceleration, the number of incorrect answers, the average keystroke length, the average tap pressure, and the average keystroke speed were found to be predictors of depressive symptom scores. The results indicated that a higher average keystroke acceleration and length as well as a greater number of incorrect answers were associated with higher PHQ-9 scores. Conversely, a lower tap pressure and stroke speed were associated with higher PHQ-9 scores. The R 2 value was 0.0252. The results of this linear regression are presented in Fig. 2. Additionally, a dot plot illustrating the correlation between the standardized variables with a focus on the highest coefficient observed between the average stroke length and PHQ-9 score is displayed in Fig. 3.

Figure 2.

Heatmap showing the associations between PHQ-9 scores and keystroke features.

Figure 3.

Association between the PHQ-9 score and average stroke acceleration.

Discussion

This study represents a pioneering effort to reveal the digital phenotype of depressive symptoms among adolescents. The ultimate goal of the study was to enable the early diagnosis and timely intervention of mental health issues among students within the school setting. In this study, the digital data variables that explained the PHQ-9 scores included the number of incorrect answers; the length, speed, and acceleration of strokes; the average vertical position of strokes; and the average pressure exerted while typing on the keyboard. Higher PHQ-9 scores were associated with a lower tap pressure, a greater number of incorrect answers, and more frequent use of strokes toward the upper part of the tablet. Additionally, PHQ-9 scores decreased as the average stroke length increased, the average stroke speed decreased, and the average keystroke acceleration increased.

The regression analysis results indicated that greater tap pressure was associated with lower PHQ-9 scores. A lack of energy and lethargy are typical symptoms of depression. We hypothesized that these depressive symptoms might influence tap pressure. Moreover, studies published in 2018 and 2014 reported that adults exhibit greater keyboard pressure under high stress (Exposito, Hernandez, & Picard, Reference Exposito, Hernandez and Picard2018; Hernandez, Paredes, Roseway, & Czerwinski, Reference Hernandez, Paredes, Roseway and Czerwinski2014). These studies suggested that the observed phenomenon could be due to increased muscle tension during stressful situations. While stress and depressive states are distinct, the correlation between mental symptoms and tap pressure, mediated by muscle tone, is noteworthy. Depressive symptoms can manifest as either psychomotor agitation or psychomotor retardation. In adolescents, psychomotor retardation is more commonly observed in the presence of depressive symptoms, which could contribute to the negative correlation between tap pressure and depressive symptoms observed in this study. However, the correlation between tap pressure and depressive symptoms is not well documented, suggesting the need for more extensive research in this area.

The regression analysis results revealed that higher PHQ-9 scores were associated with a greater number of incorrect answers and a greater distribution of strokes toward the upper part of the tablet. Depression is closely linked to impaired cognitive functions such as concentration and is known to cause functional impairments. Notably, a decline in academic performance among students is a well-documented issue associated with depression (Mangione et al., Reference Mangione, Barry, Nicholson, Cabana, Chelmow, Coker and Wong2022). Thus, the positive correlation between the number of incorrect answers and the depression score is consistent with established knowledge. However, the correlation between strokes occurring predominantly in the upper part of the tablet and symptoms of depression is more challenging to explain. Considering that stroke data were collected mainly during math problem-solving sessions, where problems are typically positioned at the top and answers are typically positioned at the bottom, it is plausible that individuals might spend more time thinking about problems than writing answers, resulting in strokes being predominantly located in the upper part of the tablet. This could be related to depression given its close association with cognitive impairments such as reduced concentration. Nevertheless, this remains a hypothesis, and more detailed data collection and analysis are required for validation.

This study revealed that more depressive symptoms were associated with longer stroke duration, slower stroke speed, and faster stroke acceleration. Previous research focusing on writing patterns in patients with depression, particularly individuals aged 30–40 years, revealed that patients with depressive symptoms typically exhibit slower stroke speeds with greater variation and longer durations and lengths of strokes (Mergl et al., Reference Mergl, Juckel, Rihl, Henkel, Karner, Tigges and Hegerl2004, Reference Mergl, Pogarell, Juckel, Rihl, Henkel, Frodl and Hegerl2007). The authors of these studies argued that these observations could be related to the psychomotor retardation and basal ganglia dysfunction observed in patients with depression. These findings are consistent with the results of the present study, in which longer stroke lengths and slower stroke speeds were observed. Additionally, the high variability in stroke speed may suggest greater stroke acceleration. Therefore, the results of this study align with previous research findings. Furthermore, a previous study reported a correlation between baseline typing instability and depressive symptoms, which also corresponds with the positive correlation between depressive symptoms and acceleration observed in the present study (Stange et al., Reference Stange, Zulueta, Langenecker, Ryan, Piscitello, Duffecy and Leow2018).

In previous research conducted with children as participants, depressive symptoms were reported to be associated with faster typing speeds (Braund et al., Reference Braund, O'Dea, Bal, Maston, Larsen, Werner-Seidler and Christensen2023). The authors of this study attributed the observed phenomenon to psychomotor agitation and noted that other research predominantly showed that slower typing speeds could be due to psychomotor retardation. Consequently, the authors advocated for additional research to further explore these disparities. The variations are likely due to the different manifestations of depressive symptoms in children, adolescents, and elderly people. In children, depressive symptoms often present as prominent psychomotor agitation, whereas in adolescents, psychomotor retardation is more pronounced (Sadock, Sadock, & Ruiz, Reference Sadock, Sadock and Ruiz2017). An earlier study in which data were collected from participants with an average age of 8 years revealed results indicative of psychomotor agitation. In contrast, the present study, which targeted adolescents, revealed findings that reflect psychomotor retardation.

The research findings highlight a significant relationship between higher depression scale scores and noticeable changes in keystroke and stylus data in contrast to lower depression scale scores. These differences could be attributed to psychomotor disruptions and cognitive impairment. The primary aim of this study was to pinpoint potential digital indicators that represent approximately four (depressive/irritable mood, psychomotor retardation/agitation, loss of energy/fatigue, loss of concentration) out of the nine symptoms (depressive/irritable mood, diminished interest/pleasure, significant increase/decrease in weight/appetite, insomnia/hypersomnia, psychomotor retardation/agitation, loss of energy/fatigue, feelings of excessive guilt, loss of concentration, suicidal ideation) of depression by analyzing keystroke and stylus data obtained in an examination context (Thapar, Eyre, Patel, & Brent, Reference Thapar, Eyre, Patel and Brent2022). In addition, children and adolescents frequently exhibit depressive symptoms without a concurrent sense of sadness and often overlook or inadequately report their symptoms. This study investigated the association between passively collected digital data in a routine academic testing environment and the manifestation of depressive symptoms. The cumulative efforts of this research, which aimed to establish a digital phenotype for depression in adolescents, could empower educators to identify and address depressive symptoms in students in a timely manner.

Strengths and limitations

This study has several limitations. First, only keystroke and stylus data were collected, and other passive data were not obtained. Additionally, the data were collected in less than 3 hours, making it challenging to adequately capture longitudinal changes. These factors may have contributed to the limited explanatory power observed in the multiple regression analysis. The fact that clinicians did not diagnose the participants directly but relied on self-reported scores to measure depressive symptoms is also a limitation. Furthermore, the use of the PHQ-9 instead of the PHQ-9 modified for Adolescents (PHQ-A) may also be a limitation. Additionally, the sample used in this study included almost no adolescents with clinically significant psychopathology, which may be considered a limitation. Above all, due to the scarcity of research identifying digital phenotypes associated with depression in adolescents, this study assumes a preliminary nature, making it challenging to design and test precise hypotheses based on existing research.

This study aimed to replicate a school examination scenario to passively collect data. This could serve as a foundation for establishing a system within schools where teachers can regularly screen students for depressive symptoms without additional effort and allow for prompt intervention when necessary. Another notable strength of this research compared to previous digital phenotype studies of depression is the considerably large sample size. Furthermore, ethical considerations concerning the capacity for informed consent in digital phenotype studies involving adolescents are scarce. This study's strength lies in its focus on 13-year-old adolescents.

Conclusions

This research can be considered a foundational study in the emerging field of digital phenotype research rather than a conclusive study, particularly for understanding depressive symptoms in adolescents. Building upon this foundation, more rigorous hypotheses and experimental designs must be formulated for further research. Based on this research, by conducting a sufficiently long longitudinal study to collect digital data over an extended period and performing subsequent research including a long-term and meticulous evaluation of clinically significant pathology, it would be plausible to screen for and promptly intervene in depressive symptoms in adolescents, which would lead to significant societal costs if they become chronic. Additionally, research that focuses on identifying digital phenotypes capable of predicting the development of various mental disorders, including suicide risk and schizophrenia, in students will not only facilitate early diagnosis and timely intervention but also ultimately aid in reducing community health expenses.

Acknowledgments

The authors extend their deep gratitude to the Seoul Metropolitan Office of Education for their outstanding support and collaboration. As the Office spearheaded the Digital Buddy (Di-Bud) project aimed at digitally transforming schools, their assistance was invaluable in gathering essential digital data using the Dr. Simon (3R Innovation Inc. URL: https://drsimon.ai). This research was supported by the Bio & Medical Technology Development Program, the Brain Science Convergence Research Program, and an Institute of Information & Communications Technology Planning & Evaluation (IITP) grant (2021M3A9E408078412, RS-2023-00266120, and RS-2021-II212068), which is funded by the Ministry of Science & ICT. This research was also supported by the Alchemist Project (1415187369, 20025696), which is funded by the Ministry of Trade, Industry & Energy (MOTIE, Korea).

References

Abbas, A., Sauder, C., Yadav, V., Koesmahargyo, V., Aghjayan, A., Marecki, S., … Galatzer-Levy, I. R. (2021). Remote digital measurement of facial and vocal markers of major depressive disorder severity and treatment response: A pilot study. Frontiers in Digital Health, 3, 610006. doi: 10.3389/fdgth.2021.610006CrossRefGoogle ScholarPubMed
Bennett, C. C., Ross, M. K., Baek, E., Kim, D., & Leow, A. D. (2022). Predicting clinically relevant changes in bipolar disorder outside the clinic walls based on pervasive technology interactions via smartphone typing dynamics. Pervasive and Mobile Computing, 83, 101598. doi: 10.1016/j.pmcj.2022.101598CrossRefGoogle Scholar
Berardelli, I., Innamorati, M., Sarubbi, S., Rogante, E., Erbuto, D., Lester, D., & Pompili, M. (2021a). Demographic and clinical correlates of high-lethality suicide attempts: A retrospective study in psychiatric inpatients. Journal of Psychiatric Practice, 27(6), 410–416. doi: 10.1097/pra.0000000000000579CrossRefGoogle ScholarPubMed
Berardelli, I., Rogante, E., Sarubbi, S., Erbuto, D., Lester, D., & Pompili, M. (2021b). The importance of suicide risk formulation in schizophrenia. Frontiers in Psychiatry, 12, 779684. doi: 10.3389/fpsyt.2021.779684CrossRefGoogle ScholarPubMed
Berardelli, I., Sarubbi, S., Rogante, E., Erbuto, D., Cifrodelli, M., Giuliani, C., … Pompili, M. (2022). Exploring risk factors for re-hospitalization in a psychiatric inpatient setting: A retrospective naturalistic study. BMC Psychiatry, 22(1), 821. doi: 10.1186/s12888-022-04472-3CrossRefGoogle Scholar
Berardelli, I., Serafini, G., Cortese, N., Fiasche, F., O'Connor, R. C., & Pompili, M. (2020). The involvement of hypothalamus-pituitary-adrenal (HPA) axis in suicide risk. Brain Sciences, 10(9), 653. doi: 10.3390/brainsci10090653CrossRefGoogle ScholarPubMed
Braund, T. A., O'Dea, B., Bal, D., Maston, K., Larsen, M., Werner-Seidler, A., … Christensen, H. (2023). Associations between smartphone keystroke metadata and mental health symptoms in adolescents: Findings from the future proofing study. JMIR Mental Health, 10, e44986. doi: 10.2196/44986CrossRefGoogle ScholarPubMed
Brietzke, E., Hawken, E. R., Idzikowski, M., Pong, J., Kennedy, S. H., & Soares, C. N. (2019). Integrating digital phenotyping in clinical characterization of individuals with mood disorders. Neuroscience & Biobehavioral Reviews, 104, 223–230. doi: 10.1016/j.neubiorev.2019.07.009CrossRefGoogle ScholarPubMed
Cao, B. K., Zheng, L., Zhang, C. W., Yu, P. S., Piscitello, A., Zulueta, J., … Leow, A. D. (2017). DeepMood: Modeling mobile phone typing dynamics for mood detection. Kdd'17: Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, 747–755. doi: 10.1145/3097983.3098086CrossRefGoogle Scholar
Costanza, A., Vasileios, C., Ambrosetti, J., Shah, S., Amerio, A., Aguglia, A., … Berardelli, I. (2022). Demoralization in suicide: A systematic review. Journal of Psychosomatic Research, 157, 110788. doi: 10.1016/j.jpsychores.2022.110788CrossRefGoogle ScholarPubMed
Cuijpers, P., van Straten, A., Smit, F., Mihalopoulos, C., & Beekman, A. (2008). Preventing the onset of depressive disorders: A meta-analytic review of psychological interventions. American Journal of Psychiatry, 165(10), 1272–1280. doi: 10.1176/appi.ajp.2008.07091422CrossRefGoogle ScholarPubMed
Exposito, M., Hernandez, J., & Picard, R. W. (2018). Affective keys: Towards unobtrusive stress sensing of smartphone users. Paper presented at the Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, Barcelona, Spain. https://doi.org/10.1145/3236112.3236132CrossRefGoogle Scholar
Hernandez, J., Paredes, P., Roseway, A., & Czerwinski, M. (2014). Under pressure: Sensing stress of computer users. 32nd Annual ACM Conference on Human Factors in Computing Systems (CHI 2014), 51–60. doi: 10.1145/2556288.2557165CrossRefGoogle Scholar
Huang, H., Cao, B. K., Yu, P. S., Wang, C. D., & Leow, A. D. (2018). dpMood: Exploiting local and periodic typing dynamics for personalized mood prediction. 2018 IEEE International Conference on Data Mining (ICDM), 157–166. doi: 10.1109/Icdm.2018.00031CrossRefGoogle Scholar
Huckvale, K., Venkatesh, S., & Christensen, H. (2019). Toward clinical digital phenotyping: A timely opportunity to consider purpose, quality, and safety. NPJ Digital Medicine, 2, 88. doi: 10.1038/s41746-019-0166-1CrossRefGoogle ScholarPubMed
Kamath, J., Leon Barriera, R., Jain, N., Keisari, E., & Wang, B. (2022). Digital phenotyping in depression diagnostics: Integrating psychiatric and engineering perspectives. World Journal of Psychiatry, 12(3), 393–409. doi: 10.5498/wjp.v12.i3.393CrossRefGoogle ScholarPubMed
Malhi, G. S., & Mann, J. J. (2018). Depression. The Lancet, 392(10161), 2299–2312. doi: 10.1016/S0140-6736(18)31948-2CrossRefGoogle ScholarPubMed
Mangione, C. M., Barry, M. J., Nicholson, W. K., Cabana, M., Chelmow, D., Coker, T. R., … Wong, J. B. (2022). Screening for depression and suicide risk in children and adolescents: US preventive services task force recommendation statement. JAMA, 328(15), 1534–1542. doi: 10.1001/jama.2022.16946Google ScholarPubMed
Mastoras, R. E., Iakovakis, D., Hadjidimitriou, S., Charisis, V., Kassie, S., Alsaadi, T., … Hadjileontiadis, L. J. (2019). Touchscreen typing pattern analysis for remote detection of the depressive tendency. Scientific Reports, 9(1), 13414. doi: 10.1038/s41598-019-50002-9CrossRefGoogle ScholarPubMed
Mendelson, T., & Tandon, S. D. (2016). Prevention of depression in childhood and adolescence. Child and Adolescent Psychiatric Clinics of North America, 25(2), 201–218. doi: 10.1016/j.chc.2015.11.005CrossRefGoogle ScholarPubMed
Mergl, R., Juckel, G., Rihl, J., Henkel, V., Karner, M., Tigges, P., … Hegerl, U. (2004). Kinematical analysis of handwriting movements in depressed patients. Acta Psychiatrica Scandinavica, 109(5), 383–391. doi: 10.1046/j.1600-0447.2003.00262.xCrossRefGoogle ScholarPubMed
Mergl, R., Pogarell, O., Juckel, G., Rihl, J., Henkel, V., Frodl, T., … Hegerl, U. (2007). Hand-motor dysfunction in depression: Characteristics and pharmacological effects. Clinical EEG and Neuroscience, 38(2), 82–88. doi: 10.1177/155005940703800210CrossRefGoogle ScholarPubMed
Miller, L., & Campo, J. V. (2021). Depression in adolescents. The New England Journal of Medicine, 385(5), 445–449. doi: 10.1056/NEJMra2033475CrossRefGoogle ScholarPubMed
Mohr, D. C., Zhang, M., & Schueller, S. M. (2017). Personal sensing: Understanding mental health using ubiquitous sensors and machine learning. Annual Review of Clinical Psychology, 13, 23–47. doi: 10.1146/annurev-clinpsy-032816-044949CrossRefGoogle ScholarPubMed
Richardson, L. P., McCauley, E., Grossman, D. C., McCarty, C. A., Richards, J., Russo, J. E., … Katon, W. (2010). Evaluation of the patient health questionnaire-9 item for detecting major depression among adolescents. Pediatrics, 126(6), 1117–1123. doi: 10.1542/peds.2010-0852CrossRefGoogle ScholarPubMed
Sadock, B. J., Sadock, V. A., & Ruiz, P. (2017). Kaplan & Sadock's comprehensive textbook of psychiatry (10th ed., Vols. 1–2). Philadelphia, PA, USA: Lippincott Williams & Wilkins Publishers.Google Scholar
Sequeira, L., Perrotta, S., LaGrassa, J., Merikangas, K., Kreindler, D., Kundur, D., … Strauss, J. (2020). Mobile and wearable technology for monitoring depressive symptoms in children and adolescents: A scoping review. Journal of Affective Disorders, 265, 314–324. doi: 10.1016/j.jad.2019.11.156CrossRefGoogle ScholarPubMed
Shorey, S., Ng, E. D., & Wong, C. H. J. (2022). Global prevalence of depression and elevated depressive symptoms among adolescents: A systematic review and meta-analysis. British Journal of Clinical Psychology, 61(2), 287–305. doi: 10.1111/bjc.12333CrossRefGoogle ScholarPubMed
Stange, J. P., Zulueta, J., Langenecker, S. A., Ryan, K. A., Piscitello, A., Duffecy, J., … Leow, A. (2018). Let your fingers do the talking: Passive typing instability predicts future mood outcomes. Bipolar Disorders, 20(3), 285–288. doi: 10.1111/bdi.12637CrossRefGoogle ScholarPubMed
Substance Abuse and Mental Health Services Administration. (2021). Key substance use and mental health indicators in the United States. Retrieved from https://www.samhsa.gov/data/sites/default/files/reports/rpt39443/2021NSDUHFFRRev010323.pdfGoogle Scholar
Thapar, A., Eyre, O., Patel, V., & Brent, D. (2022). Depression in young people. The Lancet, 400(10352), 617–631. doi: 10.1016/s0140-6736(22)01012-1CrossRefGoogle ScholarPubMed
Torous, J., Bucci, S., Bell, I. H., Kessing, L. V., Faurholt-Jepsen, M., Whelan, P., … Firth, J. (2021). The growing field of digital psychiatry: Current evidence and the future of apps, social media, chatbots, and virtual reality. World Psychiatry, 20(3), 318–335. doi: 10.1002/wps.20883CrossRefGoogle ScholarPubMed
Upthegrove, R., Marwaha, S., & Birchwood, M. (2017). Depression and schizophrenia: Cause, consequence, or trans-diagnostic issue? Schizophrenia Bulletin, 43(2), 240–244. doi: 10.1093/schbul/sbw097Google ScholarPubMed
Vesel, C., Rashidisabet, H., Zulueta, J., Stange, J. P., Duffecy, J., Hussain, F., … Leow, A. (2020). Effects of mood and aging on keystroke dynamics metadata and their diurnal patterns in a large open-science sample: A BiAffect iOS study. Journal of American Medical Informatics Association, 27(7), 1007–1018. doi: 10.1093/jamia/ocaa057CrossRefGoogle Scholar
Weavers, B., Heron, J., Thapar, A. K., Stephens, A., Lennon, J., Bevan Jones, R., … Rice, F. (2021). The antecedents and outcomes of persistent and remitting adolescent depressive symptom trajectories: A longitudinal, population-based English study. The Lancet Psychiatry, 8(12), 1053–1061. doi: 10.1016/S2215-0366(21)00281-9CrossRefGoogle ScholarPubMed
Zulueta, J., Piscitello, A., Rasic, M., Easter, R., Babu, P., Langenecker, S. A., … Leow, A. (2018). Predicting mood disturbance severity with mobile phone keystroke metadata: A BiAffect digital phenotyping study. Journal of Medical Internet Research, 20(7), e241. doi: 10.2196/jmir.9775CrossRefGoogle ScholarPubMed
Figure 0

Table 1.Demographic and clinical characteristics of the participants

Figure 1

Figure 1.Examples of problems solved by students during the process of digital data collection. Although the problems and options were presented in Korean, they have been translated to English for this paper. (a) An example of a Korean language problem for which responses were typed. (b) Example of a math problem for which responses were provided using a stylus pen.

Figure 2

Figure 2.Heatmap showing the associations between PHQ-9 scores and keystroke features.

Figure 3

Figure 3.Association between the PHQ-9 score and average stroke acceleration.