Text-to-image models are enabling efficient design space exploration, rapidly generating images from text prompts. However, many generative AI tools are imperfect for product design applications as they are not built for the goals and requirements of product design. The unclear link between text input and image output further complicates their application. This work empirically investigates design space exploration strategies that can successfully yield product images that are feasible, novel and aesthetic – three common goals in product design. Specifically, users’ actions within the global and local editing modes, including their time spent, prompt length, mono versus multi-criteria prompts, and goal orientation of prompts, are analyzed. Key findings reveal the pivotal role of mono versus multi-criteria and goal orientation of prompts in achieving specific design goals over time and prompt length. The study recommends prioritizing the use of multi-criteria prompts for feasibility and novelty during global editing while favoring mono-criteria prompts for aesthetics during local editing. Overall, this article underscores the nuanced relationship between the AI-driven text-to-image models and their effectiveness in product design, urging designers to carefully structure prompts during different editing modes to better meet the unique demands of product design.

This research proposes a novel conceptual framework that combines the concepts of Human-Computer Interaction (HCI) and Ambient Intelligence (AmI). The proposed framework aims to shed light on the importance of considering the needs and the social interactions of various building occupants in different types of buildings and designing HBI strategies accordingly. Specifically, we take educational buildings as a case that is less explored in the HBI research and apply the proposed framework, investigating how HBI strategies and interactions should be designed to address the needs of students, as primary occupants. Focus groups and semi-structured interviews were conducted among students in a flagship smart engineering building at Virginia Tech. Qualitative coding and concept mapping were used to analyze the qualitative data and determine the impact of occupant-specific needs on the learning experience of students. “Finding study space” was found to have the highest direct impact on the learning experience of students, and “Indoor Environment Quality (IEQ)” was found to have the highest indirect impact. The results show a clear need to integrate occupant needs in designing HBI strategies in different types of buildings. Finally, we discuss new ideas for designing potential Intelligent User Interfaces (IUI) to address the identified needs.

Virtual reality (VR) sketching has many advantages for product design and tends to be more and more used among designers and non-designers (end-users). Nevertheless, few studies have focused on the skills needed to use VR sketching for non-designers especially VR novices in VR software. This study focuses on identifying the cognitive impact of VR sketching compared to traditional sketching on VR expert and VR novice in an experimental setting. Thirty-one participants composed of VR experts (N = 15) and VR novices (N = 16) completed a mental rotation test and then performed one traditional paper and pencil sketching task and two VR sketching tasks. We also measured the participants’ movements when using the VR sketching. Results show that VR experts perform better than VR novices in VR sketching because training is an essential element for the quality of traditional and VR sketching. Nevertheless, VR novices with previous training in traditional drawing and/or high mental rotation skills will be able to produce good-quality sketches. In addition, the results show that users moving more in the immersive environment performed better quality sketches if the drawing requires more complex shapes. Our results suggest that VR sketching can be complex to use for a part of the population that may be end-users, especially for those with little experience in traditional and VR sketching and with poor visuospatial abilities. We, therefore, advise to check the non-designers’ prior skills, otherwise, it will be necessary to train these users in VR sketching.

Mixed Reality enables individuals to visualise and interact with artefacts and environments through a combination of physical and virtual assets. It has received increased interest from the design community as a means to accelerate, enrich and enhance prototyping activities. This article concerns MR’s ability to deceive an individual through the combination of virtual and physical assets and their underlying traits (e.g., mass, size), and a user’s cognitive ability to ‘join the dots’. If properly implemented, MR could save time and resources by reducing the required prototype fidelity and the need to fully realise variants. However, there is a gap in understanding how the traits of physical and virtual assets and cognition combine to form reality. This article presents a study investigated the role mass, virtual and physical model size played on users perception of an MR prototype. The relative impact of these factors was determined by varying these parameters and assessing the user’s perceived change. The key finding from this study was that the virtual model size had a far greater influence on prototype perceived by the user. This suggests that the required physical fidelity of an MR prototype can be lower than the virtual. Furthermore, exploring size design variants can be achieved exclusively through changes to the virtual model.

Evaluating the algorithmic behavior of interactive systems is complex and time-consuming. Developers increasingly recognize the importance of accountability for their algorithmic creations’ unanticipated behavior and resulting implications. To mitigate this phenomenon, developers not only need to concentrate on the observable inaccuracies that can be measured quantitatively but also the more subjective outcomes that can perpetuate social bias, which are challenging to identify. We require a new approach that involves humans in scrutinizing algorithmic behavior. It leverages a combination of quantitative and qualitative methods to support an ethical, value-aligned design and a system’s lifecycle, informed by users’ perception and values. To date, the literature lacks an agreed-upon framework for such an approach. Consequently, we propose an oversight framework, Modular Oversight Methodology (MOM), which aids developers in assessing the behavior of their systems by involving a carefully crowdsourced society-in-the-loop. The framework facilitates the development and execution of an oversight process and can be tweaked according to the domain and application of use. Through such an oversight process, developers can assess the human perception of the algorithmic behavior under inspection, and extract valuable insights that will aid in assessing its implications. We present the MOM framework, as a first step toward tailoring more robust, domain-specific solutions to exercise human oversight over algorithms, as a means for software developers to keep the generated output of their solutions fair and trustworthy.

Within the broad context of design research, joint attention within co-creation represents a critical component, linking cognitive actors through dynamic interactions. This study introduces a novel approach employing deep learning algorithms to objectively quantify joint attention, offering a significant advancement over traditional subjective methods. We developed an optimized deep learning algorithm, YOLO-TP, to identify participants’ engagement in design workshops accurately. Our research methodology involved video recording of design workshops and subsequent analysis using the YOLO-TP algorithm to track and measure joint attention instances. Key findings demonstrate that the algorithm effectively quantifies joint attention with high reliability and correlates well with known measures of intersubjectivity and co-creation effectiveness. This approach not only provides a more objective measure of joint attention but also allows for the real-time analysis of collaborative interactions. The implications of this study are profound, suggesting that the integration of automated human activity recognition in co-creation can significantly enhance the understanding and facilitation of collaborative design processes, potentially leading to more effective design outcomes.

Loneliness and social isolation are prevalent concerns among older adults and can lead to negative health consequences and a reduced lifespan. New technologies are increasingly being developed to help address loneliness and social isolation in older adults, including monitoring systems, social networks, robots, companions, smart televisions, augmented reality (AR) and virtual reality (VR) applications. This systematic review maps human-centered design (HCD) and user-centered design (UCD) approaches, human needs, and contextual factors considered in current technological interventions designed to address the problems of loneliness and social isolation in older adults. We conducted a scoping review and in-depth examination of 98 papers through a qualitative content analysis. We found 12 studies applying either an HCD or UCD approach and observed strengths in continuous user involvement and implementation in field studies but limitations in participant inclusion criteria and methodological reporting. We also observed the consideration of important human needs and contextual factors. However, more research is needed on stakeholder perspectives, the functioning of applications in different housing environments, as well as studies that include diverse socio-economic groups.

Managing cognitive load is central to designing interactive systems, particularly within augmented reality (AR) environments that impose complex and immersive demands. This study investigates two complementary approaches in parts to managing cognitive load in AR: refining interaction modalities and integrating adaptive physiological feedback. In Part 1, eye-tracking and hand-based modalities are evaluated across tasks of varying difficulty, using skin conductance responses (SCRs) as a proxy for cognitive load. Results show that while hand gestures improved task performance in simple tasks, cognitive load levels are comparable across modalities. In Part 2, an adaptive feedback system based on a signal-derived metric, cumulative SCR (CSCR), is developed to trigger short rest interventions during sustained cognitive load. Statistical analyses illustrate that rest interventions significantly reduced cumulative cognitive load, though their effect on task performance was inconclusive. These findings emphasize the trade-offs between cognitive relief and performance continuity and highlight the potential of physiologically adaptive systems in supporting cognitive-aware interaction design.