1. Introduction
The increasing importance of intelligent systems today, especially in the context of creative processes, has fuelled the need to make systems that are based on stimulating human creativity more efficiently (Shneiderman Reference Shneiderman2002; Frich et al. Reference Frich, MacDonald Vermeulen, Remy, Biskjaer and Dalsgaard2019; Shireen Reference Shireen2020). Human-computer co-creative systems are characterised by the ability of computers to collaborate with humans in creating new ideas (Karimi et al. Reference Karimi, Grace, Maher and Davis2018; Kantosalo Reference Kantosalo2019). Currently, despite the vast contributions, according to Rezwana & Maher (Reference Rezwana and Maher2023), human-computer co-creative systems still present several challenges due to the unpredictable nature of human creativity and the different strategies and reasoning individuals use when developing creative artefacts. Given this unpredictability, it is expected that humans will vary in their level of control during the creative process, but also that the final result will be influenced by both the process followed and the objectives or experience (Shneiderman et al. Reference Shneiderman, Fischer, Czerwinski, Resnick, Myers, Candy, Edmonds, Eisenberg, Giaccardi, Hewett, Jennings, Kules, Nakakoji, Nunamaker, Pausch, Selker, Sylvan and Terry2006; Rezwana & Maher Reference Rezwana and Maher2023; Rovira Martins et al. Reference Rovira Martins, Cunha, Martins and Boavida2024). This leads us to the need to approach the development of human-computer co-creative tools not merely as a technical challenge, but as a process inherently characterised by subjectivity and uniqueness (Poon & Maher Reference Poon and Maher1997). This is even more evident when these tools are applied to design, which is already a field with procedural nuances, making the balance between scientific rigour and creative practice even more difficult (Lawton et al. Reference Lawton, Ibarrola, Ventura and Grace2023). We therefore need to understand how different behaviours lead to distinct experiences (Lawton et al. Reference Lawton, Ibarrola, Ventura and Grace2023).
Considering the aspects mentioned above, we aim to understand how users’ prior knowledge of design influences the perception of an ideation human-computer co-creative system, including the level of trust, the role the system adopts, the dynamics of the creative process and even the final result. Given the novelty of examining how expertise shapes human-computer co-creative interactions throughout a complete design workflow, we adopted an exploratory approach. Therefore, instead of testing pre-defined hypotheses to guide the study design and data analysis, we formulated the following research questions:
-
• RQ1: How does the level of design expertise influence the human-AI collaborative experience and the process during a design task?
-
• RQ2: How does the level of design expertise influence the user’s perception of the computational partner in terms of trustworthiness and the role it adopts during the design process?
-
• RQ3: How does the level of design expertise influence the level of creativity in the final result and during a human-AI collaborative design task?
We used PatternPursuit, developed by Rovira Martins et al. (Reference Rovira Martins, Cunha, Martins and Boavida2024), as our study environment. We chose this system because it enables human-AI collaboration throughout the creative process, from exploring references to finalising a design artefact, within a structured yet flexible environment that reflects real-world design workflows. While we recognise the limitations of testing our hypotheses using a single system, PatternPursuit stands out as, to the best of our knowledge, the only available example that combines these characteristics with explainability strategies to improve users’ trust in the creative process. These features support a deeper exploration of how the level of expertise influences the collaborative experience between humans and AI (RQ1) and the perception of the AI partner (RQ2).
Furthermore, the system initially provides users with visual references, specifically works by Portuguese graphic designer Sebastião Rodrigues, although it could operate with another set of references. This system uses conceptual and visual exploration techniques to decompose the works’ elements into graphic raw materials, enabling the development of graphic patterns for wrapping paper (see Figure 1). This complete process of using Rodrigues’ elements to create something new not only allows users to become deeply involved, supporting the analysis of the impact of the design experience on the level of creativity and the type of process during collaboration, but also makes it possible to analyse how it influences the level of creativity and the perception of the presence of references in the final result (RQ3). Although the system only explores the works of a single designer, it counters design fixation by allowing the user to explore different ideas (Jansson & Smith Reference Jansson and Smith1991; Son et al. Reference Son, Choi, Kim, Kim and Kim2024). We conducted a study with 30 participants to explore the mentioned aspects and present the findings as insights for designing effective human-AI co-creative systems, grounded in a deeper understanding of the target audience’s diverse needs and their grasp of the human-AI creative process. In this paper, we make the following contributions:
-
• We demonstrate that expertise creates a fundamental division in the relationship between the process and the perception of the final result. In other words, while non-experts are motivated by originality and exploration, which is linked to the quality of the final result, experts operate on the basis of a dependency between agency and value, in which the validation of the final result is conditioned by their perception of control during the process.
-
• We have identified a dichotomy of interaction in which the same system plays opposing roles: for non-experts, it acts as a capability extender that fills skill gaps; for experts, it acts as a semi-autonomous partner that challenges authorship, highlighting the need for future systems to support explicit negotiation phases.
Graphic patterns for wrapping paper developed by the participants in this study using PatternPursuit.
Figure 1. Long description
The figure consists of three square panels arranged horizontally.
* The left panel features a grid of the capital letter F in a bold black sans-serif font. Each letter is slightly tilted and accompanied by a gray stippled or splattered texture behind its lower half. The letters are arranged in four rows and four columns.
* The center panel displays a vertical, symmetrical pattern. Two thick vertical bands of gray and orange textured triangles run down the left and right sides. Between these bands and on the far edges are small, repeating black floral or leaf-like icons arranged in vertical columns.
* The right panel shows a scattered pattern of orange and yellow shapes. The primary motif is a stylized orange floral or fan-like shape with white dots. These are paired with small yellow triangles. The elements are arranged in a loose, repeating circular or radial formation across the white background.
In addition to leading to new knowledge about the design of human-computer co-creative systems, this research also establishes a basis for future studies in this area.
2. Related work
2.1. Creative support systems
Creativity support tools, according to Frich et al. (Reference Frich, MacDonald Vermeulen, Remy, Biskjaer and Dalsgaard2019), can be defined as digital systems that encompass one or more creativity-centred features and are used to positively influence one or more phases of the creative process. These tools have evolved in such a way that they have moved from the field of understanding how tools can help creativity in general (Shneiderman Reference Shneiderman2001) to the study of how these tools can be developed in a more specialised way and adapted to specific aspects and phases of the creative process (Davis, Wambsganss et al. Reference Davis, Wambsganss, Jiang, Kim, Käser and Dillenbourg2024). One of these phases is ideation, characterised by divergent and convergent thinking, which are forms of searching in the design space of a specific domain (Frich et al. Reference Frich, MacDonald Vermeulen, Remy, Biskjaer and Dalsgaard2019). However, as the continuous accumulation of existing concepts expands these search spaces, it imposes a growing cognitive burden on human designers. Artificial Intelligence (AI) is increasingly employed to alleviate this limitation and enhance creativity (Sarica & Luo Reference Sarica and Luo2024). Currently, such systems are more effective at supporting the initial divergent ideation, while the convergent phase remains predominantly human-led (Chen et al. Reference Chen, Song, Guo, Sun, Childs and Yin2025).
Supporting human creativity and stimulating new ideas can be done through real-time collaboration with artificial creative systems. We refer to these systems as human-computer co-creative systems, where the human and AI are considered creators during the creative process (Davis Reference Davis2013). This differs from autonomous AI systems, which generate creative artefacts independently, and human creativity support systems (Davis, Deshpande et al. Reference Davis, Deshpande, Rezwana and Magerko2024). These systems, which have emerged in recent years (Jacob & Magerko Reference Jacob and Magerko2015; Kantosalo Reference Kantosalo2019), have applications either for sketching (Davis et al. Reference Davis, Hsiao, Singh, Li, Moningi and Magerko2015; Karimi, Maher et al. Reference Karimi, Maher, Davis and Grace2019; Karimi, Grace et al. Reference Karimi, Grace, Davis, Maher and Gero2019; Ibarrola, Lawton & Grace Reference Ibarrola, Lawton and Grace2024) or in the exploration of other disciplines, such as game design or design ideation (Yannakakis, Liapis & Alexopoulos Reference Yannakakis, Liapis and Alexopoulos2014; Lucas & Martinho Reference Lucas and Martinho2017; Koch et al. Reference Koch, Lucero, Hegemann and Oulasvirta2019).
Many recent examples of human-computer co-creative systems are in the field of drawing. In these systems, the interaction usually starts with a sketch by the user, and the computer partner contributes iteratively, promoting an alternating process until the final result is achieved. These include Drawing Apprentice (Davis et al. Reference Davis, Hsiao, Singh, Li, Moningi and Magerko2015), Collabdraw (Fan, Dinculescu & Ha Reference Fan, Dinculescu and Ha2019), DuetDraw (Oh et al. Reference Oh, Song, Choi, Kim, Lee and Suh2018) and Reframer (Lawton et al. Reference Lawton, Ibarrola, Ventura and Grace2023). There are cases (Ha & Eck Reference Ha and Eck2017; F. Liu et al. Reference Liu, Deng, Lai, Liu, Ma and Wang2019; Ge et al. Reference Ge, Goswami, Zitnick and Parikh2021) in which the computational agent even explores the conceptual value of what is being designed, thereby making the computational contributions more appropriate. However, some examples (Sun et al. Reference Sun, Chen, Xiang, Chen, Gao and Zhang2019; Li et al. Reference Li, Wang, Wang, Greuter and Mueller2020) adopt a task-divided co-creative approach (Kantosalo & Toivonen Reference Kantosalo and Toivonen2016), where humans and computers work on separate tasks during the process.
In the design field, we have the Arny system (Abdellahi, Maher & Siddique Reference Abdellahi, Maher and Siddique2020), may AI (Koch et al. Reference Koch, Lucero, Hegemann and Oulasvirta2019) and CreativeConnect (Choi et al. Reference Choi, Hong, Park, Chung and Kim2024). Arny is based on emotional feedback to give answers to the human collaborator (Abdellahi et al. Reference Abdellahi, Maher and Siddique2020). Besides considering the contributions to the task in question, it uses an interaction method to communicate perceptions and expectations similar to those of human-human collaboration; i.e., through facial expressions, a passive interaction channel is established between the agents. Koch et al. (Reference Koch, Lucero, Hegemann and Oulasvirta2019) designed a system to support the design ideation process, with particular focus on mood board design. It uses a variant of machine learning methods known as cooperative contextual bandits. CreativeConnect (Choi et al. Reference Choi, Hong, Park, Chung and Kim2024) supports graphic designers’ ideation process by helping to extract elements from reference images and suggesting a wide range of recombinations of these elements.
2.2. Influencing factors in the perception of creative systems
As mentioned, user perceptions of creative support systems may change due to the unpredictability inherent in human creativity and the different reasoning humans can adopt during the process (Rezwana & Maher Reference Rezwana and Maher2023). However, other factors can influence users’ perception of these tools.
The computational partner’s role is a fundamental factor in the success of the collaboration (Bown Reference Bown2015), and it can adopt various roles (Kantosalo & Toivonen Reference Kantosalo and Toivonen2016). For this reason, the positive experience of human-AI collaboration does not depend solely on the creative capacity of the artificial partner (Wegner Reference Wegner1997; Louie et al. Reference Louie, Coenen, Huang, Terry and Cai2020). The degree of similarity between the contributions of the user and the AI partner and how this communication is carried out also shapes the users’ perception of the system (Rezwana & Maher Reference Rezwana and Maher2023). In comparison, systems that produce results similar to the user’s contributions can be perceived as more “pleasant” than systems that challenge the user with divergent or “provocative” contributions (Rezwana & Maher Reference Rezwana and Maher2023). Moruzzi (Reference Moruzzi2022) argues that agency and creativity are associated characteristics; if a system is considered creative, it will also be capable of having agentive capacity. However, to fully understand human-AI collaboration, it is crucial to distinguish between the machine’s capabilities and the human’s experience. For the computational partner, this agentive capacity is operationalised as autonomy, the ability to independently choose creative actions and determine when to contribute (Davis, Deshpande et al. Reference Davis, Wambsganss, Jiang, Kim, Käser and Dillenbourg2024). Conversely, for the human user, the focus lies on the Sense of Agency (SoA), defined as the subjective awareness of initiating, executing and controlling actions within the environment (Jeannerod Reference Jeannerod2003; Pagliari, Chambon & Berberian Reference Pagliari, Chambon and Berberian2021; Moruzzi Reference Moruzzi2022). A fundamental component of this experience is feeling in control throughout the creative process (Pagliari et al. Reference Pagliari, Chambon and Berberian2021). Furthermore, as systems transition from mere tools into collaborative partners, user trust emerges as a critical dimension; successful interfaces must go beyond usability to actively cultivate trustworthiness and foster human agency (Carroll et al. Reference Carroll, Slattery, Renaud and Bauerly2026). In this sense, both the interaction and the process defined by the interface developed can limit the user’s possibilities for exploration, influencing their level of creativity and agency (Heer Reference Heer2019; Moruzzi Reference Moruzzi2022).
User characteristics can also be influential in the dynamics of acceptance, perception and response to technology (Davis Reference Davis1989; Venkatesh et al. Reference Venkatesh, Morris, Davis and Davis2003). Studies suggest that people with different levels of design experience may have different expectations, preferences and levels of satisfaction when using AI tools (X. Liu, Maedche & He Reference Liu, Maedche and He2019; Liu & Huang Reference Liu and Huang2025). For instance, in terms of the creative process, while experts tend to have an easier time projecting and anticipating what they can do in a design task, non-experts or less experienced designers tend to have more difficulty in this projection and even in exploring references (Bonnardel & Marmèche Reference Bonnardel and Marmèche2005; Kim et al. Reference Kim, Dontcheva, Li, Bernstein and Steinsapir2015).
The growing accessibility of these tools, alongside advances in computational creativity, has sparked debate about their impact on creative work. While some professionals consider these technologies important in supporting creative practices, others fear they will be replaced. It is therefore essential that the development of these systems goes beyond the technical aspects and considers the dynamics of human-computer interaction and the social dimension of collaboration (Gero, Long & Chilton Reference Gero, Long and Chilton2023; Lim et al. Reference Lim, Leinonen, Lipponen, Kato, Palresha, Thomas and Jaramillo2024).
Given the aspects mentioned above, our study investigates human perceptions of collaboration with AI, considering the level of user expertise. This analysis considers the creative process, how the user perceives the computational partner, its effectiveness as an ideation system, the result and the perception of references. This research aims to answer growing questions regarding AI as a collaborative creator and to explore users’ perceptions of interaction dynamics in co-creative tasks with AI agents. To the best of our knowledge, while research exists on how expertise shapes general tool expectations (Bonnardel & Marmèche Reference Bonnardel and Marmèche2005; Kim et al. Reference Kim, Dontcheva, Li, Bernstein and Steinsapir2015; X. Liu et al. Reference Liu, Maedche and He2019; Liu & Huang Reference Liu and Huang2025), its specific influence on the perception of agency and trustworthiness throughout a complete human-computer co-creative design workflow remains underexplored. This study aims to contribute to this context by providing a comparison between professional and novice mental models in human-computer collaborative design.
3. Methodology
3.1. PatternPursuit: a collaborative ideation system
PatternPursuit is a collaborative ideation system that enables the development of innovative graphic patterns (Rovira Martins et al. Reference Rovira Martins, Cunha, Martins and Boavida2024). PatternPursuit promotes a human-computer co-creative process and takes advantage of pre-digital design work’s conceptual and visual exploration to develop graphic patterns. Using this system as a study environment was particularly pertinent, as it reflects all stages of the actual graphic design process, including studying references, exploring forms and developing a final artefact. This factor allows us to study not only how users with different levels of expertise perceive a collaborative human-computer system, but also which stages of the process are most affected. Such cases are not common in the literature, so PatternPursuit proved to be a favourable environment for exploring the research hypothesis. Besides, graphic patterns are a fundamental element of graphic design, and the reasoning underlying its development process can be applied to other graphic design tasks. The restrictions associated with pattern creation challenge designers to find a balance between visual impact and functionality, understood in this context as the structural viability of a base unit to connect and repeat seamlessly into a continuous surface, a skill present in other graphic design tasks. In addition to the aspects mentioned above, the authors use Sebastião Rodrigues, a Portuguese graphic designer who reinvented and adapted international graphic design movements (Baltazar, Bártolo & Rosa Reference Baltazar, Bártolo and Rosa2021), as a case study. Sebastião Rodrigues was an ethnographic designer (Lisbon, 1929–1997), representing the “portugality” of the time through his own traditionalist modernism. Although the use of third-party works may raise copyright concerns from a commercial perspective, PatternPursuit’s dataset-independent architecture offers a distinct advantage over other generative models (e.g., DALL-E). Unlike models trained on opaque, non-consensual datasets, this system can be implemented using a closed-loop approach with materials from, for instance, a company or a brand’s archives. This allows designers to take advantage of AI-assisted design while maintaining strict compliance with copyright, demonstrating the system’s viability for commercial practice.
The proposed hypotheses fall within the context of human-computer co-creative systems in graphic design, so it was considered appropriate to select a stable, representative process within the discipline that operates with a body of work of recognised value. These aspects offer a well-established theoretical and practical structure, providing resilience to potential disruptions introduced by the study or the system.
This system consists of five modules (see Figure 2), which enable an interactive process that users can monitor, adjust and directly influence. Initially, users interact with a gallery of works by Sebastião Rodrigues. Users can perform a semantic search via text input to explore the works conceptually. The system processes the user-entered concepts using the NLTK library (Bird, Loper & Klein Reference Bird, Loper and Klein2009) and WordNet (Princeton University 2010) to identify semantically related words. The CLIP model (Radford et al. Reference Radford, Kim, Hallacy, Ramesh, Goh, Agarwal, Sastry, Askell, Mishkin, Clark, Krueger and Sutskever2021) is used to associate the resulting words with the works in the gallery. The user is then presented with the Selected Gallery, which contains only works semantically related to the concepts entered. From this subset, users can select the works they find visually interesting to be decomposed into graphical elements (regular and irregular shapes, textures or typography). To obtain them, this module uses image processing and computer vision algorithms, including k-means for colour quantisation (Dhanachandra, Manglem & Chanu Reference Dhanachandra, Manglem and Chanu2015) and the MSER detector (Matas et al. Reference Matas, Chum, Urban and Pajdla2004).
PatternPursuit workflow, showing the approaches used in each module and the methodological basis of each phase (Rovira Martins et al. Reference Rovira Martins, Cunha, Martins and Boavida2024).
Figure 2. Long description
The workflow begins with an Input Concept leading into the first diamond, encompassing Module 1 (Semantic Search), Module 2 (Work’s Selection), and Module 3 (Image Decomposition). The output of this phase becomes the input for the second diamond, which includes Module 4 (Unit Evolution) and Module 5 (Pattern Evolution), culminating in the final Output of wrapping paper.
These decomposed elements, shown in the Decomposed Gallery, are the starting point for developing graphic patterns. Using Interactive Evolutionary Computation (Parmee, Abraham & Machwe Reference Parmee, Abraham and Machwe2008; Eiben & Smith Reference Eiben and Smith2015) the user iteratively develops the unit and then the complete graphic pattern. In both phases, the system acts as a partner and autonomously generates a set of candidate solutions. During the development of the unit, the system automatically generates various compositions using the decomposed elements in order to create the pattern unit. Next, during pattern development, it applies geometric transformations (such as translation, rotation and reflection) to the selected units to create different compositional structures. The user’s role is to evaluate these machine-generated options via a point-and-click interface, selecting the most suitable ones to guide the next generation of the evolutionary process. Users have different functionalities that can help them to develop solutions more in line with what they want, for instance a solution archive or an expanded mode (see Figure 3) (Rovira Martins et al. Reference Rovira Martins, Cunha, Martins and Boavida2024).
PatternPursuit system interface for the Decomposed Gallery and unit development modules.
Figure 3. Long description
The image shows two screens from the PatternPursuit interface. The screen on the left, entitled ‘Decomposed Gallery’, displays a reference artwork on the left and a grid of extracted graphic elements. The screen on the right, relating to the evolution of pattern units, displays a grid of twelve generated ‘individuals’, featuring combinations of shapes. The user has the option to select, save to the archive, expand and export each of these ‘individuals’. Below the main grid, an ‘Archive’ section displays the saved ‘individuals’
3.2. Participants
We invited 30 participants who took part in the study voluntarily. Most participants (23 out of 30) were recruited by the authors of this study via email invitation. The remaining seven participants were recruited through contact with a design studio. The selection criteria were defined in accordance with the study’s need for a group of design experts and a diverse group of people from different areas who did not meet the criteria for the first group. As for the participants’ ages, 19 were between 21 and 30, three were between 31 and 40, five were between 41 and 50, one was between 51 and 60 and two were aged 61 or over. Table 1 summarises each participant’s profession and educational background.
Information about the study participants
Table 1. Long description
A table with 30 rows detailing each participant. The non-experts group (P01-P15) includes professions like Management Controller, Music Teacher, Architect, and Researcher, with education ranging from BSc to MSc, mostly having 5 years or less of design experience. The experts group (P16-P30) includes Graphic Designers, Product Designers, and Lecturers, with education from BSc to PhD, and having between 6 and 30 years of professional design experience.
3.3. Level of expertise
Regarding experience in the design field, the group comprised 15 design experts and 15 non-experts. Each group was defined according to years of professional experience in the design field. The actual coding of design expertise and domain used the following conventions:
-
• Non-expert: individuals with less than 5 years of professional design experience. The 5-year threshold aligns with professional development models, marking the transition to a specialised level characterised by a consolidated mental model and workflows. This group encompasses both junior designers and professionals from unrelated fields (domain novices). Of the 15 non-experts, 11 had previously used design-related tools; of these 11, five had studied or worked in the design field for less than 5 years. It is important to note that the inclusion of participants from different backgrounds in this group was deliberate, as it allowed us to observe how people with different disciplinary perspectives interacted with the system, providing richer insights. Furthermore, this grouping stems from the premise that junior designers, although possessing academic training, are still in the early stages of consolidating their professional practice. Therefore, they share a similar cognitive flexibility and exploratory openness with domain novices when interacting with novel generative systems. This shared behaviour also occurs because, unlike experts who are accustomed to the authority and decision-making burden of authorship (Rock Reference Rock1996), both junior designers and novices often experience a sense of relief when the AI assumes this responsibility.
-
• Expert: people with more than 5 years of professional experience in the field of design. Of the 15 experts, seven had between 6 and 10 years of experience, three between 11 and 20 years and five between 21 and 30 years. Beyond the temporal threshold, these participants were selected based on their active professional practice in design studios or academia, ensuring they possessed a consolidated design process and critical vocabulary typical of domain experts.
3.4. Protocol
The study used a mixed-methods protocol, consisting of a creative task followed by a structured quantitative questionnaire and a semi-structured qualitative discussion (see Figure 4). The interviews were conducted in person and recorded on audio and video; all interviewees gave their consent to participate in the study and verbal and written authorisation for audio and video recording during the interview. The questionnaire items were displayed on the screen to the participant. The discussion at the end gave participants the opportunity to freely elaborate on their experiences and challenges. Participants did not receive any compensation for participating in the study, but were highly motivated by the topic.
Structure of the tests conducted.
Figure 4. Long description
The flowchart is divided into two main sections: First Part and Second Part.
First Part consists of two main blocks:
1. A vertical column on the far left labeled 3 min, containing the text Introduction plus Personal Questions. An arrow points from this column to the next block.
2. A large central block labeled 30 min, titled Creative Task. This block is subdivided into three color-coded horizontal layers representing a digital interface:
* Top layer (blue): Semantic Exploration, showing a grid of book covers or posters.
* Middle layer (yellow): Visual Exploration, showing abstract shapes and silhouettes.
* Bottom layer (green): Creation Phase, showing a workspace with small green icons and patterns.
An arrow points from the Creative Task block to the Second Part.
Second Part is a single large block labeled 12 min, titled Survey. It lists five sequential evaluation steps:
* Evaluating the Collaborative Process (22 Questions).
* Creative Support Index (C S I) (12 plus 15 Questions).
* Evaluating the Final Result (8 Questions).
* Discussion.
* Survey (at the bottom base).
3.5. Data collection and instruments
The first part involved initial personal questions and a creative task. Participants were guided by a design brief: they were tasked with creating graphic patterns for gift wrapping paper. Although providing this concrete goal was essential to give users a starting point and prevent the paralysis of not knowing what to create, we deliberately kept the brief relatively open, without highly constrained requirements like a specific context. This trade-off aimed to encourage free exploration and allow us to observe how users naturally navigate the human-computer co-creative process. Within this brief, users could interact freely with the system until they developed one or more patterns that satisfied them (Rovira Martins et al. Reference Rovira Martins, Cunha, Martins and Boavida2024).
The second part consisted of a structured quantitative questionnaire followed by a semi-structured qualitative discussion (see Supplementary Material). This part was subdivided into four different topics: (i) evaluating the collaborative process; (ii) Creative Support Index (CSI); (iii) evaluating the final result and (iv) general discussion.
Regarding the evaluation of the collaboration process, this assessed topics including Trustworthiness (“I felt the system was trustworthy”), Human Control (“I felt I had control over the process”), Freedom of Choice (“The system offered me freedom of choice”), System as an Idea Facilitator (“The system was effective as an idea facilitator”), Usefulness in Exploring Ideas (“The system was useful in exploring new ideas”) and Perception of how users’ skills and prior knowledge could have influenced the process. These items were measured on the defined 5-point Likert scale. Given that the main objective of this exploratory research was to study user perception within a broad context, encompassing the analysis of the artificial partner, the collaborative process and the creative impact, we opted to use single-item Likert scales to measure global perceptions such as trust and control. This methodological choice aimed to prevent participant fatigue, a practical constraint where single-item measures offer a viable alternative for overarching global constructs (Wanous, Reichers & Hudy Reference Wanous, Reichers and Hudy1997; Fisher, Matthews & Gibbons Reference Fisher, Matthews and Gibbons2016). While this precludes a granular analysis of specific sub-dimensions, it provides a general overview of user perceptions that is aligned with the study’s exploratory objectives. Other measuring factors, such as autonomy, contributions, the role the system played during the process and the quantity of ideas explored, were assessed using a multiple-choice answer format.
To evaluate the system’s level of autonomy in terms of action and creative choices, we used categorical choice instrument based on the characteristics of autonomy of the five-pillar framework proposed by Davis, Deshpande et al. (Reference Davis, Wambsganss, Jiang, Kim, Käser and Dillenbourg2024): (i) Fully autonomous – the computer takes the initiative to act and chooses the creative action; (ii) Semi-autonomous – the computer does not take the initiative to act, but chooses its creative action and (iii) User directed – the user guides the computer and it behaves in a defined way.
Regarding the system’s contributions, users had the following options (Davis, Deshpande et al. Reference Davis, Wambsganss, Jiang, Kim, Käser and Dillenbourg2024): (i) Fully emergent – produces contributions that are highly variable, spontaneous and unpredictable and cannot be deduced; (ii) Partially emergent – produces contributions that are unexpected but deducible considering the initial conditions and the set of rules defined and (iii) Non-emergent – produces contributions that are predictable and deducible. We chose this structure because it allows us to assess the system’s alignment with the user’s creative expectations and its potential to promote unexpected solutions. The system’s level of autonomy and the nature of its contributions were fixed, meaning the system behaved consistently across all participants. Therefore, the questions on these topics assessed the users’ inferred perceptions of the system’s behaviour, revealing how their level of expertise shaped their mental model of the AI’s capacity for action.
When asked about the predominant role the system played during the creative process, we used the options established by Maher (Reference Maher2012) (Q13): (i) Enhance – the system provides knowledge and incentives to increase the user’s creative capacity; (ii) Support – the computer provides techniques and support tools that can help human creativity and (iii) Generator – the computer generates creative ideas that the human being interprets, evaluates and can integrate into the final product.
In the second subsection, participants answered a CSI questionnaire proposed by Cherry & Latulipe (Reference Cherry and Latulipe2014), consisting of 12 questions to assess the tool’s support for creativity, measuring the six dimensions of creativity: Collaboration, Enjoyment, Exploration, Expressiveness, Immersion and Results worth the effort. Also related to the CSI, participants directly compared two factors to understand which factors were most frequently mentioned by the user.
Finally, the participants evaluated the final results regarding value, novelty and quality to understand the level of creativity of the result, as proposed by Boden (Reference Boden2004). To ensure conceptual clarity within this specific graphic design task, these three parameters were defined to the participants. This provided a standardised evaluation baseline across users, as the task, while having a specific objective, was partially open-ended in terms of context. “Novelty” referred to the visual and conceptual originality of the generated pattern, assessing its capacity to produce unexpected results that depart from typical wrapping paper conventions. “Value” measured the professional utility and practical relevance of the pattern for its intended real-world application. “Quality” was evaluated based on the aesthetic success and visual cohesiveness of the overall composition. We specifically looked for a unified whole in the Gestalt sense, where the final design feels like a complete piece rather than a mere aggregation of separate parts. Participants also evaluated the level of recognition of the references, and the “key elements” they considered for recognition. In addition, the final segment was a semi-structured qualitative discussion. Here, participants were encouraged to describe what they enjoyed most and the challenges encountered, and to suggest improvements. The open-ended nature of this discussion allowed the interviewer to pursue interesting emergent themes for deeper exploration.
For data analysis, given the sample size (
$ N=30 $
) and the ordinal nature of the Likert-scale data, we used non-parametric tests. The Mann–Whitney U test was employed to identify significant differences between the two independent groups (experts versus non-experts). Furthermore, to understand the relationships between variables (e.g., how control influences perceived value), we conducted Spearman’s rank correlation coefficient analyses for each group separately. Statistical significance was defined at
$ p<0.05 $
.
4. Results
4.1. Reflecting on the user’s collaborative experience (RQ1)
This subsection focuses on the global collaboration experience and the dynamics between the parties during the creative process. Regarding the system’s workflow, both groups considered that the phases of the system helped the creative process (Q14), with a relatively higher mean for non-experts (mean = 4.33 ± 0.87, mode = median = 5.00) compared to experts (mean = 4.00 ± 0.82, mode = median = 4.00), though this difference was not statistically significant (
$ p=0.201 $
). Similarly, regarding the influence of prior knowledge (Q19), no significant difference was found (
$ p=0.982 $
), with both groups reporting high confidence levels (
$ mean=4.00\pm 0.82 $
for experts;
$ mean=4.00\pm 0.89 $
for non-experts).
As observed in Table 2, the level of trustworthiness in using the system (Q8) was rated positively by both groups. This indicates that, regardless of the level of experience, the system’s interface helps to support user confidence, although non-experts emphasise it slightly more.
User perception of the computational partner considering expertise. Bold values indicate statistical significance (p ≤ 0.05)
Table 2. Long description
This table details the user perception of the computational partner, comparing 15 non-experts and 15 experts. The parameters were evaluated on a 5-point Likert scale, presenting the Mean, Standard Deviation (SD), Mode, Median (Med.), and p-value for each question. Data for Non-experts vs Experts: Q8—Trustworthiness (Mean 3.87 vs 3.53, both Mode/Med 4, p=0.397); Q9—Human Control (Mean 3.13 vs 2.53, both Mode/Med 3, p=0.050); Q12—Freedom of choice (Mean 3.93, Mode 5, Med 4 vs Mean 3.00, Mode 4, Med 3, p=0.026); Q15—System as an idea facilitator (Mean 4.33 vs Mean 3.87, both Mode/Med 4, p=0.042); Q28—Usefulness in exploring ideas (Mean 4.40, Mode 5, Med 4 vs Mean 3.80, Mode/Med 4, p=0.068). Bold values indicate statistical significance (p ≤ 0.05).
However, significant differences emerged regarding agency. Non-experts reported feeling more human control (Q9,
$ p=0.050 $
) and freedom of choice (Q12,
$ p=0.026 $
) than experts. Consistent with this, non-experts felt that the system played more of a facilitating role in generating ideas (Q15,
$ p=0.042 $
) and was marginally more useful in exploring them (Q28,
$ p=0.068 $
) than experts.
Behavioural data support these quantitative findings. Non-experts adopted a divergent exploration strategy, with the majority exploring “4 or 5” or “6 or more” ideas before deciding. In contrast, experts adopted a more convergent approach, exploring fewer distinct ideas (“2 or 3”), but reporting a higher frequency of returning to previous steps to modify things. This behaviour demonstrates a desire for refinement and specific control that the system’s semi-autonomous nature may have constrained.
4.2. Reflecting on the user’s perception of the computational partner (RQ2)
Considering the user’s perception of autonomy, 80% of experts (12 out of 15) categorised the system as Semi-Autonomous, demonstrating a higher consensus. In contrast, non-experts were more divided, with the majority (60%, 9) perceiving the system as User-Directed, while 40% (6) saw it as Semi-Autonomous. This suggests that non-experts, relying on the system to bridge skill gaps, perceived the tool’s utility as an extension of their own agency.
Regarding the system’s role, 66.7% of experts (10) viewed the system as an Enhancer. Non-experts were more divided between Enhancer (46.7%, 7) and Generator (40%, 6), indicating that a significant portion of non-experts attributed the core creative act to the creative artificial partner.
Furthermore, we also analysed perceptions of the system’s contributions. While 100% of the experts (15) rated the contributions as “Partially emergent”, that is unexpected but deducible from a set of rules, 33.3% (5) of the non-experts perceived them as “Fully emergent” and 66.7% (10) as “Partially emergent”. This indicates that while experts could identify the underlying logic of the generation, a third of the non-experts perceived the output as non-deducible novelty.
4.3. Reflecting on the level of creativity (RQ3)
When calculating the Creative Support Index for expert and non-expert groups, it was found that the non-experts had a higher score (83.28) than the experts (71.22), indicating that the system was far more effective at supporting the creativity of novices (
$ p=0.011 $
).
As shown in Table 3, non-experts considered the system particularly effective in Results Worth the Effort (
$ 9.40\pm 1.0 $
) and Enjoyment (
$ 8.47\pm 1.7 $
), suggesting that they felt rewarded during creative exploration. In contrast, experts gave the lowest ratings to Expressiveness (
$ 6.33\pm 1.8 $
), highlighting that they felt the system did not allow them to be as expressive as they would have liked.
Creativity Support Index results considering user expertise
Table 3. Long description
This table displays the results of the Creativity Support Index (CSI) questionnaire, comparing the average factor counts and average factor scores between expert and non-expert groups across six dimensions. CSI average scores for Experts vs Non-experts: Collaboration (7.00 vs 6.46); Enjoyment (7.40 vs 8.47); Exploration (7.13 vs 8.33); Expressiveness (6.33 vs 8.13); Immersion (6.87 vs 7.67); Results Worth Effort (7.93 vs 9.40). The Total CSI Score highlights a difference in the overall creative support, with Experts scoring 71.22 and Non-experts scoring 83.28.
Regarding the final pattern, non-experts considered it to have significantly higher quality (
$ p=0.005 $
) and value (
$ p=0.046 $
) than the experts (see Table 4). Concerning novelty, although non-experts rated it highly, no significant difference was found compared to experts (
$ p=0.086 $
).
Evaluation of the final result considering user expertise. Bold values indicate statistical significance (p ≤ 0.05)
Table 4. Long description
This table presents the evaluation of the final design patterns, comparing non-experts and experts across parameters related to creativity and reference recognition. The data presents the Mean, Standard Deviation (SD), Mode, Median (Med.), and p-values. Final result evaluation (Non-experts vs Experts): Q51—Quality (Mean 4.67 vs 3.80, p=0.005); Q52—Value (Mean 4.67 vs 4.07, p=0.046); Q53—Novelty (Mean 4.67 vs 4.13, p=0.086); Q54—Memory Recognition (Mean 2.67 vs 2.73, p=0.832); Q55—Side by side Recognition (Mean 3.73 vs 3.27, p=0.285); Q56—Balance between novelty and authorship (Mean 3.33 vs 3.53, p=0.726). Bold values indicate statistical significance (p ≤ 0.05).
Finally, the analysis of the balance between novelty and authorship (Q56) reveals a subtle but important distinction in perception. Experts perceived the final patterns as slightly more balanced (
$ mean=3.53\pm 0.81 $
) than non-experts (
$ mean=3.33\pm 1.07,p=0.726 $
). This result is revealing when contrasted with recognition (Q54). Although experts struggled to explicitly identify Sebastião Rodrigues’ works in the final output (
$ mean=2.73\pm 1.00 $
), their higher rating for balance suggests they value the overall structural equilibrium between the new composition and the source material, rather than merely looking for literal visual citations. In contrast, non-experts, lacking this depth of design nuance, may have struggled to perceive this abstract balance, focusing instead on obvious visual references, which led to a lower perception of equilibrium despite their higher overall satisfaction.
5. Discussion and research opportunities
The results demonstrate how users’ level of expertise significantly shapes their experience and interpretation of collaboration with AI-assisted design tools. This validates the need for adaptable interfaces (Shneiderman Reference Shneiderman2002) and underscores how user groups conceptualise the same system differently.
5.1. RQ1: influence on the collaborative experience and process
The findings show that the level of design expertise dictates distinct creative strategies, leading to fundamental differences in the collaborative process. Non-experts favoured a divergent exploration approach, exploring more ideas, appreciating the system’s ability to generate and extend ideas beyond their competence. This approach is quantified by their perception of the system being highly useful in exploring ideas (Q28,
$ mean=4.40 $
). For this group, the system acted as a capability extender, with one participant noting, “I consider myself an uncreative person, and the tool exceeded all my expectations” (P08), illustrating how the tool successfully bridges the gap between low technical skill and high creative output.
Experts, in contrast, adopted a convergent refinement process, revisiting previous iterations more often to refine the results, expecting the system’s contributions to be more in line with their vision. When asked specifically if their prior knowledge made them feel inhibited or confident (Q19), experts reported high confidence (
$ mean=4.00 $
), matching non-experts. This indicates that their expertise did not act as an internal psychological barrier or a source of inhibition. However, this internal confidence stands in contrast to their significantly lower perception of freedom of choice (Q12,
$ p=0.026 $
). This suggests that while their expertise made them feel capable of defining a creative direction, the system’s semi-autonomous nature restricted their ability to execute that specific vision. Unlike non-experts, who felt supported by the tool, experts felt constrained because the system could not accommodate the type of control over the process that they were accustomed to in their professional activity. This confirms a greater need for negotiation phases and explainable system behaviour for experts to feel confident in incorporating contributions.
5.2. RQ2: influence on perception of the computational partner
One of the most significant divergences concerns the conceptualisation of the system’s role: non-experts (60%) categorised the system as User-directed, while experts (80%) conceptualised it as Semi-Autonomous. This dichotomy is fundamental to understanding the dynamics of collaboration. The non-experts perceived the AI as an empowering tool. This User-directed view aligns with their perception of balanced control (Q9,
$ mean=3.13 $
), which stands in significant contrast to the experts’ perception of system control (
$ mean=2.53 $
) and their feeling that the system helps them overcome skill limitations to produce high-quality work (Q51,
$ mean=4.67 $
). Their focus is on the structured guidance provided by the tool.
Conversely, experts conceptualised the AI as a creative partner that challenges their objectives. Their Semi-Autonomous categorisation suggests they expect the AI to have its own creative agency, prompting them to find a “middle ground” between the computer’s idea and their own expectations, mirroring human-human collaboration dynamics. This need for alignment confirms a strong requirement for negotiation phases and explainable behaviour for experts to feel confident in incorporating the AI’s contributions. As one expert illustrated, the lack of this negotiation led to a sense of restriction: “I felt that it controlled many of the images it gave me, so I felt I was a bit limited” (P21). This vividly highlights the friction experts experience when a semi-autonomous system does not offer the precise control required to align the AI’s contributions with their professional intent.
5.3. RQ3: influence on the level of creativity and final result
The collaborative process resulted in a significantly lower overall CSI score for experts (71.22) than for non-experts (83.28). This divergence confirms that the structured guidance, while successfully accelerating the creative capacities of non-experts, led to reduced creative satisfaction for experts. For the latter group, the difficulty in aligning the AI’s output with their established domain knowledge contributed to a lower assessment of the system’s effectiveness in key areas they valued, such as Exploration (
$ mean=7.13 $
vs.
$ mean=8.33 $
) and Expressiveness (
$ mean=6.33 $
vs.
$ mean=8.13 $
).
Regarding the final result, non-experts rated the patterns significantly higher in Quality (
$ p=0.005 $
) and Value (
$ p=0.046 $
). This disparity is clarified by our correlation analysis. For non-experts, a positive correlation was found between perceived Quality and Novelty (
$ r=0.55,p=0.032 $
). This suggests that non-experts conflate the originality of the generation with the quality of its design, maximising their satisfaction with the final result.
On the other hand, although experts also rated Novelty as moderately high (
$ mean=4.13 $
), their perception of Value is correlated with Human Control (
$ r=0.48 $
). This indicates that, for experts, the professional usefulness of the artefact is linked to their SoA, that is without control, they do not value the result, even if they recognise it as innovative. Unlike non-experts, who were satisfied with the surprise, experts distinguished between the novelty of the generation and its actual usefulness. While experts acknowledged that the system “presents options that I had not seen before through the selection of different parts and alternative combinations. It reverses the logic of visualisation/interpretation” (P17), this intellectual recognition of novelty did not translate into professional value without control. In contrast, non-experts embraced the lack of control as a creative feature, with one stating that although they had a definite initial idea, the unexpected results “opened up my horizons” (P02), confirming the strong correlation found between Novelty and Quality for this group.
5.4. Implications and research opportunities
This divergence in findings highlights a key tension in co-creative systems: the same functionalities may be perceived as empowering or intrusive, depending on the user’s expertise and expectations.
Although these conclusions are drawn from interactions with PatternPursuit, they offer relevant insights for the broader discussion on human-AI collaboration. We acknowledge that PatternPursuit operates fundamentally differently from widely used generative AI tools (such as text-to-image diffusion models), as it provides structured control over the recombination of existing visual materials through an iterative, step-by-step collaborative loop, rather than generating entirely new visual elements from scratch through one-shot prompting. This fundamental distinction in the creative workflow makes direct comparisons with other generative systems difficult. However, these results demonstrate that a user’s domain expertise is a critical factor that influences their perception of control and autonomy. Ultimately, this study highlights that the level of expertise can lead users to perceive the same system differently regarding its role, its contributions, the creative process and the evaluation of the final result.
These findings contribute to the development of more effective co-creative systems by directly informing a framework of research opportunities:
-
• Adaptable idea exploration: Systems should support both divergent (idea generation) and convergent (idea refinement) thinking styles to allow users with different levels of expertise to follow the process they need.
-
• Negotiation phase: Systems must provide users with a way of negotiating with the computational partner after their contribution, through explainability techniques.
-
• Customisable Autonomy: Systems should contain different levels of autonomy, allowing users to either use the system entirely autonomously or with slight inclusion of the computational partner, in a co-creative manner, with balanced agency between the parties, or with the computer having greater control over the process.
By implementing this framework, future co-creative systems can better respond to their users’ needs. Such systems could promote more successful interactions, where experts feel their knowledge is reinforced rather than jeopardised, and non-experts feel empowered to create without barriers.
6. Limitations and future work
This study presents preliminary insights into how user experience shapes collaboration between humans and AI in graphic design. However, the relatively small sample size and interview-based context limited the observation of behavioural nuances that might arise with prolonged or real-world use. Larger-scale studies with diverse user groups would help validate the findings and their generalisation.
Regarding data collection and evaluation, our methodology prioritised capturing users’ subjective experiences and high-level strategies through qualitative, self-reported data. This approach extended to the evaluation of creativity, which relied on participants’ self-assessment. While this may introduce potential bias, it was a deliberate choice, since computational creativity relies heavily on subjective human judgement (Jordanous Reference Jordanous2012; Lamb et al. Reference Lamb, Brown and Clarke2019), and capturing this personal validation was essential to understanding users’ perception of autonomy and trust. Nevertheless, while this qualitative focus effectively addressed our research questions, future studies could expand on these insights by using multi-item scales and incorporating low-level system telemetry. Logging quantitative behavioural data, such as the number of semantic searches, evolutionary iterations or time spent per module, would offer a valuable opportunity to triangulate these subjective reports with objective interaction metrics.
As noted in the previous section, the end-to-end nature of PatternPursuit, encompassing everything from the initial exploration of references to the creation of the final artefact, adds a layer of complexity to direct comparisons with mainstream AI generators. Furthermore, comparison with other tools, such as Illustrator, would introduce significant evaluation biases, as results for non-experts would reflect their lack of proficiency in the software rather than their collaborative experience. Consequently, we chose to focus on evaluating how user expertise shapes the creative process and perceptions within a specific workflow. Observing the strategies of experts and novices in a uniform environment allows us to assess the collaborative differences influenced by expertise (Bonnardel & Marmèche Reference Bonnardel and Marmèche2005; Ozkan & Dogan Reference Ozkan and Dogan2013).
Future work should explore how these findings inform system design that adapts to different levels of experience. Investigating the impact of users’ prior experience with AI and drawing on models of human creative collaboration can also foster the development of more effective and adaptable co-creative systems.
7. Conclusion
The increase in the daily use of human-AI creative tools in creative tasks has fuelled the need to make creative systems more efficient. Even with the vast contributions to computational creativity, the unpredictable nature of human creativity means that the strategies and reasoning they use, the level of control in the process and the final result can vary depending on their goal and experience. These aspects contribute to the fact that systems such as human-computer co-creative still present several challenges (Rezwana Reference Rezwana2023). In this sense, it is important to understand how the same system can be perceived depending on the user’s background, initial expectations and critical thinking. The study revealed insights into how expertise level influences users’ perceptions and interactions with collaborative design systems.
We used PatternPursuit as a study environment because it provides users with a range of references, allows conceptual and visual exploration, serves as an ideation tool and facilitates the elaboration of a final result – graphic patterns for wrapping paper. Given these characteristics, we consider PatternPursuit a complete and viable system for exploring the hypothesis posed.
The results of this study highlight the differences in collaboration and perception between groups with different levels of knowledge. By comparing the experiences of non-experts and experts using PatternPursuit, we observe trends in how knowledge impacts the various aspects of the creative process. While non-experts felt empowered by the tool, experts felt constrained by the system’s semi-autonomous nature. Our analysis reveals that, for experts, the perceived value of the final artefact is linked to their SoA, i.e., if they are unable to guide the process, they do not validate the result as useful, regardless of its novelty. This reinforces the conclusion that, for adoption by experts, co-creative systems must go beyond generating surprise and offer the necessary control to align AI contributions with the user’s professional intent. These conclusions can be used to develop more effective co-creative systems that consider the target audience.
Supplementary material
The supplementary material for this article can be found at http://doi.org/10.1017/dsj.2026.10078.
Financial support
This work is funded by national funds through FCT – Foundation for Science and Technology, I.P., within the scope of the research unit UID/00326 – Centre for Informatics and Systems of the University of Coimbra, https://doi.org/10.54499/UID/00326/2025. Joana Rovira Martins was supported by FCT – Foundation for Science and Technology, I.P., through the PhD scholarship with reference 2025.00925.BD and DOI identifier https://doi.org/10.54499/2025.00925.BD.
Declaration of generative AI
The authors acknowledge the use of AI-assisted language tools, including Grammarly (Pro version, accessed January 2026 and May 2026) and Gemini (Pro, accessed January 2026 and May 2026) models, for supporting the proofreading and grammatical refinement of this manuscript. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the content of the published article.
Competing interests
The authors report there are no competing interests to declare.
Open access
