Introduction
Interior design, including the selection of furniture objects, their layout and materials, is a challenging task that requires professional designers (Kan and Kaufmann, Reference Kan and Kaufmann2017). Furniture placement is challenging because it requires jointly optimizing a variety of functional and visual criteria. Experienced designers learn to balance the tradeoffs between the guidelines through an iterative trial-and-error process (Merrell et al., Reference Merrell, Schkufza, Merrell, Schkufza, Li, Agrawala and Koltun2011). With the growing popularity of large-scale virtual environments for architectural visualization and the game industry, the manual interior design of virtual scenes becomes prohibitively expensive in terms of time and resources (Kan and Kaufmann, Reference Kan and Kaufmann2017). Manual authoring of environments at these scales is impractical and there is a need for algorithms that can automatically generate realistic virtual environments (Henderson et al., Reference Henderson, Subr and Ferrari2019). Automatic and semi-automatic indoor scene synthesis is beneficial for interior designer, typically from faster, easier-to-use tools, employed as they consult with customers, to generate suggested layouts (Zhang et al., Reference Zhang, Zhang, Liang and Hall2019). Various approaches for automatization have been proposed, including constraint satisfaction, optimization of hand-crafted interior design principles, statistical priors on pairwise relationships between objects, and human-centric relationship priors (Wang et al., Reference Wang, Savva, Chang and Ritchie2018). However, layout design with complex constraints is still a challenging problem to solve due to the non-uniqueness of the solution and the difficulties in incorporating the constraints into the conventional optimization-based methods (Zhang and Ye, Reference Zhang and Ye2019). Modern approaches to solving many scene analysis and modeling problems have been data-driven, resorting to machine learning (Li et al., Reference Li, Patil, Xu, Chaudhuri, Khan, Shamir, Tu, Chen, Cohen-Or and Zhang2019b).
With data-driven and learning-based approaches receiving more and more attention in recent years, there has been a steady accumulation of indoor scene datasets (Fu et al., Reference Fu, Cai, Gao, Zhang, Wang, Li, Xun, Sun, Jia, Zhao and Zhang2021). Despite this interest, no one to the best of our knowledge has made a dataset with input features from real-life kitchen design considerations. As machine learning becomes more widely used, it becomes more important to acquire data and label data, especially for state-of-the-art neural networks (Roh et al., Reference Roh, Heo and Wang2021).
Kitchen represents a space with great importance for one family. That is why the goal is to have a kitchen with the best possible organization, maximal space usage, and aesthetic appearance in accordance with the owner's taste. Many interior design guidelines considered kitchen as one of the most complex rooms in the entire house (Lane, Reference Lane1963; Powell and Svendsen, Reference Powell and Svendsen2011). Kitchens require work with a custom furniture that is made specifically for a given room but satisfies predefined design rules (Wilson, Reference Wilson1947). For example, the length of working surfaces is a continual value that should satisfy certain constraints, and some kitchen elements (sink, oven, etc.) have discrete length value. The building blocks of the kitchen should be carefully coordinated to satisfy complex functional and aesthetic constraints (e.g., elements should appear in a row without any space between them (Wang et al., Reference Wang, Savva, Chang and Ritchie2018), the fridge should be at the beginning of the working sequence (Powell, Reference Powell2005). Kitchen planning involves positions of plumbing, electrical, heating, ventilation, and sometimes gas lines (Hepler et al., Reference Hepler, Wallach and Hepler2012; House, Reference House2020). That is why a kitchen designer has to be both analytical and creative (Made, Reference Made2019). In some cases, there is not one correct kitchen layout, and choosing acceptable one relies purely on designer's creativity. Due to the kitchen design complexity, existing research in the area of procedural indoor scene synthesis recognizes the kitchen as a specific functional group (Wang et al., Reference Wang, Savva, Chang and Ritchie2018, Reference Wang, Lin, Weissmann, Savva, Chang and Ritchie2019; Henderson et al., Reference Henderson, Subr and Ferrari2019) or does not work on the kitchen layout generation at all (Zhang et al., Reference Zhang, Yang, Ma, Luo, Huth, Vouga and Huang2018; Ritchie et al., Reference Ritchie, Wang and Lin2019).
In order to avoid enumerating all high-level constraints, we develop a method that will learn kitchen design rules from a dataset of linear kitchen layouts. The proposed method is a pipeline of six machine learning classifiers that automatically populate one-wall empty space with 3D models of kitchen elements. Machine learning models are trained and tested on a dataset created and presented in this work. Dataset consists of 1,000 linear kitchen layouts with annotated features from real-life kitchen design considerations such as positions of electricity and ventilation. We believe that this dataset is a valuable source for future data-enabled approaches for kitchen layout design with input parameters encountered in a common designer's practice. Quantitative and perceptual studies suggest that the method generates sensible kitchen designs, for particular scenes close to the one generated by professionals. Moreover, perceptual study participants noted that the tool can be used to speed up designer's work and to improve communication with clients. Students of interior design see potential for use of PKG model as a learning tool. Our code and data are available at https://github.com/jelena6k/ProceduralKitchenGeneration.
Kitchen layout
Kitchens can be classified into six basic types according to the layout shape (Fig. 1): Linear (I-Shape), II-Shape, L-Shape, U-Shape, G-Shape, and With island (Pejic et al., Reference Pejic, Jovanovic, Marinkovic and Krasic2019a).
Six basic kitchen types.
The main principle of kitchen layout design is that the elements should be organized to ensure a safe and comfortable cooking environment. Therefore, the “Working triangle” (WT) principle was developed in the 1940s by the University of Illinois School of Architecture. This principle defines rules for the mutual position of the three major work stations in a kitchen: Cooking (primarily the stove), Storage (primarily the refrigerator), and Washing (primarily the sink), separated with work surface (Ranne, Reference Ranne1950; Wormer, Reference Wormer2003).
These principles are possible to implement in the case of new building design and development. Implementation in the case of remodeling will be feasible only if position of existing: ventilation and sink drain ducts, window, etc., are aligned with WT principle. If not, changing position of windows and duct system to align with WT principle can be costly and, in most cases, impossible. In this case, designer's creativity has a main role in layout design in order to ensure safe and comfortable kitchen environment. Linear or one-wall kitchen have all elements positioned only on one straight wall. Ideal linear kitchen element disposition (starting from the entry door) is: Storage–Washing–Cooking, with Work surface between them (Fig. 2).
Ideal linear kitchen layout organization.
Research overview
The aim of our work is to develop a system that will support the kitchen design process. In this paper, we propose a novel PKG model (Fig. 3), the pipeline of ML classifiers for the automatic linear kitchen layout generation. Generated layouts are visualized in Unity Game engine using predefined kitchen styles.
Overview of the PKG model pipeline.
Input features to PKG model are kitchen length, ventilation position, sink drain position, window position, kitchen entry door position, and exit door position. This paper focuses on linear kitchens that contain three major elements: “Washing” – one sink, “Cooking” – one stove, and “Storage” – one refrigerator, while the space between them is filled with “Work surface”. PKG model is a pipeline of six machine learning classifiers (Fig. 3), motivated by a real-life interior design process (Wormer, Reference Wormer2003):
1. Washing subtype classifier
2. Washing position classifier
3. Cooking subtype classifier
4. Cooking position classifier
5. Storage subtype classifier
6. Storage position classifier.
To train and test ML classifiers, a group of 5 interior designers create a dataset of 900 train and 100 test examples of linear kitchen layouts. Dataset examples contain input features and target fields (Fig. 4). For each example, one acceptable layout is given in the form of six discrete target fields: washing subtype, washing position, cooking subtype, cooking position, storage subtype, and storage position. It is presumed that the kitchen layout can be completely described with these six target fields (Fig. 4) while space between these elements is filled with work surfaces. PKG model was trained to predict the same layout as in an example, for given input features.
Dataset input features and target fields.
For each step of PKG pipeline (Fig. 3), we tested the accuracy of the following classifiers: Random forest, Decision tree, AdaBoost, Naive Bayes, MLP, SVM, and L2 Logistic regression. Results show that random forest outperforms other classifiers in all prediction tasks.
As the best performing classifier, random forest is used in each step of PKG pipeline and integrated into Unity Game engine development environment as a module for automatic 3D kitchen generation and presentation. Providing possibility for easy implementation in video games or virtual reality applications. A visual representation of the PKG model inside Unity is used for perceptual studies performed by experts. The module is used for the generation of 100 kitchen layouts for the test set. In 77.4% cases, PKG generated layouts are marked as “acceptable” by interior designers. That is 8.2% better than the prior rule-based method performance.
Contributions
The main contributions of this research are:
• Kitchen layout Dataset, created by experts, consists of different linear kitchen layouts;
• A novel machine-learning-based approach for linear kitchen layout generation;
• Comparison of ML classifiers for kitchen layout prediction task. Determination and analysis of best performing algorithm – Random forest.
Related work
Regarding the construction of virtual buildings, there exists an overwhelming amount of research papers (Freiknecht and Effelsberg, Reference Freiknecht and Effelsberg2017). One group of research addresses generation of buildings (Brenner, Reference Brenner2000; Birch et al., Reference Birch, Browne, Jennings, Day and Arnold2001; Saldana and Johanson, Reference Saldana and Johanson2013; Rodrigues et al., Reference Rodrigues, Neto and Vidal2015) and another automatic indoor space generation (Katz, Reference Katz, Kocatürk and Medjdoub2011; Wang et al., Reference Wang, Savva, Chang and Ritchie2018; Pejic et al., Reference Pejic, Mikic and Milovanovic2019b).
There is several research addressing the problem of automatic layout disposition using different principles. The first group of authors (Katz, Reference Katz, Kocatürk and Medjdoub2011; Schultz et al., Reference Schultz, Bhatt and Bormann2017; Stojakovic et al., Reference Stojakovic, Budak, Obradovic, Korolija-Crkvenjakov and Santosi2017; Pejic et al., Reference Pejic, Jovanovic, Marinkovic and Krasic2019a, Reference Pejic, Mikic and Milovanovic2019b) uses rule-based parametric principles. Second group uses optimization-based methods (Kan and Kaufmann, Reference Kan and Kaufmann2017, Reference Kan and Kaufmann2018). In order to form the cost function in their optimization, they used mathematical expressions representing the aesthetic, ergonomic, and functional rules used in professional interior design. The third group of researchers is oriented toward example-based methods which utilize the information from existing layouts (Henderson et al., Reference Henderson, Subr and Ferrari2019). These data-driven methods avoid enumerating complex rules which are highly dependent on the knowledge given by the individual designers. As machine learning is becoming a more widely used data-driven approach, we are seeing new applications of it in automatic furniture layout generation (Daniel et al., Reference Daniel, Wang and Lin2018.; Wang et al., Reference Wang, Savva, Chang and Ritchie2018; Chaillou, Reference Chaillou2019).
On the other hand, there are just a few paper in the literature dealing with automatic kitchen layout generation (Wang et al., Reference Wang, Savva, Chang and Ritchie2018, Reference Wang, Lin, Weissmann, Savva, Chang and Ritchie2019; Pejic et al., Reference Pejic, Jovanovic, Marinkovic and Krasic2019a, Reference Pejic, Mikic and Milovanovic2019b; Li et al., Reference Li, Patil, Xu, Chaudhuri, Khan, Shamir, Tu, Chen, Cohen-Or and Zhang2019b), providing solutions with certain limitations. Deep convolutional networks struggle with generating contiguous placement of separate kitchen countertop sections (Wang et al., Reference Wang, Savva, Chang and Ritchie2018). Convolutional operations are oblivious to the underlying structures in the data, which often play an essential role in scene understanding. This may explain in part why deep convolutional neural networks (CNNs), which have been so successful in processing natural images, have not been widely adopted for the analysis or synthesis of 3D indoor scenes (Li et al., Reference Li, Patil, Xu, Chaudhuri, Khan, Shamir, Tu, Chen, Cohen-Or and Zhang2019b). Framework for interior scene synthesis that combines a high-level relational graph representation with spatial prior neural networks is presented by Wang et al. (Reference Wang, Lin, Weissmann, Savva, Chang and Ritchie2019). They suggest refining the treatment of superstructures (kitchens, wardrobes against the wall, a table surrounded by chairs) to guarantee more precise arrangements. While their method delivers comparable quality of generated layouts to other state-of-the-art methods, they reported that it is not yet at the level of human-like scene design capabilities, especially in the case of kitchens. An unconditional scene generator that trains variational recursive autoencoder is presented by Li et al. (Reference Li, Patil, Xu, Chaudhuri, Khan, Shamir, Tu, Chen, Cohen-Or and Zhang2019b), where new scene can be generated by decoding from a randomly generated code. They reported that for kitchens, the training set is biased toward kitchen-cabinets, which limits the variety of object classes in the generated scenes. The method developed by Henderson et al. (Reference Henderson, Subr and Ferrari2019) model a kitchen using Markov chain with the terminal state.
None of the abovementioned works or any other prior work in automatic kitchen generation of which we are aware considers input features from real-life kitchen design considerations, different subtypes of major kitchen elements and continual length of working surfaces.
In our previous research (Pejic et al., Reference Pejic, Jovanovic, Marinkovic and Krasic2019a), we create a simple system for parametric 3D modeling of linear kitchen, assuming that input parameters are in an ideal order. In the second paper (Pejic et al., Reference Pejic, Mikic and Milovanovic2019b), we designed a rule-based system that takes into account any order of input parameters. Due to the complexity and limitations of the applied approach, the final system shows only 69.2% accuracy.
A series of studies emphasized the importance of data in interior design automatization (Fisher et al., Reference Fisher, Ritchie, Savva, Funkhouser and Hanrahan2012; Zhu et al., Reference Zhu, Guo and Ma2018; Wu et al., Reference Wu, Fu, Tang, Wang, Qi and Liu2019; Fu et al., Reference Fu, Cai, Gao, Zhang, Wang, Li, Xun, Sun, Jia, Zhao and Zhang2021). Although there is a steady accumulation of datasets in recent years (Fu et al., Reference Fu, Cai, Gao, Zhang, Wang, Li, Xun, Sun, Jia, Zhao and Zhang2021), as long as we know there is no publicly available kitchen layout dataset with complete kitchen annotations such as positions of canalization pipes, ventilation, and electricity. Therefore, existing data-enabled kitchen design approaches are limited to a specific group of real-life kitchen design problems.
Dataset
The dataset is created by five interior designers with an average of 9.4 years of experience in kitchen design and layout organization. To inspire more research created, dataset is publicly available at https://github.com/jelena6k/ProceduralKitchenGeneration. It consists of 1,000 examples of acceptable linear kitchen layouts, separated into four subsets that simulate different real-life use cases:
-
Subset 1 – different kitchen lengths, with only one input parameter (position of entry door). Simulates the cases of new building in the design phase. The position of sink drain and ventilation is not determined yet, allowing creation of ideal kitchen elements disposition. Existing datasets cover only examples from this subset.
-
Subset 2 – different kitchen length, with input parameters positioned in ideal order: entry door at the beginning of the kitchen, sink drain, ventilation, and exit at the end of the kitchen. Simulates the cases of existing space where WT principles are implemented to have an ideal kitchen elements disposition.
-
Subset 3 – different kitchen length, with different input parameters without ideal order and without overlapping of input parameters or very close position (e.g., the position of sink drain and ventilation is not allowed to be closer than 30 cm). Simulates the cases of existing space where WT principles are not implemented and ideal kitchen elements disposition is not possible.
-
Subset 4 – different kitchen length, with different input parameters and overlapping or very close positioned input positions of sink drain, ventilation, and window (e.g., the position of sink drain and ventilation is less than 30 cm). Simulates the cases of existing space where WT principles are not implemented and ideal kitchen elements disposition is not possible. Overlapping does not allow direct connection to the ventilation or sink drain, making these the most challenging real-life situations.
Each example in dataset consists of six discrete target fields (labels) and six continuous input features (Fig. 4).
Target fields are subtypes and positions of three major kitchen elements: (1) Washing subtype; (2) Washing position; (3) Cooking subtype; (4) Cooking position; (5) Storage subtype; and (6) Storage position.
We assume that these fields completely describe linear kitchen layout. Space between major kitchen elements is filled with “Work surface” elements. Prediction task determines target fields (generate linear kitchen layout) for a given input features. For each training example, we train our PKG model to predict the same layout as in example, for a given example's input attribute.
In order to create a dataset, the kitchen layout annotation tool was developed (Fig. 5). Annotation tool is based on the previously created software for rule-based linear kitchen layout generation (Pejic et al., Reference Pejic, Mikic and Milovanovic2019b).
Annotation tool used for dataset creation.
Experts created acceptable linear kitchen layout using the annotation tool. Annotation tool randomly generates input features for each example. The rule-based system, then helps expert by suggesting four different layouts that are visually presented as a two-dimensional top-down view (Fig. 5). The expert then chooses one acceptable layout from the presented four. If he cannot find an acceptable layout between four presented, he is able to create a new layout manually using “Force” tool.
We hold out 100 examples from dataset for testing and the rest is used for training.
Methodology
ML problem definition
To create PKG model based on ML classifier, it is necessary to define input features. Based on previously conducted interviews with professionals (Pejic et al., Reference Pejic, Mikic and Milovanovic2019b), six important features in determining kitchen layout are singled out: kitchen length, ventilation position, sink drain position, window position, entrance position, and exit position. Due to standard depth and height of kitchen elements (Pejic et al., Reference Pejic, Jovanovic, Marinkovic and Krasic2019a), three-dimensional PKG problem is addressed as one-dimensional. Important input features are marked only by their position from the beginning of a kitchen.
The kitchen is composed of three major kitchen elements: washing – one sink, cooking – one oven, and storage – one refrigerator. Space between major kitchen elements is filled with working surfaces, with a minimal length of 30 cm and a maximal of 90 cm for each one of them. Accordingly, we assume that kitchen layout can be described by six parameters: washing position, washing subtype, cooking position, cooking subtype, storage position, and storage subtype. Based on that assumption, PKG model consists of two main components:
• Predicting major kitchen elements subtype.
• Predicting major kitchen elements position.
Both sub-problems are addressed as classification problems.
Proposed machine learning approach
Elements subtype prediction
Element subtype prediction is considered a supervised classification problem. For all three major kitchen elements, there are three subtype classes that are given in Table 1. Standard (modular) types and sizes are used (Pejic et al., Reference Pejic, Mikic and Milovanovic2019b).
Kitchen elements types, subtypes, and their lengths
If the cooking element belongs to the first class, cooking and storage forms the same element. In that case, storage belongs to zero subtype and there is no need for storage subtype prediction.
Element position prediction
Element position prediction is considered a supervised classification problem. Kitchen length is limited to the range of 90–600 cm. Following modular design principles, possible starting positions for major kitchen elements can be at intervals of 30 cm from one or another edge of the kitchen. Considering that, we divided element positions into 40 classes. First 20 classes represent positions at 0, 30, 60 cm, etc. from the left edge. Classes 21, 22…40 represents positions at 0, 30, 60 cm, etc. from the right edge. For example, 127 cm long kitchen have 10 possible starting points of the elements (Fig. 6).
Methodology for elements starting points determination.
Kitchen layout prediction
The proposed PKG model consists of six sequentially arranged modules, as given in Table 2.
Kitchen layout generation pipeline
Each module solves one of two main problems: subtype or position prediction for one of the three major kitchen elements. According to this, one classifier was trained for each step. Respecting the pipeline, the output of one classifier is an additional feature for the next classifier.
The model takes as input six features (Fig. 4) given by the user. The first module predicts washing subtype. Next module takes washing subtype and input features as its input and predicts washing position. Next module takes washing subtype, washing position, and input features as its input and predicts cooking subtype. Cooking position module takes washing subtype, washing position, cooking subtype, and input features as its input and predicts the cooking position, etc. The last module has 11 features. This order of steps is highly motivated by an architects’ design procedure that presumes washing positioning at first (based on sink drain position mainly) then cooking (based on ventilation position mainly) and finally storage positioning. Conducted tests show that the different order of steps decreases accuracy 3% on average. During the model training phase, each model's input features are taken from training example and models are trained independently. Models are trained to predict target fields from training example (Fig. 4).
Machine learning classifiers
Data-preprocessing, classifier training, and testing were implemented using Python 3.6 and scikit-learn package. Various supervised classifiers have been tested for each PKG model's step.
Decision tree classifier continuously split data according to a certain parameter. Each node is labeled with an input feature and each leaf is labeled with a class. Random forest is an ensemble method that consists of many uncorrelated decision trees that have low bias, but high variance, created by choosing random features. The prediction is the average of decision trees, so the low bias property is preserved while over fitting is decreased. Adaptive Boosting (AdaBoost) combines many “weak” leaners (e.g., Decision tree with a single split) into a weighted sum that represents final output, by putting more weight on instances that are difficult to classify and less on those already well classified. Naive Bayes is a probabilistic classifier that is based on Bayes’ theorem with a strong, naive assumption that all features are independent. Multilayer perceptron (MLP) is an artificial feed forward neural network that models nonlinear problems using principles from biological neural nets. It is based on a collection of connected units called neurons. Support vector machine (SVM) uses kernel trick techniques to transform data into a high-dimensional space where it finds optimal linear boundary between examples that belong to different classes. L2 logistic regression models the probability of a certain class, and penalized high weights with L2 norm. One-vs-all algorithm is used to combine logistic regression binary classifiers for multiclass problem.
For hyper-parameters tuning grid search with cross-validation is performed on the training set. Optimal parameters for each algorithm are given in Table 3.
Optimal algorithms parameters
Kitchen elements 3D models
Five most popular kitchen styles in 2020 (Research, Reference Research2020) are: Transitional – 21%, Contemporary – 16%, Modern – 15%, Traditional – 11%, and Farmhouse – 11%. Based on these facts, 3D models of kitchen elements in five most popular design styles are created using SketchUp 3D modeling software.
Based on used subtypes and their lengths (Table 1), 3D models are created for washing, cooking, storage, and additional working surface. Totally, 50 kitchen elements 3D models were created, 10 per each style (Fig. 7). Models’ geometry scaling is defined inside Unity and converted to final 3D prefabs.
3D models of kitchen elements in contemporary design style.
Results and discussion
In this section, we define evaluation metrics (Metrics) and make a comparison between different ML classifiers for each step of PKG pipeline (Classifier selection). Random forest gives the best results on the given dataset, so we analyze its performances (PKG model quantitative analysis). PKG model is then integrated into the Unity engine as a module for automatic kitchen 3D model generation and presentation (3D model generation). We generated 100 kitchen layouts using the Game engine module and evaluated them in perceptual study (Perceptual study). Finally, results are compared against rule-based approach and outperformed it for 8.2% (Comparison against rule-based approach).
Metrics
To evaluate our approach, we measure the quality of generated layouts using three different metrics:
• Exact matching – measures the similarity of generated to human-designed layouts. Kitchen element is considered correctly classified only if it is positioned exactly on the same place as in human-designed layout. This is the variation of IoU (Intersection-over-Union) – measure of the intersection between the predicted and the human-designed layout, used in Di and Yu (Reference Di and Yu2021).
• Livability – measures the suitability for living. Participants were asked to judge all generated scenes if they are livable or not (i.e., if they satisfy the aesthetic, ergonomic, and functional requirements which are necessary for living in these scenes). We use the same study design as in Kan and Kaufmann (Reference Kan and Kaufmann2018).
• Applicability – measures the degree to which a layout is applicable in designers practice. Human subjects were asked to rate each of generated scenes on a scale of 1 to 5, with 5 indicating that layout can be used without any correction and 1 indicating that layout must be designed from scratch. Our experimental setup was inspired by QI et al. (Reference Qi, Zhu, Huang, Jiang and Zhu2018) and Li et al. (Reference Li, Patil, Xu, Chaudhuri, Khan, Shamir, Tu, Chen, Cohen-Or and Zhang2019b).
Classifier selection
In the model selection phase, aforementioned classifiers (Table 3) are tested for each PKG model's step independently. It is presumed that previous steps of the pipeline are 100% accurate and input features for each classifier are taken from the example. We compare the output of each classifier with human-designed layouts from train set using Exact matching. To select the best classifier for each step, we performed a 5-fold cross-validation technique on the training set and use classification accuracy to measure the performance. The results are given in Table 4.
Classifiers accuracy on train set using cross-validation (%)
The significance of bold is the best performance classifier.
Input features in our work forbid same furniture dispositions (e.g., cooking must be as far as possible from washing) and even if the intersection with human-designed layout is high, predicted layout can be practically unacceptable, so we consider each prediction as correct only if the intersection with human-designed is 100%. To the best of our views, the current state-of-the-art models do not consider the same input features as in this paper so we cannot make a fair comparison. It could nevertheless be argued that our method at least leads to good results: if we assume zero IoU for positions that are not acceptable, Random Forest IoU is 80%. These results are better than the IoU for kitchen layouts reported in Di and Yu (Reference Di and Yu2021) using methods proposed by Wang et al. (Reference Wang, Lin, Weissmann, Savva, Chang and Ritchie2019) – 62%, Li et al. (Reference Li, Yang, Hertzmann, Zhang and Xu2019a) – 61%, and Di and Yu (Reference Di and Yu2021) – 73%.
PKG model quantitative analysis
Random forest proved to be better than the other classifiers in all prediction tasks. As the best classifier, it is used in each step of the PKG model. PKG model performances are tested on 100 test set examples, using Exact matching. Input features of test set examples are used as input to the PKG model. Accuracy of the complete PKG model is calculated, where an example is classified as correct only if all predicted subtypes and positions are exactly as in the test example (i.e., if all PKG model's steps return right prediction). In order to get insight which steps are more difficult to predict, the accuracy of each step of the PKG model is also calculated (Table 5).
PKG model exact matching accuracy on test set using random forest in each step (%)
To explore Random forest decisions, the features importance is calculated as Gini Importance (Mean decrease gini):
$$\eqalign{&{{\rm importance}( {{\rm feature}} })= \cr &\ \qquad {\displaystyle{1 \over {N_T}}\mathop \sum \limits_T \displaystyle{{\mathop \sum \nolimits_{{\rm node\,}\epsilon \,T\colon {\rm feature}( {{\rm node}} ) = {\rm feature}} {\rm importance}( {{\rm node}} ) } \over {\mathop \sum \nolimits_{{\rm node\,}\epsilon \,T} \;{\rm importance}( {{\rm node}} ) }}}},$$where N T is the number of trees in Random forest, T is a set of trees, feature(node) is a feature used for splitting the tree in the node, and importance(node) is calculated as follows:
$$\eqalign{{\rm importance}( {{\rm node}} ) & = w( {{\rm node}} ) {\rm gini}( {{\rm node}} ) \cr & \quad -\;w( {{\rm nod}{\rm e}_{{\rm left}}} ) {\rm gini}( {{\rm nod}{\rm e}_{{\rm left}}} ) \cr & \quad -w( {{\rm nod}{\rm e}_{{\rm right}}} ) {\rm gini}( {{\rm nod}{\rm e}_{{\rm right}}} ),} $$where w(node) is the weighted number of samples reaching node and gini(node) is gini impurity for the node.
Features importance for every PKG model's step reveals a good fit of the Random forest decision making algorithm to the experts one (Fig. 8).
Feature importance: 0 – kitchen length, 1 – ventilation position, 2 – sink drain position, 3 – window position, 4 – entrance position, 5 – exit position, 6 – washing subtype, 7 – washing position, 8 – cooking subtype, 9 – cooking position, 10 – storage subtype.
The most important feature for washing subtype prediction is kitchen length and for washing position is a sink drain position. Cooking subtype mostly depends on kitchen length and washing subtype while cooking position depends on ventilation position, kitchen length, and washing position. Storage subtype prediction is mostly based on kitchen length, washing, and cooking subtype, and storage position is decided based on cooking and washing positions mostly. These rules are easily interpreted and followed by experts in most kitchen layout design processes.
Cooking position prediction highly depends on washing position (Fig. 8), which explains why cooking position accuracy is always lower than washing position accuracy (Table 5). On the other hand, storage prediction has the highest accuracy, despite great dependency on washing and cooking predictions. If we look at the training set (and kitchen layouts generally), most of the time storage is placed in the corner of the kitchen. Therefore, even if the algorithm does not put washing or cooking in the exact position, that does not affect its decision to put a storage in the right corner.
It is worth mentioning that joint accuracy is smaller than separate accuracy, which reveals that mistakes are not always correlated that is there are examples where only one prediction step makes a mistake.
3D model generation
We implement our PKG model in Unity engine version 2019.3.4f1. in order to allow 3D visual representation of generated kitchen layouts (Fig. 9) and conduction of perceptual study.
Spatial kitchen representation inside unity scene.
Generated kitchen designs can be easily edited by the designer, using the Unity Editor. Moreover, our algorithms can be easily integrated into interior visualizations, video games, virtual and augmented reality systems, or other real-time applications to generate virtual kitchen environments.
Perceptual study
In reality, kitchen design is not exact – rule-driven process, it is an artistic one. In some cases, this results in multiple correct layouts for same kitchen input parameters, depending on designers’ taste and aesthetic preferences. On the other side, even if the generated layout is not acceptable, it needs just a little correction. For this reason, perceptual – expert evaluation is performed with 23 participants, divided in four groups in accordance with their professions:
• 8 interior designers, with an average of 8.2 years of experience,
• 5 architects, with an average of 9.5 years of experience,
• 3 game level designers, with an average of 3.1 years of experience, and
• 7 master students of interior design, with no work experience.
Same test set used in the quantitative PKG model analysis is used for kitchen layout generation inside the Unity game engine. A visual 3D representation of 100 kitchen layouts is evaluated in two perceptual studies.
In the first study, we ask participants to judge all generated layouts if they are livable or not. In some cases, generated kitchen layout was acceptable to one expert, but not to another, so the final result represents an average of their decisions for each subset. Results of Livability study for each subset of the test set and the complete test set are presented in Table 6.
PKG model exact matching versus livability study (%)
In the second study, the same set of participants are asked to rate each of layouts based on their applicability. Participants express their opinion for each layout, based on the need for additional corrections in order to be applicable in their current work practice. We use scale 1 (have to be designed from scratch) to 5 (does not need any corrections). Mean ratings of applicability study for each group of participants and total results are presented in Table 7.
PKG model applicability study
In addition, participants are asked to give qualitative feedback regarding the PKG model and how they see such data-enabled process in their practice. Most of participants reported that the created tool is useful and can speed up the process of kitchen design. All participants, except game-level designers express a need for PKG model integration in other, profession specific 2D and 3D software's such as: AutoCAD, 3D Max, and Sketchup.
Interior designers see potential of PKG model to be used as a tool for easier communications with clients and faster design creation. The reason for such claims can be found in the traditional kitchen design process. Based on homeowner preferences and dimensions, the designer creates 3D visualizations. Created 3D kitchen design is then evaluated by homeowners and usually redone a couple of times until final approval. Each change requires additional 3D modeling that takes a couple of hours and a new meeting for evaluation of the new design. PKG model significantly shortens the 3D modeling process and can allow change of 3D design on the spot, which removes the need for multiple meetings. For the same reason, they also suggested larger database of 3D models in final version, with options for editing. Due to the limitations of having only one stove, fridge and sink, they commented that the model is unsuitable for luxury interiors.
Architects noted that the PKG model can be useful in the building design phase during 2D floor plans creation. It can make process easier and faster. Reason can be found in the purpose of 2D floor plans, which do not require a precise and final kitchen layout. A presented 2D kitchen layout has to be logical and presented in the right location, for which the PKG model presents an ideal solution.
Game-level designers reported that the PKG model can be very useful for levels with a lot of interiors. They appreciate integration inside Unity engine and reported that except kitchen length all input features are redundant in their practice. The reason is that in-game 3D models of the interior do not need to be functional (the sink is not connected to the drain, the stove is not connected to ventilation, etc.). The only important feature is a player's visual experience which should give the impression of a realistic interior.
Master students of architecture and interior design see potential for use of PKG model as a learning tool. The reason is the complexity of the rules that have to be implemented during the kitchen layout design. PKG model allows them to easily generate kitchen layouts and to learn based on examples. They noted that biggest problem in their residential interior design projects is design and 3D modeling of kitchen and created tool can help them a lot.
Livability study results demonstrate measurably acceptable rate increase compared to Exact matching, for subset 3 – 21.3%, subset 4 – 10%. Meaning that, in 10.4% cases, PKG model generates acceptable layout but different than one in test set. Furthermore, applicability study indicates that resulting kitchens designs require minimal corrections to be useful in practice. Overall, the perceptual study shows that the PKG model can be considered as a good supporting tool for kitchen design.
Comparison against the rule-based approach
In addition, we compare the results obtained in perceptual study with the results from the rule-based approach (Pejic et al., Reference Pejic, Mikic and Milovanovic2019b). Both researchers use the same test set and methodology for perceptual study, allowing comparative analyses of the PKG model and rule-based system. Results for each subset individually and for the complete test set are presented in Figure 10.
Comparison of PKG model accuracy versus rule-based system accuracy in perceptual study for each subset and for complete test set.
Results show that in cases of subset 1 and 2, both approaches give maximal results. In the case of subset 3 and 4, the PKG model based on Random forest classifier outperforms the rule-based system. In total, the PKG model have 77.4% accuracy compare to the rule-based approach which has 69.2%. The difference of 8.2% in benefit of the PKG model reveals that ML has managed to learn some complex patterns that cannot be described by kitchen design rules.
Conclusion
In this research, we propose a novel data-enabled approach to linear kitchen layout design which is based on machine learning models that learn complex design constraints from a dataset. Our tool takes features from real-life kitchen design considerations as input and generates layout suggestions that can be used without or with a small correction. We create and publish a dataset of linear kitchen layouts which contains annotations from real-life kitchen design considerations which we believe represents a valuable source for future data-enabled kitchen design approaches. The Procedural Kitchen Generation model presented in this work is a pipeline of six Random forest ML classifiers trained and tested on a given dataset. PKG model is integrated into the Unity Game Engine for the purpose of automatic 3D kitchen generation and presentation. The developed Unity module presents a significant resource for easy and fast kitchen 3D model generation that could speed up a design process in the case of interior design, computer games, or serve as a tool for creation and visualization of some predefined layouts. In the case of interior design, it can be used to show the client different variations of chosen layout type in their kitchen space and assists individuals who does not have enough design experience.
The experimental results demonstrate that our model can generate sensible layouts which can highly support kitchen design process. We explain our experimental settings, evaluate different classifiers, present quantitative and perceptual study results, and make comparison to close alternatives.
Results in this paper indicate that ML is capable to some extent of learning patterns for solving complex problems that require design and engineering skills, such as kitchen layout generation. The proposed pipeline is highly motivated by experts’ design procedure. Feature importance reveals great matching of the algorithms and experts’ decision-making process.
Moreover, the performance measurements reveal that for each subset and for a complete test set machine learning-based model has equal or better performances than a rule-based system created by experts. PKG model achieved 100% accuracy on subset 1 and 2 but on subset 3 and 4 there is still room for accuracy improvement.
Perceptual study participants reported that kitchen layout designs generated by the PKG model are useful in 3D kitchen design, 2D floor plans design, game level design, and education. They expressed that created tool provides useful layout suggestions and can speed up designers’ work, but to reach the final design in some cases it still requires human corrections. The greatest value of such data-enabled design can be found in interior designers’ practice, as the process can significantly speed up the work and make communication with clients easier.
Overall, experimental results indicate that even though the PKG model needs human intervention for the final output, it has the potential to tremendously help designers in their practice by providing valuable layout suggestions based only on the previously seen layouts from the given dataset.
Future work
Despite the success of PKG model, there are still opportunities for further enhancements. Chain dependency impacts model accuracy and other approaches need to be considered.
Bigger dataset would allow usage of deep neural networks instead of Random forest, or prediction of complete layout in one pass. Leveraging deep neural networks seems like a fruitful direction for future work.
Further research should also consider other types of kitchens (II-shape, L-shape, etc.). This will bring another level of complexity. For future studies, we propose linearizing kitchen layouts or leveraging convolutional neural network for image understanding.
Created model is also limited to only one of each major kitchen workstations (“Washing” – one sink, “Cooking” – one stove, and “Storage” – one refrigerator). Further research should take into account the unlimited number of same types of workstation elements. One possible direction for future work is to use PKG pipeline in an iterative process, where new elements will be added until the termination criteria is met. Another possible direction is to use recurrent neural networks.
Despite the limitations, we believe that the presented results are encouraging and can serve as a base for future studies in the area of procedural kitchen generation.
Jelena Pejic received the BS and MS degrees in Computer Science from the Faculty of Science and Mathematics, University of Nis, Serbia. She is currently pursuing the PhD degree Computer Science from the Faculty of Science and Mathematics, University of Nis, Serbia. Since 2019, she has been a Teaching Assistant at the Computer Science Department, Faculty of Science and Mathematics, University of Nis, Serbia.
Petar Pejic received the BS and MS degrees in architecture from the Faculty of Civil Engineering and Architecture, University of Nis, Serbia. In 2019, he received the PhD degree from the Faculty of Technical Science, University of Novi Sad, Serbia. Since 2019, he has been an Assistant Professor with the Game Development Department, Faculty of Information Technology, Belgrade Metropolitan University, Serbia.
