1. INTRODUCTION
In the context of computer-aided design (CAD), the term free form is often used to express the geometry that does not have an explicit mathematical expression and can only be modeled approximately with polynomial patches. In addition, free forms can be described as unique and unrepeatable. Free forms are most often used in design and digital reconstruction of industrial design objects, art objects such as sculptures (Pernot et al., Reference Pernot, Falcidieno, Giannini and Leon2008), as well as during digital reconstruction of biological forms such as human bones, bone implants, and scaffolds (Hieu et al., Reference Hieu, Sloten, Hung, Khanh, Soe, Zlatov, Phuoc and Trung2010). Of course, digital reconstruction is just the first and indispensable step in a series of design activities that have to be undertaken in order to fabricate objects with free-form characteristics. In all of these activities, it is necessary to manipulate the geometry of free forms, which also makes the challenge more current.
One of the biggest challenges associated with design and digital reconstruction of free forms comes from uniqueness and unrepeatability of these shapes. While digitally reconstructing these forms, the designer has to choose the right set of geometric features from the palette provided by the CAD program and then compose them in a way that will enable the most accurate reconstruction of the geometry. At the same time, one should bear in mind that the free-form reconstruction process also must satisfy a required level of productivity. A designer primarily relies on personal experience gained through work with objects of similar geometry. However, to make decisions regarding the method of reconstruction, besides geometric characteristics, a designer also takes into account other, so-called semantic features of the free-form object. For example, those could include knowledge about the production process of the free-form object, or in the case of bone implant, the knowledge about implantation or about the functions of certain anatomical elements. Therefore, the incorporation of knowledge in the integrated model of the product, at the very beginning of product creation, is a valuable step in the effort to prevent errors of individual participants, increase productivity, and create extra time for creativity. This knowledge can be in the forms of requirements, restrictions, rules, and recommendations and originating from all participants in the product creation process.
In this paper, an approach is described where a new semantic model called the active semantic model (ASM) and accompanying cognitive data processing algorithms were used for modeling and the semantic interpretation of free-form semantic features facilitating selection and composing of geometric features for efficient creation and/or reconstruction of the geometry of free-forms. The ASM structure and the approach are explained, and demonstrated through a case based on a real situation. Implementation details are given for the web application that has been developed in order to test the approach.
2. RELATED WORK
2.1. Aiding design tools
Design decisions can be supported with different tools operating on different kinds of knowledge. Some of them are case-based reasoning (CBR), analogy-based reasoning (ABR), and constraint filtering. They have been initially proposed for product design, but now are also widely used for aiding the design of systems, processes, or services.
CBR (Riesbeck & Schank, Reference Riesbeck and Schank1989; Kolodner, Reference Kolodner1992; Aamodt & Plaza, Reference Aamodt and Plaza1994) is frequently used approach to problem solving and learning. In CBR a new problem is solved by remembering a previous similar problem situation (case) and by reusing information and knowledge of that problem situation. The general CBR cycle is very well described by Aamodt and Plaza (Reference Aamodt and Plaza1994).
CBR is a methodology (Watson, Reference Watson1999), and is not suitable for every domain of application. Because CBR may use any appropriate technology, the challenge is to come up with technologies appropriate for problem solving and learning in a specific domain and for a specific application environment.
The first problem faced by researchers in application of CBR is knowledge representation. Three major types of case representation are feature vector cases, structured cases, and textual cases (Bergman et al., Reference Bergman, Kolodner and Plaza2006). Feature vector approach represents a case as a vector of attribute–value pairs, while structured approach represents a case as clusters of relations between the kinds of elementary objects that comprise it.
Case representation and the way similarity is assessed during retrieval are strongly related to each other. In some applications of CBR, similarity of stored cases is assessed in terms of their surface features, which are parts of their description typically represented using attribute–value pairs. Various methods exist: k-nearest neighbor (k-NN) based on Euclidean distance; mixed neural networks (Chang et al., Reference Chang, Lin and Dzan2012); fuzzy logic (Begum et al., Reference Begum, Ahmed, Funk, Xiong and Scheele2009); and genetic algorithms (Passone et al., Reference Passone, Chung and Nassehi2006). Structured cases often require knowledge-intensive matching algorithms to assess structural similarity. Experiments confirmed that both surface and structural similarity assessment are necessary for sound retrieval (Forbus et al., Reference Forbus, Gentner and Law1995). Retrieval based solely on similarity has limitations. That is why similarity is increasingly being combined with other criteria to guide the retrieval process, such as adaptability of the retrieved case (Smyth & Keane, Reference Smyth and Keane1998; Negny et al., Reference Negny, Riesco and Le Lann2010).
There are two main ways for case reuse: reuse of the past case solution (transformational reuse), and reuse of the past method that constructed the solution (derivational reuse; Aamodt & Plaza, Reference Aamodt and Plaza1994). Past case solution is rarely used without modification. Solution adaptation is a difficult step of CBR approaches and mostly is considered a human process. Knowledge for case adaptation is harder to acquire and demands a significant knowledge engineering effort (Policastro et al., Reference Policastro, Carvalho and Delbem2006). According to Romero Bejarano et al. (Reference Romero Bejarano, Coudert, Vareilles, Geneste, Aldanondo and Abeille2014) only the tacit knowledge of the designer can be used for adaptation. This kind of knowledge is tied to experiences, intuition, unarticulated models, and implicit rules of thumb (Chandrasegaran et al., Reference Chandrasegaran, Ramani, Sriram, Horvath, Bernard, Harik and Gao2013). Negny et al. (Reference Negny, Riesco and Le Lann2010) present one of the few approaches to solution adaptation for the system where the cases are structured with attribute–value pairs.
Most CBR systems nowadays make use of general domain knowledge in addition to knowledge represented by cases. General domain knowledge can be expressed as constraints linking the variables of the problem (Chenouard et al., Reference Chenouard, Granvilliers and Sebastian2009), that is, formalized as the constraint satisfaction problem (CSP). In these situations the tool of choice for operating with general knowledge would be constraint filtering. The reasoning process consists in reflecting requirements defined by the user through the constraints network to the other variables by limiting their domains only to consistent values. The constraint satisfaction problem has been used in many works dealing with aiding design (e.g., Bodirsky & Dalmau, Reference Bodirsky and Dalmau2006; Vareilles et al., Reference Vareilles, Aldanondo and Gaborit2007; White et al., Reference White, Dougherty, Schmidt and Benavides2009). There are also several studies focused on the sequential use of CBR operating with contextual knowledge, and constraint filtering operating with general knowledge (Sqalli & Freuder, Reference Sqalli and Freuder1998; Roldan et al., Reference Roldan, Neagny, Le Lann and Cortes2010). Vareilles et al. (Reference Vareilles, Aldanondo, Codet de Boisse, Coudert, Gaborit and Geneste2012) used constraint filtering and CBR tools simultaneously to support design decisions of maintenance processes for helicopters, which led to more accurate result and better quality information.
An ontology may provide a formal semantic representation of the objects for structured case representation in CBR methodologies (Lau et al., Reference Lau, Tsui and Lee2009), as well as methods for similarity assessment (Cordi et al., Reference Cordi, Lombardi, Martelli and Mascardi2005; Batet et al., Reference Batet, Sanchez and Valls2011). In the approach presented by Romero Bejarano et al. (Reference Romero Bejarano, Coudert, Vareilles, Geneste, Aldanondo and Abeille2014), the CBR process for system design in the aeronautic domain is based on an ontology to assist requirements definition, the retrieval of compatible cases, and the solutions definition. To take into account uncertainty and the unavailability of similarity measures between attributes values and to enlarge the scope of retrieval, requirements are modeled using flexible constraints defined upon the designer's preferences. The retrieval process has two phases: the first is based on conceptual similarity measures, while the second is based on the compatibility between flexible constraints and preselected solutions. Guo et al. (Reference Guo, Hu and Peng2012) developed a CBR system for injection mold design, based on the use of ontology, which can work under the context of incomplete design information, with two grades of retrieval strategy: the first one is ontological semantic retrieval, while the second one measures numerical similarities of structure parameters. Semantic similarity measurement is improved by combining information content similarity and node distance similarity in ontology. Another interesting approach is the cognitive experience feedback framework (Jabrouni et al., Reference Jabrouni, Kamsu-Foguem, Geneste and Vaysse2011, Reference Jabrouni, Kamsu-Foguem, Geneste and Vaysse2013) used for exploitation of expert knowledge during problem solving processes. The authors used the conceptual graphs formalism (Sowa, Reference Sowa2000) for the semantic conceptualization of the domain vocabulary. Information retrieval is enabled by formal reasoning mechanisms in conceptual graphs.
Studies (Gentner, Reference Gentner1983; Carbonell, Reference Carbonell, Michalski, Carbonell and Mitchell1986) show frequent use of past experience in solving new and different problems by analogy. ABR is often used to characterize methods that solve new problems based on past cases from a different domain, while typical CBR methods focus on indexing and matching strategies for single-domain cases (Aamodt & Plaza, Reference Aamodt and Plaza1994). ABR research focuses on finding mechanisms for identification and utilization of cross-domain analogies (Kedar-Cabelli, Reference Kedar-Cabelli and Helman1988; Hall, Reference Hall1989), primarily on finding a way to transfer the solution of an identified analogue problem to the present one.
Analogical transfer requires the use of generic abstractions, which express the structure of relationships between generic types of objects and processes (Goel, Reference Goel1997). In analogy-based design, these abstractions specify the structure of relations among the elements of the design problem, solution, domain, or strategy, and where transfer can occur to fulfill any design task in the new situation (Chakrabarti et al., Reference Chakrabarti, Shea, Stone, Cagan, Campbell, Hernandez and Wood2011).
2.2. Knowledge-based engineering
In general, knowledge incorporation, meaning representation, and application of domain knowledge in appropriate engineering software packages [CAD/computer-aided manufacturing (CAM), or computer-aided engineering] is realized by their upgrading, which usually implies the use of so-called knowledge base and inference engine. These upgraded engineering software systems are called knowledge-based engineering (KBE) systems. KBE is defined as the use of dedicated software language tools in order to capture and reuse product and process engineering knowledge in a convenient and maintainable fashion. The ultimate objective of KBE is to reduce the time and cost of product development by automating repetitive, noncreative design tasks and by supporting multidisciplinary integration in the conceptual phase of the design process and beyond (Cooper & LaRocca, Reference Cooper and LaRocca2007). Successful KBE application cases can be found in automotive (Chapman & Pinfold, Reference Chapman and Pinfold2001; Stojkovic et al., Reference Stojkovic, Manic and Trajanovic2005), aerospace (La Rocca & Van Tooren, Reference La Rocca and Van Tooren2007; Corallo et al., Reference Corallo, Laubacher, Margherita and Turrisi2009), and manufacturing domains (Kulon et al., Reference Kulon, Mynors and Broomhead2006; Van der Laan et al., Reference Van der Laan, Curran, Van Tooren and Ritchie2006). Modern CAD systems have the capability of knowledge representation and utilization by means of production rules. However, production rules and modular expert systems, such as the current standard KBE add-in in CAD software packages, can be effectively used in situations of very similar design solutions, where the individual design solutions are provided and built in a knowledge base. Stokes (Reference Stokes2001) mentioned some situations where the use of KBE is not indicated: when the design process consists of creative processes and products that are highly subject to change and when the design process cannot be clearly defined (it is not possible to isolate and define particular stages in the design process). In addition, some shortcomings of KBE can be noticed, such as case-based, ad hoc development of KBE applications; a tendency toward development of blackbox applications; and a lack of knowledge reuse (Verhagen et al., Reference Verhagen, Bermell-Garcia, van Dijk and Curran2012). One of the promising approaches implies using ontologies for modeling product and process engineering knowledge and making knowledge sharing and reuse easier among different participants (Tomiyama, Reference Tomiyama2007).
2.3. Semantics in the free-form domain
Certain modern approaches to geometric modeling take into account semantic features of free-form geometric elements. Cheutet (Reference Cheutet2007) discussed a semantic-based modeling environment used in the conceptual phase of a car design. Such an environment was based on a structuration of the semantics embedded in the first two-dimensional sketches of the car. To manipulate such sketches, process grammar has been integrated to describe and manipulate the curves and finally implemented through the use of a deformation engine adapted to NURBS curves. Cheutet et al. (Reference Cheutet, Catalano, Giannini, Monti, Falcidieno and Leon2007) also created an ontology to guide more easily the creation and manipulation of curves, which are key elements of the car description in the early design phase. This ontology included the taxonomy of the aesthetic key lines and their aesthetic properties and two-dimensional curve grammar, providing a description of the curve geometry in terms of high-level operators that stylists can easily understand, avoiding the manipulation of low-level geometric parameters. Piegl (Reference Piegl2006) developed a knowledge-guided NURBS modeling system. The authors' main goal was to acquire knowledge about design intent and embed it into the geometric model in order to support design reproducibility. Other goals were to introduce capabilities for knowledge acquisition, deduction, and mining and to ensure compatibility between different platforms.
2.4. Analysis of the related work
Although numerous approaches to supporting design decisions can be found in the literature, only a few of them are from the free-form domain. In these approaches, authors are relying on ontology developed for the specific narrow problem domain, like car design. Taking into account uniqueness and unrepeatability of free forms, one can imagine what would be the difficulties faced by researchers trying to develop an ontology for supporting design decisions during digital reconstruction of free-form objects. Because the designer is primarily relying on experience in these situations, it is obvious that the CBR methodology should be used. The challenge is to select the appropriate technologies. Based on the analysis of the process of free-form objects, digital reconstruction, and analysis of the literature, some conclusions can be made. Possible problem situations should be represented as structured cases where elements of the design problem (like names of the free-form object parts and their features), solution (like the names of geometric features used), domain, or strategy are connected by relations. The structure of the possible problem situations is unpredictable. During the retrieval of the compatible problem situations, both surface and structural similarity should be assessed (Forbus et al., Reference Forbus, Gentner and Law1995). Because it is impossible to predict a finite number of potential free-form variants, efficient surface similarity assessment should be used. Finally, during the transfer of the solution from the selected compatible case, a structure of relations among the elements of the design problem, solution, domain, or strategy should be used (Goel, Reference Goel1997).
Analysis of related work shows that in any attempt to apply mentioned approaches in free-form domain, one faces their limitations. The KBE approaches are highly effective for routine design with repetitive, noncreative design tasks. These approaches cannot be applied for creative design or the design of products that are highly subject to change (Stokes, Reference Stokes2001). In most of the CBR approaches to supporting design decisions, knowledge is represented as feature vector cases, and proposed reasoning mechanisms are suited for this type of knowledge representation. Only few approaches address the adaptation of the solution, implementing it only for feature vector cases. In all other CBR approaches, the adaptation is made by the users.
3. APPROACH
We share the belief that the analysis of free-form objects geometry has to rely upon semantic interpretation of their features, geometric and other. A survey conducted among CAD experts showed that during the analysis of geometry, they, first of all, recognize the semantics of a free-form object's features, or some of its elements features (Stojkovic et al., Reference Stojkovic, Manic, Trajanovic and Korunovic2007). When they were asked to describe the object in Figure 1 to other skilled CAD operators, they tried to describe it by associating with geometric features that CAD operators already know from CAD packages. They especially liked using CAD terms and methods that could create the shape in a way that is familiar based on a CAD software package they use: “it is a kind of sweep protrusion (for ProEngineer users), with a profile that looks like reverse channel of ‘U’ letter shape.”; “it is a kind of rib feature (for Catia users) with a profile that looks like reverse channel of ‘U’ letter shape.”
User-defined geometric feature.
In the context of free-form object design or digital reconstruction, during the semantic interpretation, a designer tries to find similarities between the current (and unpredicted) free-form object and one or more free-form objects that were analyzed earlier. Determination of similarity is followed by semantic categorization of the current free-form object. The final step is the conclusion that is usually the application of a solution (which geometric features should be used and how should they be composed) or parts of solutions for a similar free-form object. In our opinion, the greatest challenge of automation of digital reconstruction and free-form object design in general is closely related to automation of the semantic interpretation of geometric and other free-form object features.
A precondition for semantic interpretation of a free-form object's features is existence of its semantic model. The main expectation from the semantic model of a free-form object is to provide semantic content that could be interpreted in order to indicate to the designer what geometric features are more or less applicable for creation of a corresponding free-form object CAD model. In addition, semantic interpretation of the free-form object's features should help the designer to choose and compose appropriate design process sequences. The main challenges in semantic interpretation of the free-form object's features are their recognition and semantic categorization. Any semantic interpretation process starts with recognition of semantics, that is, the meaning of an object of interest, and ends with semantic categorization of that very same object. The semantic categorization is a term from cognitive psychology, used to express the process of assigning meaning to the (new) item (Deutch & Deutch, Reference Deutch and Deutch1963; Norman, Reference Norman1968). Within the semantic data models, this process is usually performed by creation of a semantic relation between the concepts (nodes) in a semantic network.
3.1. ASM
For the case of bioform digital reconstruction presented in this paper, ASM (Stojkovic et al., Reference Stojkovic, Manic, Trifunovic and Misic2011) was used for modeling semantic features of free-form objects. The ASM is a new semantic model, which was developed in-house, and it was primarily aimed to capture and interpret semantics of design features related to manufacturability issues (Stojkovic, Reference Stojkovic2011). The ASM intends to introduce an alternative approach of knowledge representation compared to existing semantic models by moving the focus of data structuring from concepts to semantic relations or associations (the term that is used in ASM). This idea of structuring the meaning in associations is chosen to support the thesis stating that the knowledge people have about items (e.g., visual representations, objects, situations, etc.) is contained in associations between concepts that abstractly represent those items (Anderson & Bower, Reference Anderson and Bower1973). Furthermore, the ASM is more flexible and productive in capturing and interpreting semantics of data compared to existing semantic models (Stojkovic et al., Reference Stojkovic, Manic, Trifunovic and Misic2011). Here, we will briefly explain ASM.
3.1.1. Structure
The structure of the ASM consists of the following:
• concepts,
• associations between concepts,
• concept bodies, and
• contexts.
ASM concepts are at the same time nodes of the ASM semantic network and abstract representations of objects, features, situations, and so forth. Concept data structures consists of only one parameter, name. Concept bodies are their realizations.
An ASM association structure is characterized by 11 parameters (Stojkovic et al., Reference Stojkovic, Manic, Trifunovic and Misic2011): names (cpt i, cpt j) of two concepts (or contexts) that are associated; topological parameters: roles (r i, r j) of concepts (i.e., type and subtype), type (t) of association (i.e., classifying), direction (d) of associating (←, ↔, →), and character (c) of associating (+, –); weight parameters: accuracy (h) of associating for given context (0; 0.25; 0.5; 0.75; 1) and significance (s) of associating for given context (0; 0.25; 0.5; 0.75; 1); and affiliation parameters: context id to which association belongs, and user id to identify who created the association (Fig. 2). There can be only one concept with a given name, but there can be many associations belonging to different contexts associating it with other concepts. Type of association declares the relationship between the two concepts, that is, the role of each concept in the association. In this way, values for parameters roles of concepts and type of association are coupled and can be considered as ordered triplets.
The active semantic model association structure: several associations with specified parameters belonging to a context.
The ASM also introduces contexts that are sets of semantically close associations. Context brings abstract meaning of a certain object, situation, or event, and therefore is semantically designated. General context is defined and built in the ASM structure independently from the user. So-called particular contexts require additional semantic description by the user. All associations from particular contexts are assigned (usually with different parameters) to the general context. The association plexus in the ASM is, in general, a context subset (mathematical structure) and can be considered without specific abstract meaning. This difference between the association plexus and the context has no effect on data processing and their use for semantic interpretation of the data in the ASM.
The ASM structure is not domain specific and can be used for representation of knowledge in any field.
3.1.2. Cognitive data processing
For semantic interpretation of data (cognitive data processing) a set of algorithms is used. There are three complex algorithms: for determination of similarity of associations (Trifunovic et al., Reference Trifunovic, Stojkovic, Trajanovic, Misic and Manic2013), for determination of similarity of contexts, and for upgrading the contexts (Trifunovic et al., Reference Trifunovic, Stojkovic, Trajanovic, Misic and Manic2014).
The first algorithm enables semantic categorization of new concepts by recognizing similarity of associations between new and known concepts in the semantic network. This procedure results in the creation of new associations between new and known concepts in the semantic network. The number of semantic relations that describe the new concept is increased in this way (“deepening” knowledge about the new concept).
The second algorithm enables detection of semantic similarity between different association plexuses or contexts that can be determined according to the similarity of associations that connect them (when they are treated as semantic network nodes) or their semantic content (i.e., their members and associations). This procedure results in the creation of new associations between new and known association plexuses or contexts, thus actually conducting semantic categorization of a new association plexus or context. The most common and probably most significant case of semantic content similarity between different association plexuses or contexts is called topological analogy (sameness; Fig. 3 left). Topologically analogous association plexuses or contexts have the same type of topology (combination of appropriate values of topological parameters of associations) and the same structure. Associations belonging to two different association plexuses or contexts that have similar values of weight parameters and the same values of topological parameters are called topologically correspondent associations (TCA; associations represented by the same type of line in Fig. 3 left). Concepts belonging to TCA-s of two different association plexuses or contexts, which have the same role in these TCA-s, are called topologically correspondent concepts (TCC). Two types of topologically analogous association plexuses or contexts are distinguished: semantically distant (association plexuses or contexts do not share concepts, nor are their concepts similar, synonyms, or connected over a series of up to four associations); and semantically close (association plexuses or contexts share one or more concepts, or have concepts which are similar, synonyms, or connected over a series of up to four associations). The importance of the ASM's ability to recognize topologically analogous association plexuses or contexts reflects that, on the basis of this ability, the ASM can recognize semantic similarity among
• known and unknown association plexuses or contexts,
• partially and more completely described association plexuses or contexts, and
• semantically distant association plexuses or contexts (thus creating the potential for creative responses to inputs).
Recognizing topological analogy between new association plexus (representing a new object or situation) and certain known association plexus (which is the subset of some context) in the ASM semantic network is a precondition to put into action the third algorithm. Regardless of whether they are semantically close or semantically distant, the ASM will upgrade a new association plexus modeled on the remainder of the context whose subset is recognized as a topologically analogous association plexus (Fig. 3 right). This procedure is called association plexus upgrading and is done through several attempts that will be explained later.
Topologically analogous (left) association plexuses (PLXX and PLXN) and (right) association plexus (PLXX) upgrading. Topologically correspondent associations are represented by the same type of line.
3.1.3. How does it work?
At the beginning of the semantic interpretation process the user creates the context (called communication context) in which he wants to communicate with the ASM; that is, he defines what he wants from ASM and in what circumstances. This is done in a simple and intuitive way through the communication context definition dialog (Fig. 4). Communication context is actually an association plexus (Fig. 5) whose nodes are elements of the semantic network (concepts, contexts, etc.) connected by predefined types of associations. By creating this association plexus, the user helps the ASM in conducting a parametric directed search of the semantic network in order to find similar communication contexts (Object and Product of communication context have significant impact on search guidance). The result of this search is a narrowed semantic network space that will be examined by the ASM in order to provide an answer to a question defined through the communication context.
Communication context definition dialog.
Communication context association plexus.
In the next step the user semantically describes the Object of the communication context (which is the object of interest for cognitive data processing). The semantic description is done by creating new concepts and associations between new and known concepts. The result of these activities is a new association plexus that semantically describes the communication context Object.
To run cognitive data processing and get the results (assessments, judgments, inferences, and decisions) from the ASM, the user needs a semantic description of the communication context Object, but also some knowledge to match it with. This knowledge is stored in the semantic network of the ASM. Thus, to get some relevant reaction and results from the ASM, some domain knowledge is necessary. The more the knowledge is embedded, the more the relevant reaction would be. This knowledge originates from other users from the same or a similar field and is added gradually.
The ASM provides an answer to a question defined through the communication context by recognizing a topological analogy between new and known association plexuses (from the narrowed semantic network space) and upgrading the new association plexus modeled on the remainder of the context (whose subset is recognized as a topologically analogous association plexus). The answer is being formulated through creating new associations between concepts from the new association plexus and known concepts in the network, which takes into account the objective and product of the semantic interpretation (Objective and Product of communication context).
Semantic interpretation of free-form object features with the ASM is described in Figure 6.
Semantic interpretation of free-form object features with the active semantic model (topologically correspondent associations are represented by the same type of line).
4. CASE STUDIES
Use of the ASM in free-form object reconstruction is demonstrated through a case based on a real situation where a patient was waiting for a partial reconstruction of his chest bone destroyed by a tumor (Stojkovic et al., Reference Stojkovic, Milovanovic, Vitkovic, Trajanovic, Grujovic and Milivojevic2010). A free-form designer was engaged to remodel the patient's chest bone, including the cartilage extensions of several ribs (Fig. 7). Two demands existed: to match the geometry of the patient's chest bone and to do it within a short deadline. Both of them indicated use of reverse modeling and rapid prototyping technologies. Because the bone implant was in question, it was obvious that the designer had to deal with free forms.
Reconstruction of chest bone (sternum): CT scan (left) and Sternum-Implant model (right).
Being the first time the designer had to perform reverse modeling of a chest bone, we wanted to see if the ASM could provide help or guidance in this complex task.
Using the communication context (Fig. 4) in this case, the designer expressed the situation where
/ Actor: “ASM” should perform …
/ Activity: “Cognitive-Data-Processing” over …
/ Object (focal concept of semantic interpretation): “Sternum-Implant” that belongs to or is related to …
/ Wider-context: “Free-Form-Design,” with
/ Objective (aim or motive) to perform: “Design-Procedure-Definition” and to obtain
/ Product: “Design-Procedure.”
The next step was to semantically describe the Sternum-Implant 3-D model, that is, the Object of the communication context (Fig. 8). There are no predefined concepts or association types used for representation of the design.
Chunk of knowledge about Sternum-Implant 3-D model.
The communication context whose Object is the Statuette 3-D model was found to be similar with the Sternum-Implant communication context (Objective and Product of communication contexts are the same). Semantic description of the Statuette 3-D model and its design procedure is presented in Figure 9. Fragments of association plexuses representing knowledge about the Sternum-Implant 3-D model and the Statuette 3-D model are topologically analogous and semantically close (Fig. 10). TCA of these two association plexuses are represented by the same type of line, while TCC are represented by the same background pattern.
Previously stored context with semantic description of Statuette 3-D model and its design procedure.
Recognized topologically analogous and semantically close association plexuses Subsets of Sternum-Implant-Design and Statuette-Design contexts. Topologically correspondent associations are represented by the same type of line, while topologically correspondent concepts are represented by the same background pattern.
The ASM tries to upgrade a new association plexus through several attempts. In the first attempt, the ASM recognizes semantically close TCC-s (TCC-s are identical, similar, synonyms, or connected over a series of up to four associations) of two association plexuses. Concepts Different-Cross-Sections and Spindle-Shaped are recognized as semantically close TCC-s because they are connected by an association of “similarity” type in the general context (this association could have been created earlier by the ASM based on recognition of similarity of associations). The next step is upgrading of a new association plexus through creating new association between concepts Different-Cross-Sections and Multi-Sections-Volume-Procedure whose parameters are identical to parameters of association between concepts Spindle-Shaped and Multi-Sections-Volume-Procedure (Fig. 11).
First attempt of upgrading new association plexus: creation of new association between concepts Different-Cross-Sections and Multi-Sections-Volume-Procedure.
In the second attempt, the ASM recognizes concepts in the semantic network similar to concepts of the new association plexus. Similar concepts are connected by association pairs over other concepts, and associations in each pair have the same topological parameters and similar weight parameters (Fig. 12 left). Concepts Sternum-Extension and Door-Handle are recognized as similar in general context because they are associating the same concepts (Spatial-Curved and Similar-Cross-Sections) in the same way (they are connected by associations pairs over concepts Spatial-Curved and Similar-Cross-Sections, and associations in each pair have the same topological parameters and similar weight parameters; Fig. 12 right). Because there exists association between concepts Door-Handle and Sweep-Procedure in the general context (Fig. 12 right), which is topologically correspondent to association between concepts Statuette-Arm and Multi-Sections-Volume-Procedure, new a association plexus will be upgraded through creating a new association between concepts Sternum-Extension and Sweep-Procedure that will have the same parameters as the association between concepts Door-Handle and Sweep-Procedure (Fig. 13).
(Left) Similar concepts are connected by association pairs over other concepts, and associations in each pair have the same topological parameters and similar weight parameters. (Right) Concepts Sternum-Extension and Door-Handle are recognized as similar.
Second attempt of upgrading new association plexus: creation of new association between concepts Sternum-Extension and Sweep-Procedure.
The new association plexus now has one new pair of semantically close TCC-s (concept Multi-Sections-Volume-Procedure) and one new pair of semantically distant TCC-s (concepts Sweep-Procedure and Multi-Sections-Volume-Procedure). Instead of further upgrading of the new association plexus through creating new associations between concept Multi-Sections-Volume-Procedure and concepts Statuette-Body, Statuette-Arm, and Statuette-Design-Procedure, the ASM will, taking into account the topology of the new association plexus, upgrade it through creating new associations between concept Multi-Sections-Volume-Procedure and concepts Sternum-Body, Sternum-Extension, and Sternum-Implant-Design-Procedure (Fig. 14). There are two answers for the question “Which design procedure should be used for sternum extension?”: Multi-Section-Volume-Procedure and Sweep-Procedure. The first answer is based only on the analysis of topology, while the second answer is based on the analysis of topology and association parameters, which makes it more accurate. At this point, the user can decide which design procedure he wants to use (it is assumed that Sweep-Procedure was chosen, which means that the association between concepts Multi-Sections-Volume-Procedure and Sternum-Extension was rejected by the user). Now the ASM recognizes the association between concepts Sweep-Procedure and Door-Handle-Design-Procedure that is topologically correspondent to the association between concepts Multi-Sections-Volume-Procedure and Statuette-Design-Procedure, but will, taking into account the topology of the new association plexus, upgrade it through creating new associations between concepts Sweep-Procedure and Sternum-Implant-Design-Procedure (Fig. 15).
Further upgrading: there are two answers for the question “Which design procedure should be used for sternum extension?”: Multi-Section-Volume-Procedure and Sweep-Procedure. The user rejects answer Multi-Section-Volume-Procedure.
The new association plexus is finally upgraded through creating new associations between concepts Sweep-Procedure and Sternum-Implant-Design-Procedure.
The answer to a question defined through the communication context is that the Sternum-Implant body should be designed using Multi-Sections-Volume-Procedure, while Sternum-Implant Extension should be designed using Sweep-Procedure (Fig. 16).
The active structure model provides an answer to a question defined through the communication context: Sternum-Implant Body should be designed using Multi-Sections-Volume-Procedure, while Sternum-Implant Extension should be designed using Sweep-Procedure.
4.1. Implementation
The AcSeMod web application, implementing the ASM structure and accompanying cognitive data processing algorithms, has been developed for testing purposes and dissemination. The web application was developed using Java programming language, precisely Java applet technologies that allowed the accessibility over the web and availability of all graphical user interface elements in Java. The Java Universal Network/Graph Framework software library was used for the visualization purposes (representation of the concepts and associations of the semantic network). This software library provides procedures and tools for the modeling, analysis, and visualization of data that can be represented as a graph or network. Apache Tomcat v6 was used as the application server. Associations and other elements of the ASM structure are stored using the MySQL Community Server v5 relational database management system. Cognitive data processing algorithms are implemented to the greatest extent on the database level through stored procedures and views. Parts of the cognitive data processing algorithms, mainly those whose implementation depends on the structure of the input data, are implemented on the program level.
AcSeMod enables users to create new contexts, concepts, and associations between concepts. Associations are represented graphically with lines, and because their structure is complex, values of some association parameters are also represented graphically (Fig. 17). Accuracy is represented by different shades of gray (from white for accuracy = 0 to black for accuracy = 1). Significance is represented by different thicknesses (minimum thickness for significance = 0, maximum thickness for significance = 1). Direction is represented by arrows, with the exception for the direction “from left to right, and from right to left,” which does not have arrows. Associations with negative character are represented by a red line. Type of association is represented by the phrase displayed when the pointer is positioned over the association (Fig. 18). Figure 19 shows a new problem situation (left) and one of the retrieved compatible problem situations (right) in AcSeMod.
Graphical representation of the association parameters (from left to right: accuracy, significance, direction, and character).
Representation of the type of association.
(Left) New problem situation and (right) one of the retrieved compatible problem situations. A recognized topological analogy between (left) the new and (right) the existing association plexus is also shown.
5. DISCUSSION
The ASM brings a fresh approach to sharing and reusing knowledge. Inspired by human memory processes, the ASM uses associations between concepts to represent knowledge. Systems that rely on knowledge only in the form of requirements, restrictions, rules, and recommendations cannot use a wide spectrum of information left that is not possible to formulate in this way. The ASM tends to make use of all information available. In the free-forms domain, this could be crucial, because there are not many rules that can be formed and applied when working with unique and unrepeatable forms.
This semantic model can provide support to the designer facing challenges of free-forms digital reconstruction, so he is not forced to rely only upon his own experience. The ASM network provides an insight to other's knowledge and experience and allows the designer to draw from it. Of course, this is gaining in importance with young and less experienced designers.
The ASM structure is not domain specific and can be used for knowledge representation in diverse fields. The knowledge from a specific domain is represented through particular context(s). Semantic relations between contexts allow knowledge from one context to be applicable to others. ASM is also capable of using general knowledge. This type of knowledge is represented through a general context. All associations from particular contexts are assigned (usually with different parameters) to a general context. In this way, the general context acts as a layer linking all the particular contexts and gathering all their associations, which enables assessment of similarity between concepts from different domains. The user can also assign associations to the general context directly. Problem situations (cases) in the ASM are represented as plexuses (sets of associations between concepts) and correspond to structured case representation in CBR systems.
The novelty of the ASM method is the communication context through which the user expresses what he wants from the ASM and in what circumstances. This information is used for the reduction of the semantic network search space. During the retrieval of the compatible problem situations, structural similarity of association plexuses and semantic similarity of their corresponding concepts are assessed simultaneously, while in some CBR approaches, this is done sequentially. This process is based on an in-house developed algorithm for graph matching adapted to the ASM structure. Retrieved compatible problem situations (association plexuses) are topologically analogous to new association plexus, while as many of their topologically correspondent concepts as possible are semantically close (same concepts, or concepts that are similar, synonyms, or connected over a series of up to four associations). This semantic closeness is determined by analyzing the similarity of associations between concepts, which stresses the importance of knowledge being contained in associations. The user decides which of the retrieved topologically analogous association plexuses will be used for the upgrading procedure. This decision is aided with the information about semantic closeness of every recognized topologically analogous association plexus. The concept of using similarities, and not just equivalence between the associations and contexts, is the ASM's essential feature: the ability to react in a semantically relevant way in cases where there is no predicted input (predicted free-form object).
The problem of adapting the solution is strongly addressed in the ASM. Upgrading of the new association plexus, that is, solution adaptation, is performed semiautomatically (new associations are proposed for the user to accept or reject) based on the remainder of the context whose subset is the topologically analogous association plexus selected by user. The selected association plexus can be semantically close (CBR) or semantically distant (ABR). The basic idea is to upgrade the new association plexus to a context that will be topologically analogous to the context whose subset is the selected topologically analogous association plexus. Topological parameters of associations for the adapted solution and their structure are known. The challenge is to find appropriate candidate concepts. In this stage, the ASM searches the whole semantic network in order to find appropriate associations, relying on or assessing semantic similarity between concepts. The upgrading procedure therefore can be based on a semantically close or semantically distant topologically analogous association plexus, while the adaptation of the solution is done by analyzing the reminder of the network, that is, all previous problem situations from all domains.
The ASM, as opposed to an ontologies inference engine, does not use predefined topologies during the semantic categorization of data. An ontologies inference engine uses set of rules containing an encoded first-order logic to create the adequate functional relation between the currently considered concept and other concepts in the semantic network for preplanned cases of subgraphs of functional relations. The ASM therefore provides greater freedom than similar models based on the Web Ontology Language. When an attempt is made to use ontologies for supporting reverse modeling of free-forms geometry, its deficiencies become noticeable. Impossibility to predict a finite number of potential free-form variants prevents, and to a great extent, preparation of rules and program code needed to automate analysis of the shape and afterward to generate the reaction of an ontology in order to provide support for the designer.
Use of KBE systems brought profit to different fields of industry, enabling cost and time reduction in the product development phase (Chapman & Pinfold, Reference Chapman and Pinfold2001; Kulon et al., Reference Kulon, Mynors and Broomhead2006; Corallo et al., Reference Corallo, Laubacher, Margherita and Turrisi2009). This system represents a very promising technology and has significantly evolved during the past decade. KBE is probably the best choice when the design in question is highly rule driven, multidisciplinary, repetitive, and demands geometry manipulation and product configuration (La Rocca, Reference La Rocca2012). However, design and digital reconstruction of bioforms do not fit into this task profile. First experiences give a reason for optimism that use of the ASM in these situations could lead to results similar to ones achieved by KBE.
KBE is fully connected with CAD software, while AcSeMod is person oriented. It provides guidelines and helps the engineers to increase efficiency of their work. Creating a semantic model of the object of interest in AcSeMod is easy and intuitive, so professionals of different profiles can do it. In the future, there's the possibility of developing a kind of translator able to convert STEP or native CAD model features into an ASM-compatible structure (graph, i.e., segment of semantic network).
The ASM, as any new model, has its shortcomings. Relying as well on similarity, and not only on equivalence, could cause not so accurate assessments and judgments. This will consequently lead to not so correct inferences and decisions. In addition, the ASM's “intelligence” and the meaningfulness of its reactions depend on the amount of acquired knowledge and on the correctness of the acquired knowledge itself. One part of the problem will actually solve itself spontaneously. The sole use of the ASM results in widening or deepening of its knowledge. With a continuously growing knowledge base, there's no fear we will run out of information. In regard to correctness of the knowledge embedded in the ASM network, this is left in the trust of users.
6. CONCLUSION
In this paper, the use of application, implementing the structure of a new semantic model and the accompanying cognitive data processing algorithms, for supporting design decisions during digital reconstruction of free-form objects has been presented. The structure of the semantic model, called ASM, enables representation of knowledge from different domains. Because knowledge is contained in associations between concepts, one of the implemented cognitive data processing algorithms enables semantic categorization of new concepts by recognizing similarity of associations between new and known concepts in the semantic network. This allows highly efficient semantic categorization of new concepts, which does not depend on preplanned inputs and predefined inference rules. Another two cognitive data processing algorithms enable ABR with a semiautomatic solution adaptation based on the analysis of the whole semantic network. Therefore, this application can be categorized as a general-purpose knowledge-based system with an implemented ABR mechanism. The application is particularly suited for responding to unpredicted inputs, which is why we decided to test it in the free-form domain.
Application of the ASM for modeling the meaning of free-form geometric elements has shown that it is possible to automate the semantic interpretation of unique, unrepeatable, and unpredictable forms, based on semantically close analogies. In this way, it opens the possibility for automation of the selecting and composing of geometric features for efficient creation and/or reconstruction of geometry of free forms, and finally, helps the designer to decide “which way to go” during the process of digital reconstruction of the free forms.
Above all, the possibility of the automatic semantic interpretation of objects geometric features opens up new directions for the development of CAD and all other CAx applications. The first results of ASM application show that proposed semantic data structure and its accompanied algorithms of cognitive data processing have the potential to turn into a new information technology paradigm: associatively oriented data model.
ACKNOWLEDGMENTS
The paper represents a summary about a part of the research that is conducted within the project “Virtual Human Osteoarticular System and Its Application in Preclinical and Clinical Practice” (Project III 41017), which was funded by the Ministry of Education, Science and Technological Development of the Republic of Serbia from 2011 to 2014.
Milan Trifunovic is a Teaching and Research Assistant in the Department for Production, IT and Management, Faculty of Mechanical Engineering, University of Nis. He graduated with a degree in mechanical engineering from the same university. His areas of research are the application of information technologies in manufacturing, artificial intelligence, and knowledge-based engineering systems (semantic networks and cognitive data processing models for CAD/CAM systems). He is the author of approximately 14 international scientific publications, 3 of them in international journals.
Milos Stojkovic is an Assistant Professor in the Department for Production, IT and Management, Faculty of Mechanical Engineering, University of Nis. He received a degree in mechanical engineering and his PhD from the same university. His areas of research are knowledge-based engineering systems (R&D in semantic networks and cognitive data processing models for CAD/CAM systems) and bioengineering (R&D in customized tissue scaffold design and fabrication and R&D in CAD methods for reverse modeling of the human bones). He is the author of approximately 36 international scientific publications, 9 of them in international journals.
Miroslav Trajanovic is a Professor in the Department for Production, IT and Management, Faculty of Mechanical Engineering, University of Nis. He received a degree in mechanical engineering and his PhD from the same university. His areas of research are the application of information technologies in manufacturing, rapid prototyping technologies and reverse engineering, and simulation of the product's behavior in the conditions of exploitation. He is the author of approximately 44 international scientific publications, 13 of them in international journals.
Miodrag Manic is a Professor in the Department for Production, IT and Management, Faculty of Mechanical Engineering, University of Nis. He received a degree in mechanical engineering and his PhD from the same university. His areas of research are the application of information technologies, including methods of artificial intelligence, computer numerically controlled machine tools and their programming, and technology and manufacturing systems. He is the author of approximately 35 international scientific publications, 9 of them in international journals.
Dragan Misic is an Assistant Professor in the Department for Production, IT and Management, Faculty of Mechanical Engineering, University of Nis. He received a degree in mechanical engineering and his PhD from the same university. His areas of research are application of information technologies in manufacturing, simultaneous design of technological processes and integration of CAD/computer-aided process planning/CAM systems. He is the author of approximately 21 international scientific publications, 5 of them in international journals.
Nikola Vitkovic is a Teaching and Research Assistant in the Department for Production, IT and Management, Faculty of Mechanical Engineering, University of Nis. He graduated with a degree in mechanical engineering from the same university. His areas of research are the application of reverse engineering techniques in information and manufacturing technologies and medicine, development of software applications using modern information technologies, artificial intelligence and its application, and free-form surface modeling and applied mathematics. He is the author of approximately 18 international scientific publications, 6 of them in international journals.
