Hostname: page-component-7b7567c8c-2n9c9 Total loading time: 0 Render date: 2026-11-17T16:46:48.405Z Has data issue: false hasContentIssue false

FROM TEXT TO IMAGES: LINKING SYSTEM REQUIREMENTS TO IMAGES USING JOINT EMBEDDING

Published online by Cambridge University Press:  19 June 2023

Cheng Chen
Affiliation:
University of Georgia
Cody Carroll
Affiliation:
University of Georgia
Beshoy Morkos*
Affiliation:
University of Georgia
*
Morkos, Beshoy, University of Georgia, United States of America, bmorkos@uga.edu

Abstract

Core share and HTML view are not available for this content. However, as you have access to this content, a full PDF is available via the 'Save PDF' action button.

Smart manufacturing enterprises rely on adapting to rapid engineering changes while minimizing the generated risk. Making informed decisions related to engineering changes and managing risks against unexpected costs requires more information to be extracted from limited data. However, limited information in early-stage design can come in many forms, namely text and images. The development of innovative design tools and processes to link multisource data together is essential to assist designers in building model-based engineering (MBE) systems. However, the formal computational linking of multisource data is yet to be realized in MBE. We propose a framework to implement transfer learning and integrate domain specific knowledge to bridge this information gap. A synthetic dataset is created using web scraping techniques based on keywords extracted from the requirements. Requirement-image pairs are used to fine tune a contrastive language-image pretraining model to acquire domain knowledge. The results demonstrate how the content of images can be used to indicate all affected requirements for tracing engineering changes in a complex system.

Information

Type
Article
Creative Commons
Creative Common License - CCCreative Common License - BYCreative Common License - NCCreative Common License - ND
This is an Open Access article, distributed under the terms of the Creative Commons Attribution-NonCommercial-NoDerivatives licence (http://creativecommons.org/licenses/by-nc-nd/4.0/), which permits non-commercial re-use, distribution, and reproduction in any medium, provided the original work is unaltered and is properly cited. The written permission of Cambridge University Press must be obtained for commercial re-use or in order to create a derivative work.
Copyright
The Author(s), 2023. Published by Cambridge University Press