Abstract

3D Reconstruction, which is the process of creating 3D model of real objects, has long been one of the main interests of Computer Vision and Augmented Reality respectively. Current algorithms enable generation of sparse and dense point clouds and 3D mesh models by capturing their shape and appearance of real objects. However, mostly the outputs of 3D Reconstruction is only at geometric level without any semantic information.

The aim of Semantic 3D Reconstruction is to generate the 3D Model of a real object, while adding semantic information to the parts of the generated 3D Model. For example, generating a geometric model of a factory is considered as 3D Reconstruction. However, generating a geometric model of a factory while defining the parts (machines, pipes, etc.) in the factory is considered as Semantic 3D Reconstruction.

In addition to advancements in the topic of 3D Reconstruction, the improvements in computing power (CPUs & GPUs) and decrease in the size of those hardwares enabled mobile devices to handle more computationally expensive tasks day by day. Augmented Reality is one of them. Right now, there are lots of mobile devices that are designed for AR such as Microsoft Hololens or Magic Leap. Furthermore, mobile devices for personal usage can handle AR tasks as well. Although there are numerous other frameworks, mainly Apple's ARKit Framework on iOS Devices, and Google's ARCore Framework on Android Devices widened the applicability of AR.

Moreover, those advances in Augmented Reality and AR Devices helped industry a lot. Currently, AR is used at training new employees, remote expert support, guiding employees for assembly, maintenance and quality assurance. In addition to them, it is also used to visualize the information about the machinery, and the flow in the factory. Basically, someone without any knowledge of the factory, can use an AR Device to view the functions of the machinery, the inputs and outputs of the machinery, and how those inputs and outputs flow in the factory.

Combining the knowledge from the 3D Reconstruction with the current computing power of mobile devices, 3D Reconstruction can be performed on mobile devices. There are also various methods proposed to handle 3D Reconstruction on mobile devices. However, as stated before, those information will be on geometric level, without semantic information. In addition to that, with the usage of AR Frameworks for mobile devices, the information gathered from the 3D Reconstruction software can be visualized on mobile devices to inform user about the machinery and the flow of the industry.

In this thesis, it is proposed to develop a mobile software that generates semantic 3D Reconstruction of industrial environments. In order to perform Semantic 3D Reconstruction, the information from mobile Augmented Reality frameworks (ARKit) and user inputs will be used. Moreover, by connecting the flow diagrams of the industrial environments to their Semantic 3D Models, a 3D Flow Diagram with visual information will be created.

Results/Implementation/Project Description

Conclusion

[ PDF (optional) ] 

[ Slides Kickoff/Final (optional)]