ARProMO: Augmented Reality for Production Optimization in SMEs through Intelligent Object Recognition

In AR-ProMO a generic and intelligent assistance system for industrial applications is being developed, which with the help of an augmented reality capable device ( for example a Tablet) visualizes data in the users’ physical surroundings, thereby helping the user comprehend highly complex cyber physical systems.

The following paragraphs present a summary of the important scientific and technical innovations in the context of this project.

Innovation 1: Development of intelligent object recognition backend to track components.

The aim here is to enhance classic image recognition algorithms in two ways. First, through machine learning techniques, specific parts of a machine or a process should be recognized. Secondly, the results should be shared with a knowledge base in order to increase the accuracy of classification, such that object recognition can be improved even in difficult lighting conditions and occlusion (Picture 1).

The example use case to demonstrate a proof of concept for this system is the assembly of a control cabinet, where various terminal blocks need to be assembled on profile rails. The system should, based on an existing order, assist the user by pointing out mistakes in component selection and arrangement.

Innovation 2: Development of AR-ProMO, a complete solution consisting of hardware and software for augmented reality applications in industrial contexts.

In order to give the user control over the work process, the system allows the user to evaluate his/her own performance during the evaluation process. Analysis mistakes, that is , terminal blocks which are not recognized or are wrongly classified, can be corrected by the user. These corrections are sent to the system backend where they are used to train the system further.

Along with the technical aspects, the ergonomic aspects have also been taken into consideration. Assembly work tends to be predominantly manual work, therefore the aim of the frontend has to be to support the user and reduce the physical and mental workload as far as possible. Hence the interaction method proposed uses an AR Tablet as an “augmented loupe” (Picture 2), which can be positioned at the required location as needed, leaving the user’s hands free for assembly tasks.