Let's shape the future together! Meet us in our offices in Germany, Switzerland and China for an inspiring talk.
Mixed Reality Showcase:
machine operation of the future using the HoloLens.
In this mixed reality project with our customer KBH Maschinenbau, we developed a trade fair showcase for the HoloLens to significantly improve the operator workflow for the CAP spray system.
The task was to optimally combine the innovative possibilities offered by the HoloLens in a scenario with KBH technology.
"This project also let us join our client in using the special features of MR to our advantage. The case offers significant added value for all machine operators thanks to new operating possibilities."
We kicked off this project with the KBH team with a collaborative scoping workshop. We used this to precisely look at the machine, get to grips with operators’ tasks and work processes, and join the KBH project team in evaluating potential ideas for the showcase scenario. To this end, the synthesis of KBH’s technical know-how and our experience in mixed reality, UX design and software engineering proved to be especially profitable. As well as exploring a suitable scenario, we also developed a storyboard during the workshop, which we used to visualize the scenario’s progression using scribbles.
The visual design was then based on a rough concept with wireframes and regular consultations with KBH. After an intense internal test phase, we carried out the final validation on the machines on site. This allowed us to ensure that special MR-specific design features, such as the placement of dialogs and the portrayal of colors through the HoloLens' additive color system, were duly taken into consideration. And finally, not only was the application handed over, but we also set it up on the KBH HoloLens so it was ready to use.
The showcase is made up of two scenarios. In the first case, the operator can use the HoloLens to change the position of the jets spraying stone within the machine. Currently, without the HoloLens, this position can only be changed via a manual remote control as the operator monitors the positions visually from inside the bottom of the machine. The HoloLens lets the operator easily position the jets from outside although these are hidden by the machine’s housing. After the new position has been set, the operator can also track the jets’ movements in real time via the HoloLens — it’s like an “X-ray” view through the machine.
The second part of the showcase consists of a maintenance scenario: the machine informs the operator of a blocked jet. Our application on the HoloLens uses specific instructions to guide them through the required steps: the machine must be opened, the affected jet located and replaced; the machine can then be properly closed and re-started. During each of these steps, the HoloLens wearer sees precisely how the work steps should be carried out — right down to the centimeter. Especially in the industrial context, this is one of the biggest benefits of mixed reality.
In this project, hardware and software form a perfectly unified unit. Sensors on machine parts, such as jets or the door to open the machine, constantly send data about their status to the HoloLens. Communication takes place via SPS: this isn’t just physically connected to the machine, but also constantly sends and receives impulses and status information. Data is transferred between SPS and the HoloLens via VisiWin, the machine controlling software provided by our partner Inosoft GmbH, and OPC-UA.
KBH Maschinenbau is part of Baustoffwerke Gebhart & Söhne GmbH & Co. KG, which boasts around 130 employees across three production sites in southern Germany. As a goal-oriented business, KBH plans, constructs and produces special systems for the finishing of concrete products, especially concrete paving tiles. What makes KBH unique in this industry is its combination of stone production and machine construction.