BMW has been at the forefront of Industry 4.0 for years. For example, the company was an early adopter of additive manufacturing, and today prints hundreds of thousands of production parts annually. The German automaker is also using data analytics for numerous applications, including predictive maintenance, control of paint and press lines, and monitoring of fastener installation.
Now, BMW is applying artificial intelligence (AI) throughout its global operations. Most recently, BMW developed software that makes light work of creating AI applications for object recognition. The core of the software is the so-called BMW Labeling Tool Lite. This tool enables users to easily label objects in images, off-line, to quickly create AI apps that can then reliably identify those objects in images during production.
The software was influenced by practical experience from the BMW Group’s production network. “Smart AI solutions make the day-to-day work of our associates noticeably easier,” says Michele Melchiorre, senior vice president of production system, planning, tool and plant engineering. “Users need no longer worry about the underlying technology. The logic is the same as with a good smartphone app: Easy to install, quick to understand, use it of your own accord. That’s the only way a solution will quickly attract widespread use and become more effective.”
Building an object-recognition app, even with no AI expertise, is quick and straightforward, without having to program any software. No special hardware or additional software is needed; a standard powerful PC is enough.
To train an app that suits their needs, production associates start by taking photos and labelling objects within them. The software then optimizes itself independently and can distinguish between “right” and “wrong” parts after just a few hours, having worked through the labels.
Once trained, the app can assess live images during production. It instantly compares them to its learned knowledge, and recognizes quickly and reliably whether the right parts have been used.
At BMW’s assembly plant in Munich, the app is already being used to inspect door sill strips for the BMW 3 Series sedan. Ten different strips are available for the sedan, depending on the model. Thanks to AI, the app can accurately identify each strip and determine if the correct part has been installed.
“With our software package, it takes just a few hours to build apps for comparing actual and target status,” says Jimmy Nassif, head of IT planning systems in logistics at BMW.
Inspection isn’t the only application for the AI object-recognition software. BMW is also using the software to teach automated guided vehicles to recognize machines and other potential obstacles in an assembly plant.
The published algorithms are freely available for software developers worldwide to use, view modify and develop the source code further. These developments will also benefit the BMW Group. Indeed, the first version of the software was published in fall 2019.
“The wealth of feedback on the algorithms we released in 2019 was overwhelming. Our BMW AI community is delighted with the appreciation we got from around the world,” says Kai Demtroeder, head of data transformation and artificial intelligence at BMW Group. “We are seeing useful enhancements based on our source code. That prompted us to publish more algorithms, to help open up AI for mainstream users.”
With this latest release, the BMW Group is offering a complete solution for AI-based object recognition. Users who value high system stability will appreciate the additional functions such as fail-over and load balancing, which have now been added to the object-recognition interfaces.
Open-Source Data Set for AI Applications
Building on its success with AI, BMW has published the world’s largest data set to streamline and significantly accelerate AI applications in manufacturing. SORDI (Synthetic Object Recognition Dataset for Industries) consists of more than 800,000 photorealistic images. They are divided into 80 categories of production resources, from pallets and pallet cages to forklifts, and include objects of particular relevance to automotive engineering and logistics.
“The BMW Group has been using artificial intelligence since 2019,” says Melchiorre. “AI has already been utilized in various quality assurance applications in production at [our] plants. SORDI makes AI models much faster to train and AI considerably more cost-efficient in production.”
According to Melchiorre, the visual data is of particularly high quality. Integrated digital labels enable basic image processing tasks to be carried out, such as classification, object detection or segmentation for relevant areas of production. BMW’s partners in the project were Idealworks, Microsoft Corp. and Nvidia Corp.
To create the data without a lot of manual labor, a simulated environment for robotics, a digital twin of the production system and an AI training environment were all fused within the Nvidia Omniverse. The rendering pipeline from the BMW Tech Office allows any number of photos, including labels, to be synthesized in photorealistic high-definition quality and used in the creation of highly robust AI models.
SORDI can be used by IT professionals to develop and tailor AI systems for manufacturing, and by production employees to maintain mature AI systems for validation purposes ready for the start of production.
Virtual Factory Planning
BMW and NVIDIA are working together on more than just AI. The two companies are also cooperating to develop a new approach to planning highly complex manufacturing systems—the Omniverse platform. NVIDIA’s virtual factory planning tool integrates a range of planning data and applications and allows real-time collaboration with unrestricted compatibility.
“Together, we’re about to make a huge leap forward and open up completely new perspectives in the field of virtual, digital planning,” says Milan Nedeljković, BMW board member for production. “In the future a virtual representation of our production network will allow us to realize an innovative, integrated approach to our planning processes. Omniverse greatly enhances the precision, speed and consequently the efficiency of our planning processes.”
“BMW does personalized manufacturing at a massive scale—its operations are among the most complex in the world,” adds Jensen Huang, founder and CEO of NVIDIA. “In their vision of future factories, people and robots work together; engineers from all aspects of factory design collaborate in a shared virtual space; and the entire factory is simulated with photorealistic detail. NVIDIA Omniverse was built to realize this future. Omniverse will enable BMW to connect its teams to design, plan and operate future factories virtually before anything is built in the physical world.”
Virtual factory planning is not new. Until now, however, it has required data to be imported from various applications. This is not only time-consuming but also raises compatibility issues. In addition, the data is not always up to date. In the future, the Omniverse platform will enable live data to be collected and collated from all databases to create a joint simulation, eliminating the need to reimport data. The goal is to enable modifications and adjustments to be assessed in the early stages of planning to create an overall view.
This transparency will allow engineers to plan highly complex production systems quickly and accurately, without interface losses or compatibility problems. Omniverse integrates data from various professional design and planning tools from a range of different producers and uses it to generate photorealistic real-time simulations in a single collaborative setting.
The latter aspect is no small thing for a company that runs assembly plants in Germany, Austria, the U.S., Mexico, South Africa, India, China and Brazil. Employees at different sites in different time zones can access the virtual simulation and work together to plan and optimize details of a process or production system whenever they need to.
In addition, Omniverse can be used in a multitude of applications. In the future, engineers will collaborate using real-time data that is synchronized in the Omniverse cloud infrastructure. They will also be able to discuss the integration of new production systems with suppliers. Omniverse allows structure and facility data to be integrated, as well as items and part numbers of materials in production. It can also be used for logistics planning, offering unprecedented data consistency, from planning through to production.
“This is redefining collaboration,” says Nedeljković. “The ability to carry out changes live will speed up decision-making noticeably. Production planners will be able to visualize the entire planning lifecycle for every plant in the global production network. This will be supported by a wide range of AI-capable application cases, from autonomous robotics to predictive maintenance and data analysis.”
