The FLAIROP (Federated Learning for Robot Picking) research project was concluded at Festo in Esslingen-Berkheim. Specifically, the German Federal Ministry for Economic Affairs and Climate Action serves as the sponsor for the said project. At the event, all project participants – as well as interested public – were onsite or online presenting their results live from Canada.

Over the past two years, Festo has been conducting joint research with the “Karlsruher Institut für Technologie” (KIT) and partners from Canada (University of Waterloo, Darwin AI). Specifically, the research aims to make picking robots more intelligent using distributed AI methods. To do this, the partners investigated how robots can learn from each other without sharing their training data. This approach is called Federated Learning. Specifically, it allows the development of more robust and efficient AI than would be possible with data from just one robot without handing out sensitive company data.

“We are proud of our success in showing that robots can learn from each other without sharing sensitive data and company secrets. This protects our customers’ data. Also, we gain speed because the robots can take over many tasks faster this way. For example, the collaborative robots can help production workers with repetitive, heavy, and tiring tasks,” says Jan Seyler, Head of Advanced Development Analytics and Control.

“We have developed a universal, simulation-based data set that we can use to train autonomous gripping robots in such a way that they are able to reliably grasp items that they have not seen before,” explains Maximilian Gilles from KIT. The group plans further development of the Federated Learning System in the future. Accordingly, the platform enables different companies to train robot systems together without having to share data among themselves. This increases the acceptance of such systems in practice.

Federated Learning Revolutionizes Picking Through Robotics

Federated Learning is a machine learning technique to create privacy-preserving AI applications. Instead of sending the training data of the robotic arms in the picking cells to a central server to train the model there, the training takes place at many different locations. Subsequently, the locally trained models are sent to the central machine learning server. This way, the sensitive training data does not leave the data provider. Nevertheless, Federated Learning enables learning across data silos by aggregating distributed models. Ultimately, it enables highly accurate and data-driven prediction of object recognition and grasps point detection.

The robot arms in the picking cells have cameras to visually detect the items in front of them. Based on the camera image, the robot arms automatically recognize the different items and select a suitable gripping method. Due to the variety of items in a warehouse, this is a complicated task. Thus, it needs large amounts of data to achieve reasonable results. Creating such large amounts of data is time consuming. With data collected from picking cells in different organizations, it was possible to improve the grasping point detection of the cells. 

During the project, a total of five autonomous picking stations were set up for training the robots. Two robots are at the KIT Institute for Materials Handling and Logistics Systems (IFL) and three at Festo SE & Co. KG based in Esslingen am Neckar.

Project Completion: Now It’s Time for Practice

At the final event, Festo focused on the usability of the results. Jan Seyler, “We are showing which Festo products it can be incorporated into.” The group reading the publishing of research results for use by all interested parties in initial pilot projects.”