European industry is moving into the future, combining automation, artificial intelligence, Internet of Things (IoT), fundamentally looking to change the way it operates, evolving to “produce more and diverse with less” while addressing essential challenges such as product variabilities, lack of high-level autonomous reasoning and accurate adaptable control, and holistic efficient planning systems. But most producers face difficulties in deploying these technologies at scale. SMARTHANDLE project organized several use cases (see https://www.smarthandle-project.eu/use_cases), which will research technologies to address these needs and support European industry, by implementing intelligent agents providing dexterity for handling applications, AI-based reasoning enablers, Higher-level planning, and coordination mechanism for successful deployment in real-life use cases.
Home assistant robots, such as Vacuum-cleaning robots, have achieved acceptance and success due to their user-friendly interface, despite being limited to one task: cleaning. Meanwhile, industrial robots are highly versatile but require specialized expertise for programming and deployment. Among the challenges addressed in SMARTHANDLE there is a use case in the Automotive field, where a dual-arm robot will be utilized as a multipurpose tool for automating the disassembly of battery packs. To simplify the process of automating new tasks, the project will continue the development of a system called on-site interactive skill programming. This system enables programming to be conducted on the site of automation deployment, allowing the robot to use its collection of learned skills while the operator cooperates with it to select and parameterize the skills using a range of visual, sound-based, and tactile interactions.
To provide an example, let us consider the disassembly of a battery pack. In this case, a dual-arm robot would hold a screwdriver and use it to remove the necessary screws. To accomplish this, the robot would rely on artificial vision to identify the location of the screwdriver and the screws. This process would be encapsulated in an unscrew skill, and the robot and the operator would work together to determine the parameters required to perform the task on a particular part of the battery pack. To illustrate, an interaction sequence for programming the task could involve the following steps, among others:
• Robot: To learn how to grasp the screwdriver, the robot asks the operator to teach by teleoperation, moving the arm (where it has a mounted camera) until it detects the screwdriver. The robot projects on its screen what the camera is capturing.
• Operator: Moves the arm until the battery pack is shown on the robot screen and the robot confirms that it correctly detected it.
• Robot: Detect screws and ask the operator to indicate which screws should be removed.
In some configurations, the operator will tell the robot to store the trajectory exactly as he/she performs it, in others will ask to optimize the taught trajectories. And they continue the interaction until the disassembly process for the specific battery pack is programmed.
To summarize, the aim of on-site interactive skill programming is to facilitate the deployment of user-friendly and adaptable robots, and it involves two main stages. Initially, the robot is equipped with pre-programmed or acquired basic skills, which are typically created by specialists. Subsequently, operators collaborate with the robot to experiment with different combinations and settings of these skills, allowing the robot to learn and automate new tasks.