Toppan is developing an AI system that allows multiple robots to recognize people

Toppan is developing an AI system that allows multiple robots to recognize people

The new feature of TransBots™ enables AI-based person recognition. ©TOPPAN INC.

AI-based people recognition added to TransBots™ solution for centralized control of multiple types of robots. Pilot test carried out as part of the “CHANGE TOMORROW” event.

Tokyo – Toppan, a global leader in communications, security, packaging, decorative materials and electronics solutions, has developed TransBots™, a digital twin solution for linking physical spaces with virtual reality reconstructions in time real and centrally manage and control multiple types of robot services.

As a new function, Toppan has now developed an artificial intelligence (AI) system that enables multiple moving robots to identify people whose images have been uploaded in advance. This person recognition feature was verified through a pilot test in which images sent by robots to various locations in an event space were analyzed and compiled in the cloud using an AI engine. It was possible to identify specific people from camera images sent by different types of bots and perform cloud-based confirmation. The test also demonstrated that people’s locations could be tracked using robot positions on TransBots™.

Toppan aims to contribute to a fulfilling and sustainable society by using TransBots™ and other solutions promoting the application of digital twins in various fields, such as security, customer analytics, wayfinding, transportation and cleaning. . Going forward, Toppan intends to help alleviate labor shortages and transform working styles by strengthening collaboration with robot manufacturers, improving the multifunction console for service robots and TransBots™, and enabling multiple use cases.


An aging society and declining birth rate in Japan are contributing to labor shortages, which are becoming severe in security, cleaning, and other areas of the facilities management industry. Robots and IT are seen as part of the solution, and specialized robots for tasks such as reception and cleaning are introduced. Bot management, however, is complex because several different types need to be controlled.

Security cameras and other Internet of Things (IoT) devices are used for the management of buildings such as hospitals and complexes as well as for the management of admission to exhibitions and concerts. But limits to the range that can be monitored by people and security cameras mean robots are expected to play a bigger role in the future.

Toppan’s TransBots™ is a digital twin solution for centralized management and control of multiple types of robots. Operators can use a multifunction console to set up paths for robot movement in virtual space. The robots in real space then move according to the routes to arrive safely at their predefined destinations. Robots can also locate their own current position, detect obstacles, and speak to ensure the safety of nearby people. Using virtual space to define paths and control robots allows tasks to be carried out remotely.

Toppan has now added an AI-based person recognition feature to the TransBots™ console. When people pre-registered on the system enter a physical space, several moving robots are able to recognize them. This should have a variety of applications, including remote management of hospitals and resorts and management of large-scale events with restricted access, such as exhibitions and concerts. In addition to telepresence robots1, TransBots™ can also be applied to a wide range of service robots, such as carrier robots and mobile picking robots.

About the TransBots™ Multifunction Console

The TransBots™ multifunction console is part of the digital twin solution. It consolidates things like robot location, battery capacity, video from cameras, display images, and audio to the robotics HUB in the cloud and allows them to be controlled through various application consoles .
Main functions include Graphical User Interface (GUI) for twin robot digital control, robot map drawing, multi-vendor robot connection, automated/manual operation, web-based robot management dashboard, multiple telepresence management, live video streaming, equipment communication, remote centralization control of multiple locations, video storage and signaling functions.

Characteristics of the person recognition function

? Location identified using pre-recorded information
When multiple robots moving in a specified area recognize a person matching pre-recorded information, the location of the person can be identified from the positions of the robots.

?Images stored on a server
Images taken by multiple bots when a person is recognized can be stored in the cloud and displayed as a list on the dashboard.

?Cost reduction for robots
Since AI-based people recognition is performed in the cloud using video images sent by robots, there is no need to equip the robots themselves with state-of-the-art computers2 to provide AI functions, thus helping to reduce costs. Connecting to TransBots™ also allows robots of several types to take advantage of various functions.

Overview of the pilot test

Dates: October 29 and 30, 2022
Location: Toppan Booth3 inside Workshop Collection at CHANGE TOMORROW (Introducing an Interesting Future Right Ahead)4 (Location: Tokyo Portcity Takeshiba)
Objective: test the new AI-based people recognition function of TransBots™

?AI-based people recognition
To verify the person recognition function, recognition data was first created by recording the images of attendees when they joined the event. Based on the data, eight robots scattered around the site recorded images of the participants as they moved around. The images were then analyzed using AI to identify the people.
? Dashboard function
The dashboard function allows real-time control of the images taken by the robots. The images taken were used to assess how well each robot had been able to recognize people.
The dashboard was used in a game in which participants could see a large screen installed on the site to check which robots had seen them and when.
Result: The effectiveness of cloud-based people recognition using an AI engine was confirmed based on the more than 200 attendees who took part in the event.

Event inside the Toppan booth

Event Name: Halloween Event: Robot Halloween — Robomon Challenge
Target: kindergarten and primary school children and their guardians
Details: Participants wore Halloween masks and took part in a puzzle-solving game. While solving puzzles with their guardians, they received clues from three “Robomon” robot monsters in Halloween costumes who also tried to scare them.

1. Telepresence robot
A robot that someone can control remotely to simulate their own presence at the robot’s location.

2. Edge computer
A computer that not only compiles and processes data in the cloud via the Internet, but also processes data on user-side devices, such as robots, IoT devices, and smartphones.

3. Toppan Booth: Workshop Collection at Takeshiba: Discover the Cities and Work of the Future! (Japanese)

4. CHANGE TOMORROW (showcase of an interesting future just ahead)
An event held on October 29 and 30 at Tokyo Portcity Takeshiba (Minato City, Tokyo) (Japanese)

#Toppan #developing #system #multiple #robots #recognize #people

Leave a Comment

Your email address will not be published. Required fields are marked *