IBM Brings Augmented Reality, Robotics to Field Engineers

IBM Research announced a new “Smart Mobility” prototype that combines augmented reality with robotics to deliver expertise to field engineers.

Using a unique combination of augmented reality and robotics, IBM has developed a new system to deliver information and remote expertise to field engineers performing maintenance and repairs on critical equipment.

IBM Research scientists announced a new mobile maintenance, repair and operations (MRO) prototype which is designed to help manufacturers and companies supplying and maintaining high-value machinery in sectors such as aerospace, oil and gas and shipping.

Leave it to IBM, which has introduced augmented reality into the shopping experience via a mobile app, to bring the benefits of augmented reality to field engineers. The new mobile system, using the combination of augmented reality and robotics, will help field engineers accurately locate equipment, provide them with critical information and receive real-time visual support from supervising experts based remotely.

The project is the result of an IBM collaboration with the University of Sheffield Advanced Manufacturing Research Centre (AMRC). The AMRC is responsible for identifying, researching and resolving advanced manufacturing problems to help businesses become more competitive through the application of new techniques, technologies and processes. It is part of the High Value Manufacturing Catapult, a network of leading manufacturing research centers backed by the UK government.

“IBM's MRO prototype is an exciting addition to the innovative toolset used by the AMRC’s researchers and engineers,” said Rab Scott, head of the Virtual Reality and Modeling Group at AMRC, in a statement. “We hope to demonstrate its usefulness and versatility in a number of situations within the manufacturing arena.”

IBM said studies have shown that remote support is much more efficient if on-site and remote engineers can share a visual representation of the site workspace and the on-site engineer's actions. To date, this has been accomplished, and only in part, by on-site engineers using hand-held cameras, mounted head-gear or specialist glasses. IBM’s new MRO system provides the supervisor with complete visual independence and a more stable video image; on-site engineers are able to work with greater freedom or, in the case of those with specialist glasses, freed from the tiring need to re-focus their eyes, IBM said.

Up to now, MRO tasks typically involve an engineer visiting a site, finding the right machine, and making sure they have an appropriate task sheet. If they run into difficulty, they might call for help from a remote supervisor or product expert.

The new system enables a supervisor to monitor an engineer's progress towards the maintenance site, using GPS. Once on site, an engineer can use a smartphone and QR codes to locate and identify an asset and receive maintenance instructions. The smartphone uses augmented-reality technology to overlay points of interest over a plan of the site, which can include the location of other engineers, first aid stations and health and safety apparatus.

If assistance is needed, a remote expert is able to view the on-site engineer’s workspace and support them with real-time video and audio links using a camera and a small projector mounted at the end of a remotely controlled robotic arm. The expert, from his management console, is also able to project a pointer and valuable information such as free-hand sketches, assembly instructions and CAD images directly onto the workspace or a nearby wall.

“The MRO prototype brings together two innovative IBM technologies, developed in our European research labs in Hursley and Haifa, into a single solution for our clients,” said Richard Lanyon-Hogg, IBM’s technical director for the industrial sector, in a statement. “It offers manufacturers the opportunity to lower their costs, provide just-in-time knowledge transfer and reduce the personal risk to engineers working in difficult environments.”

Moreover, the expert's ability to deliver information can also be further enhanced as the prototype is adapted and integrated with a company's specific MRO operations, asset management solutions such as IBM Maximo and Product Lifecycle Management systems. The ability to direct data to either the engineer's mobile phone or the robot's projector adds a further dimension to the system's portability and flexibility, IBM said.

Big Blue officials said the new system exemplifies how a new wave of mobile computing is revolutionizing how businesses can transform their processes. Data from existing back-end systems is increasingly being placed in the hands of front-line employees. Social media also is transforming the way employees are able to interact, by extending the reach of organizational expertise at minimal cost, IBM said.

A fully working IBM MRO system has been installed at the AMRC's Diamond Jubilee Knowledge Transfer Centre.