More versatile robots with gen AI

Source: NVIDIA video. Screen grab of Tallu speaking from a video of a special address ahead of CES.

NVIDIA and its partners are bringing generative AI and robotics together very successfully. Agility Robotics, NTT, and others are including generative AI in their robots, while robot vacuum cleaners from Dreame Technology are being trained in simulated living spaces created by generative AI models. In a similar vein, Electric Sheep is building a virtual world to train robots for autonomous lawn mowing.

The trend was shared by NVIDIA VP of Robotics and Edge Computing Deepu Talla. Speaking as part of a special address ahead of CES, he called the introduction of ChatGPT the "iPhone moment of AI", alluding to the fact that smartphones took off after Apple introduced easily available software together with rich-featured iPhones.

“Autonomous robots powered by artificial intelligence are being increasingly utilised for improving efficiency, decreasing costs and tackling labour shortages,” he said.

NVIDIA's robotics partners, which include Boston Dynamics, Collaborative Robotics, Covariant, Sanctuary AI, and Unitree Robotics among others, are embracing GPU-accelerated large language models (LLMs) to bring advanced levels of intelligence and adaptability to machines of all kinds, he said, with innovations expected in healthcare, construction, agriculture and retail this year. 

Traditionally, robots are custom-built for specific tasks, and cannot be repurposed easily. Today's LLMs and vision language models are changing things, Talla said, allowing robots to generalise accurately. He explained that LLMs will allow robots to understand and respond to human instructions more naturally, in addition to being able to learn continuously from humans, from each other and from their environment.

NVIDIA technologies such as the NVIDIA Isaac platform for building AI-powered robots, and the Jetson edge AI and robotics platform, are already deployed by over 1.2 million developers, as well as 10,000 customers and partners.