The team at TUM’s (Technical University of Munich) Learning Systems and Robotics Lab has developed a humble-looking robot that resembles a stick on wheels with a camera on top. However, don’t let the looks fool you. It might be one of the most useful robots designed for everyday people.
Led by Prof. Angela Schoellig, the team has built a robot that can find lost items by creating and analyzing a spatial map of its surroundings. Next time you can’t find your keys or glasses, you won’t lose your sanity, as this robot will find them for you.
How does the robot find things?
The camera provides two-dimensional images, but those pixels also contain depth information. The robot uses this to build a 3D map of its surroundings, accurate to the centimeter, and constantly updates it as things change.
One challenge with this approach is that objects are constantly moved or replaced, which quickly makes the map outdated. As a result, the robot has to rescan the entire area. To solve this problem, the researchers used an LLM-powered model not only to map the environment but also to maintain and update the data.
It tracks objects and assigns a relevant score. It then uses the score, the time since the object was last seen, and other data points to create a probabilistic model to decide which areas to scan and maintain.
What makes it genuinely clever is the layer of internet knowledge baked into it. The robot understands that glasses are likely to be left on a table or windowsill, not on a stovetop or in the sink.
A language model then translates this real-world reasoning into search probabilities, helping the robot focus on areas where the missing object is most likely to be. As a result, the robot searches nearly 30% more efficiently than when scanning rooms at random.
What’s in store for the future of this robot?
Right now, the robot is limited to open spaces. The next challenge the team is tackling is teaching it to open drawers and cupboards, so it can search in closed spaces.
It’s still early days, but a robot that genuinely understands your home and helps you find things in it feels more useful as a home robot than other AI robotic projects we have seen in the past.