The Robot Captures An Image

First, the robot captures an image of its surroundings. A robot may use an RGB camera, depth camera, stereo cameras, LiDAR, or a combination of sensors.

Take a warehouse robot. A normal camera can show that there is a box in front of it, but the robot also needs to know how far away the box is. A depth camera provides that extra information.

A self-driving vehicle has a much harder job. Waymo's autonomous driving system uses cameras, LiDAR, and radar together. Cameras show what is around the vehicle. LiDAR maps the surroundings in 3D, while radar detects nearby objects and tracks their movement.

Robots often struggle with glass, hidden objects, shadows, and reflections. A 2026 study combined vision with camera movement and achieved a 96% success rate when grasping different types of objects.

The Image Is Processed

The captured image may be dark, noisy, blurred, or affected by shadows. The vision software processes the image to identify the objects in it.

Newer robots are also using event cameras. These cameras record changes in brightness instead of capturing complete frames. They are particularly useful for reducing motion blur when objects are moving and allow robots to work better in low-light conditions. Researchers have used them with robotic arms for grasping objects in difficult lighting.

Agricultural robots face strong sunlight and shadows. FieldNet, for example, is a real-time shadow-removal system developed for field robots. Reducing shadows helped improve weed detection.

The Robot Looks For Features

Next, the robot needs to identify the objects in the image.

The robot looks for useful features such as edges, colours, shapes, corners, and textures. For example, a robot sorting objects can use their shape, colour, size, and position to tell them apart.

The Robot Identifies The Object

Once the system finds these features, it can identify the object. A trained vision model compares the patterns in a new image with patterns it learned from many examples in its large database.

Latest vision models can also help robots recognise objects they have not seen before. Research such as DINOBot uses features from vision foundation models to help robots recognise and interact with unfamiliar objects.