Each robot vacuum we consider for recommendation gets put through its paces in our test lab in Louisville, Kentucky. In addition to test floors where we run our controlled pickup tests, we monitor each robot vacuum in a special test room filled with mock furniture to gauge how well it navigates around common obstacles. Past that, we check each robot vacuum’s ability to gobble up pet hair without getting clogged or leaving loose strands behind, we consider mopping capabilities, and we check to see how well it navigates fake dog messes too.
Let’s dive a little deeper into the main considerations, starting with our performance tests.
Robot vacuum pickup power
When it comes to vacuuming prowess, we want to know how effective each robot is against common crumbs and other debris, and also how it fares against much smaller particles like dust, dirt and sand. To find out, we use sand as an analog for finer particles.
In each case, we scatter a controlled amount across three test floors: low-pile carpet, midpile carpet and hardwood floors. Low-pile carpet is shorter, less plush carpet with shorter fibers, so typically robot vacuums have an easier time picking up from it (though not always). Midpile is softer, more plush carpet with taller fibers. It tends to be more challenging for robot vacuums (though again, not always). Then, we take the robot vacuum, thoroughly empty its dust bin, send it to clean the affected area and finally measure the weight of whatever it managed to pick up. That gives us a pickup percentage of the full amount. From there, we repeat each run two more times and average the results.
In recent months, we eliminated our test for black rice on hardwood floors since, more or less, every robot vacuum we tested was scoring near 100%. We now use the sand test as our primary benchmark in evaluating cleaning performance. We consider anything 50% and above to be a good score for sand.
Robot vacuum navigation skills
Your robot vacuum will only clean your home as thoroughly as it’s capable of navigating it. The ideal cleaner will make easy work of finding its way from room to room and automatically avoiding obstacles along the way, all of which makes for proper, low-maintenance automated cleaning.
We make sure to observe each robot vacuum as it cleans to get a good sense of how well it navigates but to get the best comparison from cleaner to cleaner, we take overhead long exposure shots of each one as it cleans our darkened test room, with glow sticks attached to the top of each one directly above the vacuum intake. The images that result show us light trails that reveal the robot’s path as it navigates the room and cleans around our mock furniture.
Below is an example from the Ecovacs Deebot T30S, our best overall. It offers superb area coverage and it carried out navigation in a very organized and efficient manner. It got a 10 out of 10 in the navigation score, only taking an average of 15 minutes to complete a full-cleaning cycle.
In contrast, we have a robot vacuum with bad navigation, the Noesis Florio. It missed several spots in the room, and the light path test image contains a couple of brighter spots, potentially indicating the robot vacuum spent time spinning in place. What’s notable is the very disorganized navigation pattern. All this resulted in poor navigation scores.
In large part, it comes down to the tech at play. Over the years, we’ve consistently noted that robot vacuums that use laser-guided lidar navigation tend to be very good at mapping their environment and finding their way around. Meanwhile, 3D-mapping cameras with object recognition smarts can give robot vacuums the extra ability to identify and adapt to obstacles in their path.
Average runtime
To measure robot vacuum runtime, we run the vacuum in our test room three times and take the average of the times it takes to complete its cleaning cycle. To be clear, we do not run the vacuum until it’s drained, since unlike cordless vacuums, a robot vacuum will stop when it believes cleaning has been completed. Generally, we consider anything below 20 minutes to be a fast runtime.
Robot vacuum mopping testing
We test mopping on all the robot vacuums that have the functionality. We run the mopping test at least two or three times and evaluate its ability to remove sticky messes from different types of flooring. We’re currently in the process of formalizing our testing procedures in a quantifiable manner, but as of this writing our evaluation of mopping capabilities is anecdotal.
