Doing the Dishes

"One of the marks of a true science enthusiast", my thesis advisor once said, "is what you think about in your spare time". For example, in the shower, while driving or before going to sleep. I like thinking about vision, even (and especially) during mundane activities, such as doing the dishes. I enjoy asking - can a system endowed with state-of-the-art abilities deal with this specific task ?

Indeed the challenge inherent in computer vision can be expressed by pitting today's abilities against seemingly simple activities, such as doing the dishes :

Robots today are hardly able to fold towels. Not that this is not impressive, but the above task is probably harder : it includes dealing with a good deal of specularity, occlusion and clutter before the chaotic pile of dishes is perceived correctly. A good control system that would actually enable the robot to grasp (physically) a fork, separating it from the rest, is needed too.

Another challenge is doing dishes you never saw before (hopefully this won't happen too many times) - this would require an ability for generalization or smart modeling of what a (for example) fork actually is.

For those of us lazy enough to hope that one day soon there will be a robot doing our dishes (even putting them in the dishwasher is no small challenge) - I advise some patience.