We built a robot butler (but don't throw out the ironing board just yet)

The idea of a robotic servant is a lot older than you probably realise. It doesn’t just go back to the 1960s cartoon series The Jetsons, whose Rosie the robot could prepare meals, clean the house, and solve unexpected troubles. As early as the 3rd century BC, the ancient Greek scientist Philo of Byzantium built an iconic human-like robot maid that could pour wine when a cup was placed in its hand.

More recently we have seen vacuum cleaner robots, donkey/horse-like robots for carrying heavy loads, and even robots managing hotels. But as scientists, we are always driven by futuristic ideas. We have dreamed of intelligent machines that could help us with all our unbearable activities. After all, who likes to clean the house, wash the dishes or do the laundry? As part of a European project named CloPeMa, my colleagues and I have attempted to make the dream of a domestic robot real by developing a machine capable of understanding and manipulating clothes.

Handling clothes without ripping them is something we all do effortlessly without even thinking about it. But it’s a much more difficult task for a robot since clothes are deformable and flexible objects that change their state from the very moment they are picked up.

We do not notice the sheer amount of dexterity and sensing capacity that is required to fold garments. Try to imagine folding a towel without knowing what it looks and feels like and with your arms in plaster casts. This would be a very challenging and demanding venture for us humans, let alone a computer controlled robot.

The aim of the CloPeMa project was to develop computer algorithms that would enable a robot to perceive and understand clothes so that it could sort and flatten them. We started with a dual-arm industrial robot and gave it a pair of actuated robotic eyes, developed by my colleague Paul Siebert’s team, that allow the robot to see clothing and working area in three dimensions. The robot is then able to look around its working area and detect shapes, surfaces and edges in order to understand what it is in front of it.

The robot’s eye and its viewpoint while figuring out what to do next

We then programmed the robot to use these visual observations to create a computer model of the clothing including its size, material type and whether it has any wrinkles. The robot can also tell the difference between different categories of clothes such as jeans, shirts, sweaters and t-shirts. Its two hands (a pair of pliers to be accurate) then give it the dexterity to pick up, flatten, fold and unfold clothes.

The problem is the computational power and time required to turn the sensed information into actions. Sorting garments takes four minutes per item (video above), folding a t-shirt takes five minutes, and flattening a square towel (paper) can take up to 15 minutes. It’s fair to say commercial and industrial uses of CloPeMa are not around the corner. We’re still a long way from a situation where we could all have such a robot at home.

However, the CloPeMa team is not alone and researchers around the world have made considerable progress in developing a laundry robot. For example, the University of California Berkeley’s PR2 robot can perform numerous garment handling tasks, such as unfolding and folding towels and pairing socks. Research is advancing fast enough that in a few years we will start seeing robots that can execute more than one kind of task.

And while we tend to think of robots as anthropomorphic machines, we actually already have robots as “smart” white good appliances at homes. For instance, washing machines that adapt their washing cycles to the clothes type or automated home beer brewing machines (a kind of modern version of Philo’s robot maid).

The dream, of course, is still to have robots intelligent enough that we only have to command them once to do our household chores, and not supervise them at all.

Gerardo Aragon Camarasa received funding from the European Commission, under the contract No. 288553, CloPeMa