Robotized Laundry Manipulation With Appliance User Interface Interpretation

In this paper, a robotized laundry picking and insertion in to washing machine followed by display interpretation and adjustment of the washing cycle for a complete robotic laundry operation is described. To ensure the successful insertion of a laundry, recovery picking from the drum door region in case large clothes remain partially out from the washing machine is also evaluated. A pointcloud-based perception algorithm is proposed to detect wrinkles on the cloth surface to compute spline curves along the wrinkle-like structure and estimate grasping frames. Even more, to insure graspability in the absence of wrinkles, a blob detection approach is evaluated together with grasp pose quality ranking to aim for the optimal pose. A deep learning based washing machine user-interface detection and interpretation algorithm is also developed, to fully Automate the robotic laundry operation. A fully autonomous laundry handling and washing cycle setting on the appliance display is tested and validated extensively by performing multiple tasks using our robotic platforms (Tiago and Baxter) and AEG washing machine.