Textile manipulation remains one of the more demanding challenges in modern robotics, as the shape of the object continuously changes during manipulation, and its implementation on full humanoid platforms remains poorly explored due to the need for whole-body balance, a different arm kinematic configuration, and limited, head-mounted vision. The goals of this master's thesis are therefore to integrate an in-air cloth unfolding application on the TALOS robot, adapt the CeDiRNet model for grasp-point prediction, develop a multi-view image capture strategy, carry out an experimental evaluation, and compare single-view and multi-view perception strategies.
We first place these challenges within a theoretical framework: we present the differences between humanoid and fixed platforms as well as the background of position and torque control, model predictive control (MPC), and modern artificial intelligence approaches, followed by a classification of deformable objects, the physical and mechanical properties of textiles, approaches to their modeling and perception, three groups of manipulation strategies, and open challenges in the field.
Building on this background, we describe our own perception and execution solution. For textile perception, we combine the built-in Orbbec Astra Pro camera for point cloud analysis with an additional RealSense D435i camera for predicting the unfolding point, describing the alignment of both cameras, the use of the RANSAC method for table detection, and a developed panoramic image stitching method from three captures that compensates for the limited field of view relative to the training distribution of the CeDiRNet network. This perception pipeline is then connected to the TALOS platform through the WBC system with a task stack for stability and manipulation, as well as a position controller used to execute grasping and unfolding motions.
We evaluate the proposed approach according to an unfolding protocol on eight textile pieces (four dishtowels, four T-shirts), each in ten repetitions - five with the multi-view and five with the single-view strategy - using coverage percentage as the primary metric. When pooling all trials, the difference between the two approaches is negligible (46.2% vs. 45.3%), but when considered separately by textile type, an opposite pattern emerges: the single-view approach performs better on dishtowels (42.5% vs. 38.4%), while the multi-view approach performs better on T-shirts (53.9% vs. 48.1%), whereas the CeDiRNet inference time did not differ significantly between the two approaches.
The results suggest that the benefit of the multi-view strategy increases with the complexity of the textile relative to the camera's field of view, and are comparable to results reported at the ICRA 2024 Cloth Competition, despite the noted limitations of the evaluation, most notably the small number of trials per combination. Based on these findings, we conclude that all set goals were achieved, with the key finding being precisely the dependence of the multi-view strategy's benefit on textile type, and we propose directions for future work: a more extensive evaluation, improvements to panorama stitching, additional training of the CeDiRNet model, development of dedicated grippers and tactile sensors, and the introduction of closed-loop trial repetition.
|
