Robotic hand/eye coordination system in an assisted technology context

The world population is ageing rapidly causing increasing demand for aged care agencies and nurses. Such demand has become a strong drive for researchers to develop new assistive technology devices such as domestic robots. Unlike an industrial robot, a domestic robot faces greater challenges. This research focuses on hand/eye coordination tasks. One critical information required in the planning of a hand/eye coordination task is accurate knowledge of a target object's pose and shape. An industrial robot obtains this information either by using active sensing methods such as laser-based sensors or knowledge-based object modeling methods. These approaches are avoided due for the following reasons. Active sensing methods, although eye-safe, are considered intrusive as a domestic robot shares its workspace with humans. A knowledge-based solution requires a constant updating database which is not feasible for a domestic setting environment. Unfortunately, only very few studies have been conducted to solve issues of hand/eye coordination tasks in a domestic setting using non-intrusive sensing methods. Therefore, it is worth exploring passive sensing methods for hand/eye coordination tasks in a domestic setting. The importance of this research is that domestic robot application is becoming urgent as an assisted technology because the world's population is ageing rapidly. A popular non-intrusive sensing method is passive stereo vision. In addition to this sensor's cost e_ectiveness, it is also predicted to be popular as it is embedded in smart cameras. For these reasons, it is worthwhile conducting further research on stereo vision as it has high potential impacts for many applications. Although a stereo method has been avoided as a 3D sensing method for robotic grasping, particularly in a domestic setting, due to its noisiness and relative inaccuracy, the proposed box modeling method in this research is able to remove stereo data outliers e_ectively. The proposed box model method discards the information about an object's shape. Nevertheless, the model allows crude grasping in a non-intrusive way. Furthermore, this method allows real time implementation. The drawback of the proposed box model is that the shape information of an object is discarded. For this reason, a superquadrics model, being able to represent a large number of shapes in a compact way, is adopted to improve the proposed box model method. However, the model implemented with a single viewpoint faces the critical issue of superquadric parameter ambiguity. This issue is solved by using the multiview approach. Although the proposed multiview approach is a stereo-based method, the signi_cant achievement is that this method is not limited by the lack of object texture. It achieves such robustness by using stereo weaknesses such as stereo dropouts to produce an object's silhouette. Subsequently, the shape-from-silhouette is also combined with stereo data to achieve a minimum angle between views. Finally, the developed passive stereo-based method achieves more robustness in a domestic setting, despite stereo data noisiness and inaccuracy. It therefore allows a nonintrusive method to be utilized to perform hand/eye coordination tasks, and is deemed suitable for assistive technology due to its non-intrusiveness.