Pose estimation and collaborative data transmission in visual sensor networks equipped with RGB-D cameras

Low cost RGB-D cameras (or sensors) have significant potential for enhancing the performance of visual sensor network (VSN) applications. RGB-D sensors supplement the conventional red-green-blue (RGB) color information with per-pixel depth data. VSNs, when equipped with RGB-D sensors, open up possibilities for new and innovative application areas. However, to reach their full potential, they need to use their limited battery supplies very frugally, and operate autonomously. Distributed, scalable algorithms must form the backbone of a VSN system architecture. A fundamental requirement of autonomous operation is that a VSN node needs to determine its pose (the location and orientation of its sensors), and use its communication channels as efficiently as possible. The volume of visual and depth data, generated by the sensors of a VSN, is inevitably going to be large. When sensors operate in many hostile environments (especially for disaster recovery, search and rescue operations in confined spaces), this communication problem affects the system overall performance significantly. Such critical situations present immense challenges for efficient data transmission and storage, particularly over shared wireless channels. It should also be noted that conventional localization methods such as GPS (Global Positioning System) cannot be accessible in the places where the sensors operate, such as indoor and underwater environments. This thesis offers novel solutions to the above-mentioned problems. In the first part of the thesis, we present a solution to the sensor pose estimation problem by using color and depth information captured by each RGB-D sensor. We provide an algorithm which computes the relative pose between two sensors by matching the depth images in a distributed manner. Then, we use this algorithm together with graph theory based techniques to develop a self-calibration method which determines each sensor’s pose in a network of multiple RGB-D camera nodes. In the second part of this thesis, we address the problem of efficient data communication under bandwidth constraints. In order to reduce the VSN communication load, we provide new algorithms that allow in-node detection of redundant visual information to avoid its transmission and storage. We achieve this by determining the correlated regions in the captured imagery with the help of the sensor pose estimation methods presented in the first part of the thesis. We introduce a depth video compression scheme for a single mobile RGB-D sensor; then, we develop a collaborative color and depth data coding mechanism for multiple sensors with overlapping fields-of-view. Experimental results obtained on an experimental VSN testbed show that the sensor pose estimation and collaborative data coding mechanism presented in this thesis is able to decrease the overall communication load by approximately 40%, leading to a 55% reduction of a sensor node’s energy consumption due to a significant reduction in the number of required packet transmissions.