Skip navigation
Skip navigation

Real-time Visual Flow Algorithms for Robotic Applications

Adarve Bermudez, Juan David

Description

Vision offers important sensor cues to modern robotic platforms. Applications such as control of aerial vehicles, visual servoing, simultaneous localization and mapping, navigation and more recently, learning, are examples where visual information is fundamental to accomplish tasks. However, the use of computer vision algorithms carries the computational cost of extracting useful information from the stream of raw pixel data. The most sophisticated algorithms...[Show more]

dc.contributor.authorAdarve Bermudez, Juan David
dc.date.accessioned2018-01-11T02:41:48Z
dc.date.available2018-01-11T02:41:48Z
dc.identifier.otherb48528547
dc.identifier.urihttp://hdl.handle.net/1885/139168
dc.description.abstractVision offers important sensor cues to modern robotic platforms. Applications such as control of aerial vehicles, visual servoing, simultaneous localization and mapping, navigation and more recently, learning, are examples where visual information is fundamental to accomplish tasks. However, the use of computer vision algorithms carries the computational cost of extracting useful information from the stream of raw pixel data. The most sophisticated algorithms use complex mathematical formulations leading typically to computationally expensive, and consequently, slow implementations. Even with modern computing resources, high-speed and high-resolution video feed can only be used for basic image processing operations. For a vision algorithm to be integrated on a robotic system, the output of the algorithm should be provided in real time, that is, at least at the same frequency as the control logic of the robot. With robotic vehicles becoming more dynamic and ubiquitous, this places higher requirements to the vision processing pipeline. This thesis addresses the problem of estimating dense visual flow information in real time. The contributions of this work are threefold. First, it introduces a new filtering algorithm for the estimation of dense optical flow at frame rates as fast as 800 Hz for 640x480 image resolution. The algorithm follows a update-prediction architecture to estimate dense optical flow fields incrementally over time. A fundamental component of the algorithm is the modeling of the spatio-temporal evolution of the optical flow field by means of partial differential equations. Numerical predictors can implement such PDEs to propagate current estimation of flow forward in time. Experimental validation of the algorithm is provided using high-speed ground truth image dataset as well as real-life video data at 300 Hz. The second contribution is a new type of visual flow named structure flow. Mathematically, structure flow is the three-dimensional scene flow scaled by the inverse depth at each pixel in the image. Intuitively, it is the complete velocity field associated with image motion, including both optical flow and scale-change or apparent divergence of the image. Analogously to optic flow, structure flow provides a robotic vehicle with perception of the motion of the environment as seen by the camera. However, structure flow encodes the full 3D image motion of the scene whereas optic flow only encodes the component on the image plane. An algorithm to estimate structure flow from image and depth measurements is proposed based on the same filtering idea used to estimate optical flow. The final contribution is the spherepix data structure for processing spherical images. This data structure is the numerical back-end used for the real-time implementation of the structure flow filter. It consists of a set of overlapping patches covering the surface of the sphere. Each individual patch approximately holds properties such as orthogonality and equidistance of points, thus allowing efficient implementations of low-level classical 2D convolution based image processing routines such as Gaussian filters and numerical derivatives. These algorithms are implemented on GPU hardware and can be integrated to future Robotic Embedded Vision systems to provide fast visual information to robotic vehicles.
dc.language.isoen
dc.subjectRobotics
dc.subjectComputer Vision
dc.subjectOptical Flow
dc.subjectSpherical Images
dc.subjectImage Processing
dc.subjectGPU Programming
dc.subjectVisual Navigation
dc.subjectReal-time
dc.subjectOmnidirectional Vision
dc.subjectCameras
dc.subjectData structures
dc.subjectImage quality
dc.subjectConvolution
dc.subjectInterpolation
dc.titleReal-time Visual Flow Algorithms for Robotic Applications
dc.typeThesis (PhD)
local.contributor.supervisorMahony, Robert
local.contributor.supervisorcontactrobert.mahony@anu.edu.au
dcterms.valid2017
local.description.notesthe author deposited 11/01/18
local.type.degreeDoctor of Philosophy (PhD)
dc.date.issued2017
local.contributor.affiliationANU College of Engineering & Computer Science, The Australian National University
local.identifier.doi10.25911/5d6512efc9dda
local.mintdoimint
CollectionsOpen Access Theses

Download

File Description SizeFormat Image
Adarve Bermudez Thesis 2017.pdf19.14 MBAdobe PDFThumbnail


Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.

Updated:  19 May 2020/ Responsible Officer:  University Librarian/ Page Contact:  Library Systems & Web Coordinator