Skip navigation
Skip navigation

Q-Learning for Robot Control

Gaskett, Chris

Description

Q-Learning is a method for solving reinforcement learning problems. Reinforcement learning problems require improvement of behaviour based on received rewards. Q-Learning has the potential to reduce robot programming effort and increase the range of robot abilities. However, most currentQ-learning systems are not suitable for robotics problems: they treat continuous variables, for example speeds or positions, as discretised values. Discretisation does not allow smooth control and does not fully...[Show more]

dc.contributor.authorGaskett, Chris
dc.date.accessioned2008-06-10T05:24:31Z
dc.date.accessioned2011-01-04T02:38:33Z
dc.date.available2008-06-10T05:24:31Z
dc.date.available2011-01-04T02:38:33Z
dc.identifier.otherb21513247
dc.identifier.urihttp://hdl.handle.net/1885/47080
dc.description.abstractQ-Learning is a method for solving reinforcement learning problems. Reinforcement learning problems require improvement of behaviour based on received rewards. Q-Learning has the potential to reduce robot programming effort and increase the range of robot abilities. However, most currentQ-learning systems are not suitable for robotics problems: they treat continuous variables, for example speeds or positions, as discretised values. Discretisation does not allow smooth control and does not fully exploit sensed information. A practical algorithm must also cope with real-time constraints, sensing and actuation delays, and incorrect sensor data. ¶ This research describes an algorithm that deals with continuous state and action variables without discretising. The algorithm is evaluated with vision-based mobile robot and active head gaze control tasks. As well as learning the basic control tasks, the algorithm learns to compensate for delays in sensing and actuation by predicting the behaviour of its environment. Although the learned dynamic model is implicit in the controller, it is possible to extract some aspects of the model. The extracted models are compared to theoretically derived models of environment behaviour. ¶ The difficulty of working with robots motivates development of methods that reduce experimentation time. This research exploits Q-learning’s ability to learn by passively observing the robot’s actions—rather than necessarily controlling the robot. This is a valuable tool for shortening the duration of learning experiments.
dc.language.isoen
dc.rights.uriThe Australian National University
dc.subjectrobotics
dc.subjectreinforcement learning
dc.subjectvision
dc.titleQ-Learning for Robot Control
dc.typeThesis (PhD)
dcterms.valid2002
local.description.refereedyes
local.type.degreeDoctor of Philosophy (PhD)
dc.date.issued2002
local.contributor.affiliationResearch School of Information Sciences and Engineering. Department of Systems Engineering
local.contributor.affiliationThe Australian National University
local.identifier.doi10.25911/5d7a2a09a7dfd
local.mintdoimint
CollectionsOpen Access Theses

Download

File Description SizeFormat Image
01front.pdf151.34 kBAdobe PDFThumbnail
02whole.pdf3.15 MBAdobe PDFThumbnail


Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.

Updated:  17 November 2022/ Responsible Officer:  University Librarian/ Page Contact:  Library Systems & Web Coordinator