Mahony, RobertWilliamson, Robert2009-05-222010-12-202009-05-222010-12-20Journal of Machine Learning Research 1.9 (2001): 311-3551532-44351533-7928http://hdl.handle.net/10440/305http://digitalcollections.anu.edu.au/handle/10440/305A family of gradient descent algorithms for learning linear functions in an online setting is considered. The family includes the classical LMS algorithm as well as new variants such as the Exponentiated Gradient (EG) algorithm due to Kivinen and Warmuth. The algorithms are based on prior distributions defined on the weight space. Techniques from differential geometry are used to develop the algorithms as gradient descent iterations with respect to the natural gradient in the Riemannian structure induced by the prior distribution. The proposed framework subsumes the notion of "link-functions".45 pageshttp://www.sherpa.ac.uk/romeo/search.php "Author can archive pre-print (ie pre-refereeing) ... [but] cannot archive post-print (ie final draft post-refereeing) … [and] subject to Restrictions, 3 months for STM, author can archive publisher's version/PDF ... on institutional repository; Publisher copyright and source must be acknowledged; Must link to journal homepage; Publishers’ copyright statement must be included; Publisher's version/PDF must be used for post-print deposit." - from SHERPA/RoMEO site (as at 18/02/10)Gradient descentexponentiated gradient algorithmnatural gradientlinkfunctionsRiemannian metric2001-0910.1162/1532443017536837352015-12-10