Skip navigation
Skip navigation

A stochastic quasi-Newton method for online convex optimization

Schraudolph, Nicol; Yu, Jin; Guenter, Simon

Description

We develop stochastic variants of the wellknown BFGS quasi-Newton optimization method, in both full and memory-limited (LBFGS) forms, for online optimization of convex functions. The resulting algorithm performs comparably to a well-tuned natural gradient descent but is scalable to very high-dimensional problems. On standard benchmarks in natural language processing, it asymptotically outperforms previous stochastic gradient methods for parameter estimation in conditional random fields. We are...[Show more]

CollectionsANU Research Publications
Date published: 2007
Type: Conference paper
URI: http://hdl.handle.net/1885/38855
Source: Proceedings of The 11th International Conference on Artificial Intelligence and Statistics (AISTATS 2007)

Download

File Description SizeFormat Image
01_Schraudolph_A_stochastic_quasi-Newton_2007.pdf567.67 kBAdobe PDF    Request a copy


Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.

Updated:  12 November 2018/ Responsible Officer:  University Librarian/ Page Contact:  Library Systems & Web Coordinator