Fast iterative kernel principal component analysis
Download (3.11 MB)
Guenter, Simon; Schraudolph, Nicol; Vishwanathan, S
Description
We develop gain adaptation methods that improve convergence of the kernel Hebbian algorithm (KHA) for iterative kernel PCA (Kim et al., 2005). KHA has a scalar gain parameter which is either held constant or decreased according to a predetermined annealing schedule, leading to slow convergence. We accelerate it by incorporating the reciprocal of the current estimated eigenvalues as part of a gain vector. An additional normalization term then allows us to eliminate a tuning parameter in the...[Show more]
dc.contributor.author | Guenter, Simon | |
---|---|---|
dc.contributor.author | Schraudolph, Nicol | |
dc.contributor.author | Vishwanathan, S | |
dc.date.accessioned | 2009-05-21T05:19:22Z | |
dc.date.accessioned | 2010-12-20T06:02:43Z | |
dc.date.available | 2009-05-21T05:19:22Z | |
dc.date.available | 2010-12-20T06:02:43Z | |
dc.identifier.citation | Journal of Machine Learning Research 8 (2007): 1893-1918 | |
dc.identifier.issn | 1532-4435 | |
dc.identifier.issn | 1533-7928 | |
dc.identifier.uri | http://hdl.handle.net/10440/298 | |
dc.identifier.uri | http://digitalcollections.anu.edu.au/handle/10440/298 | |
dc.description.abstract | We develop gain adaptation methods that improve convergence of the kernel Hebbian algorithm (KHA) for iterative kernel PCA (Kim et al., 2005). KHA has a scalar gain parameter which is either held constant or decreased according to a predetermined annealing schedule, leading to slow convergence. We accelerate it by incorporating the reciprocal of the current estimated eigenvalues as part of a gain vector. An additional normalization term then allows us to eliminate a tuning parameter in the annealing schedule. Finally we derive and apply stochastic meta-descent (SMD) gain vector adaptation (Schraudolph, 1999, 2002) in reproducing kernel Hilbert space to further speed up convergence. Experimental results on kernel PCA and spectral clustering of USPS digits, motion capture and image denoising, and image super-resolution tasks confirm that our methods converge substantially faster than conventional KHA. To demonstrate scalability, we perform kernel PCA on the entire MNIST data set. | |
dc.format | 26 | |
dc.publisher | MIT Press | |
dc.rights | http://www.sherpa.ac.uk/romeo/search.php "Author can archive pre-print (ie pre-refereeing) ... [but] cannot archive post-print (ie final draft post-refereeing) … [and] subject to Restrictions, 3 months for STM, author can archive publisher's version/PDF ... on institutional repository; Publisher copyright and source must be acknowledged; Must link to journal homepage; Publishers’ copyright statement must be included; Publisher's version/PDF must be used for post-print deposit." - from SHERPA/RoMEO site (as at 18/02/10) | |
dc.source | Journal of Machine Learning Research | |
dc.source.uri | http://jmlr.csail.mit.edu/papers/volume8/guenter07a/guenter07a.pdf | |
dc.subject | step size adaptation | |
dc.subject | gain vector adaptation | |
dc.subject | stochastic meta-descent | |
dc.subject | kernel Hebbian algorithm | |
dc.subject | online learning | |
dc.title | Fast iterative kernel principal component analysis | |
dc.type | Journal article | |
local.description.notes | Article written under name Simon Günter. Affiliation in article: Guenter, Simon, Schraudolph, Nicol and Vishwanathan, S, ALL also with National ICT Australia, Statistical Machine Learning Program. Article revised April 2007. | |
local.identifier.citationvolume | 8 | |
dc.date.issued | 2007-08 | |
local.identifier.absfor | 080109 | |
local.identifier.ariespublication | u8803936xPUB181 | |
local.type.status | Published Version | |
local.contributor.affiliation | Guenter, Simon, Research School of Information Sciences and Engineering, Computer Sciences Laboratory | |
local.contributor.affiliation | Schraudolph, Nicol, Research School of Information Sciences and Engineering, Computer Sciences Laboratory | |
local.contributor.affiliation | Vishwanathan, S, Research School of Information Sciences and Engineering, Computer Sciences Laboratory | |
local.bibliographicCitation.startpage | 1893 | |
local.bibliographicCitation.lastpage | 1918 | |
dc.date.updated | 2015-12-09T07:42:19Z | |
local.identifier.scopusID | 2-s2.0-34548170925 | |
Collections | ANU Research Publications |
Download
File | Description | Size | Format | Image |
---|---|---|---|---|
Guenter_Fast2007.pdf | 3.11 MB | Adobe PDF |
Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.
Updated: 17 November 2022/ Responsible Officer: University Librarian/ Page Contact: Library Systems & Web Coordinator