Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields
Schmidt, Mark W.; Babanezhad, Reza; Ahemd, Mohamed Osama; Defazio, Aaron; Clifton, Ann; Sarkar, Anoop
Description
We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs). We describe a practical implementation that uses structure in the CRF gradient to reduce the memory requirement of this linearly-convergent stochastic gradient method, propose a non-uniform sampling scheme that substantially improves practical performance, and analyze the rate of convergence of the SAGA variant under non-uniform sampling. Our experimental results reveal that our method...[Show more]
dc.contributor.author | Schmidt, Mark W. | |
---|---|---|
dc.contributor.author | Babanezhad, Reza | |
dc.contributor.author | Ahemd, Mohamed Osama | |
dc.contributor.author | Defazio, Aaron | |
dc.contributor.author | Clifton, Ann | |
dc.contributor.author | Sarkar, Anoop | |
dc.coverage.spatial | San Diego, California, USA | |
dc.date.accessioned | 2016-06-14T23:21:18Z | |
dc.date.created | May 9-12, 2015 | |
dc.identifier.uri | http://hdl.handle.net/1885/103837 | |
dc.description.abstract | We apply stochastic average gradient (SAG) algorithms for training conditional random fields (CRFs). We describe a practical implementation that uses structure in the CRF gradient to reduce the memory requirement of this linearly-convergent stochastic gradient method, propose a non-uniform sampling scheme that substantially improves practical performance, and analyze the rate of convergence of the SAGA variant under non-uniform sampling. Our experimental results reveal that our method significantly outperforms existing methods in terms of the training objective, and performs as well or better than optimally-tuned stochastic gradient methods in terms of test error | |
dc.publisher | JMLR | |
dc.relation.ispartofseries | 18th Intrnational Conference on Artificial Intelligence and Statistics (AISTATS) 2015 | |
dc.source | Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields | |
dc.title | Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields | |
dc.type | Conference paper | |
local.description.notes | Imported from ARIES | |
local.description.refereed | Yes | |
dc.date.issued | 2015 | |
local.identifier.absfor | 080201 - Analysis of Algorithms and Complexity | |
local.identifier.ariespublication | u4334215xPUB1600 | |
local.type.status | Published Version | |
local.contributor.affiliation | Schmidt, Mark W., University of British Columbia | |
local.contributor.affiliation | Babanezhad, Reza, University of British Columbia | |
local.contributor.affiliation | Ahemd, Mohamed Osama, University of British Columbia | |
local.contributor.affiliation | Defazio, Aaron, College of Engineering and Computer Science, ANU | |
local.contributor.affiliation | Clifton, Ann, Simon Fraser | |
local.contributor.affiliation | Sarkar, Anoop, Simon Fraser University | |
local.description.embargo | 2037-12-31 | |
local.bibliographicCitation.startpage | 819 | |
local.bibliographicCitation.lastpage | 828 | |
local.identifier.absseo | 970108 - Expanding Knowledge in the Information and Computing Sciences | |
dc.date.updated | 2016-06-14T09:04:14Z | |
Collections | ANU Research Publications |
Download
File | Description | Size | Format | Image |
---|---|---|---|---|
01_Schmidt_Non-Uniform_Stochastic_Average_2015.pdf | 389.67 kB | Adobe PDF | Request a copy |
Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.
Updated: 17 November 2022/ Responsible Officer: University Librarian/ Page Contact: Library Systems & Web Coordinator