Skip navigation
Skip navigation

Rank Pooling for Action Recognition

Fernando, Basura; Gavves, Efstratios; Oramas Mogrovejo, Jose Antonio; Ghodrati, Amir; Tuytelaars, Tinne

Description

We propose a function-based temporal pooling method that captures the latent structure of the video sequence data - e.g., how frame-level features evolve over time in a video. We show how the parameters of a function that has been fit to the video data can serve as a robust new video representation. As a specific example, we learn a pooling function via ranking machines. By learning to rank the frame-level features of a video in chronological order, we obtain a new representation that captures...[Show more]

dc.contributor.authorFernando, Basura
dc.contributor.authorGavves, Efstratios
dc.contributor.authorOramas Mogrovejo, Jose Antonio
dc.contributor.authorGhodrati, Amir
dc.contributor.authorTuytelaars, Tinne
dc.date.accessioned2021-09-08T06:08:13Z
dc.identifier.issn0162-8828
dc.identifier.urihttp://hdl.handle.net/1885/247695
dc.description.abstractWe propose a function-based temporal pooling method that captures the latent structure of the video sequence data - e.g., how frame-level features evolve over time in a video. We show how the parameters of a function that has been fit to the video data can serve as a robust new video representation. As a specific example, we learn a pooling function via ranking machines. By learning to rank the frame-level features of a video in chronological order, we obtain a new representation that captures the video-wide temporal dynamics of a video, suitable for action recognition. Other than ranking functions, we explore different parametric models that could also explain the temporal changes in videos. The proposed functional pooling methods, and rank pooling in particular, is easy to interpret and implement, fast to compute and effective in recognizing a wide variety of actions. We evaluate our method on various benchmarks for generic action, fine-grained action and gesture recognition. Results show that rank pooling brings an absolute improvement of 7-10 average pooling baseline. At the same time, rank pooling is compatible with and complementary to several appearance and local motion based methods and features, such as improved trajectories and deep learning features.
dc.description.sponsorshipThe authors acknowledge the support of FP7 ERC Starting Grant 240530 COGNIMUND, KU Leuven DBOF PhD fellowship, the FWO project Monitoring of abnormal activity with camera systems, iMinds High-Tech Visualization project and the Australian Research Council Centre of Excellence for Robotic Vision (project number CE140100016).
dc.format.mimetypeapplication/pdf
dc.language.isoen_AU
dc.publisherInstitute of Electrical and Electronics Engineers (IEEE Inc)
dc.rights© 2016 IEEE
dc.sourceIEEE Transactions on Pattern Analysis and Machine Intelligence
dc.subjectAction recognition
dc.subjecttemporal encoding
dc.subjecttemporal pooling
dc.subjectrank pooling
dc.subjectvideo dynamics
dc.titleRank Pooling for Action Recognition
dc.typeJournal article
local.description.notesImported from ARIES
local.identifier.citationvolume39
dc.date.issued2017
local.identifier.absfor080104 - Computer Vision
local.identifier.absfor080106 - Image Processing
local.identifier.ariespublicationu5357342xPUB229
local.publisher.urlhttps://www.ieee.org/
local.type.statusPublished Version
local.contributor.affiliationFernando, Basura, College of Engineering and Computer Science, ANU
local.contributor.affiliationGavves, Efstratios, University of Amsterdam
local.contributor.affiliationOramas Mogrovejo, Jose Antonio, KU Leuven
local.contributor.affiliationGhodrati, Amir, KU Leuven
local.contributor.affiliationTuytelaars, Tinne, Katholieke Universiteit Leuven
local.description.embargo2099-12-31
dc.relationhttp://purl.org/au-research/grants/arc/CE140100016
local.bibliographicCitation.issue4
local.bibliographicCitation.startpage773
local.bibliographicCitation.lastpage787
local.identifier.doi10.1109/TPAMI.2016.2558148
local.identifier.absseo970108 - Expanding Knowledge in the Information and Computing Sciences
dc.date.updated2020-11-23T11:00:36Z
local.identifier.scopusID2-s2.0-85015928262
local.identifier.thomsonID000397717600013
CollectionsANU Research Publications

Download

File Description SizeFormat Image
01_Fernando_Rank_Pooling_for_Action_2017.pdf1.16 MBAdobe PDF    Request a copy


Items in Open Research are protected by copyright, with all rights reserved, unless otherwise indicated.

Updated:  19 May 2020/ Responsible Officer:  University Librarian/ Page Contact:  Library Systems & Web Coordinator