Bringing a blurry frame alive at high frame-rate with an event camera

Abstract

Event-based cameras can measure intensity changes (called 'events') with microsecond accuracy under high- speed motion and challenging lighting conditions. With the active pixel sensor (APS), the event camera allows simul- taneous output of the intensity frames. However, the output images are captured at a relatively low frame-rate and often suffer from motion blur. A blurry image can be regarded as the integral of a sequence of latent images, while the events indicate the changes between the latent images. Therefore, we are able to model the blur-generation process by as- sociating event data to a latent image. In this paper, we propose a simple and effective approach, the Event-based Double Integral (EDI) model, to reconstruct a high frame- rate, sharp video from a single blurry frame and its event data. The video generation is based on solving a simple non-convex optimization problem in a single scalar vari- able. Experimental results on both synthetic and real im- ages demonstrate the superiority of our EDI model and op- timization method in comparison to the state-of-the-art.