Conference Paper
Computer Vision

Event-Based Vision: A Survey

Guillermo Gallego(Einstein Center Digital Future), Tobi Delbrück(University of Zurich), Garrick Orchard(Intel (United States)), Chiara Bartolozzi(Italian Institute of Technology), Brian Taba(IBM (United States)), Andrea Censi(ETH Zurich), Stefan Leutenegger(Imperial College London), Andrew J. Davison(Imperial College London), Jörg Conradt(KTH Royal Institute of Technology), Kostas Daniilidis(University of Pennsylvania), Davide Scaramuzza(University of Zurich)
July 10, 2020IEEE Transactions on Pattern Analysis and Machine Intelligence2,551 citations
PDFDOI

2.6k

Citations

0

Influential Citations

IEEE Transactions on Pattern Analysis and Machine Intelligence

Venue

2020

Year

Abstract

Event cameras are bio-inspired sensors that differ from conventional frame cameras: Instead of capturing images at a fixed rate, they asynchronously measure per-pixel brightness changes, and output a stream of events that encode the time, location and sign of the brightness changes. Event cameras offer attractive properties compared to traditional cameras: high temporal resolution (in the order of μs), very high dynamic range (140 dB versus 60 dB), low power consumption, and high pixel bandwidth (on the order of kHz) resulting in reduced motion blur. Hence, event cameras have a large potential for robotics and computer vision in challenging scenarios for traditional cameras, such as low-latency, high speed, and high dynamic range. However, novel methods are required to process the unconventional output of these sensors in order to unlock their potential. This paper provides a comprehensive overview of the emerging field of event-based vision, with a focus on the applications and the algorithms developed to unlock the outstanding properties of event cameras. We present event cameras from their working principle, the actual sensors that are available and the tasks that they have been used for, from low-level vision (feature detection and tracking, optic flow, etc.) to high-level vision (reconstruction, segmentation, recognition). We also discuss the techniques developed to process events, including learning-based techniques, as well as specialized processors for these novel sensors, such as spiking neural networks. Additionally, we highlight the challenges that remain to be tackled and the opportunities that lie ahead in the search for a more efficient, bio-inspired way for machines to perceive and interact with the world.

Analysis

Why This Paper Matters

Event cameras represent a fundamental shift from frame-based to asynchronous, event-driven sensing, offering microsecond temporal resolution, 140 dB dynamic range, and low power consumption. This survey, published in IEEE TPAMI and cited over 2,500 times, provides the first comprehensive overview of the emerging field of event-based vision, bridging sensor hardware, processing algorithms, and applications. It addresses the critical need for novel methods to handle the unconventional event stream output, which encodes per-pixel brightness changes rather than intensity frames.

By consolidating research from low-level vision tasks like feature tracking and optical flow to high-level tasks such as reconstruction and recognition, the paper serves as a foundational reference for both newcomers and experts. It also highlights specialized processors and spiking neural networks, emphasizing the bio-inspired nature of the technology. The survey's timing coincides with growing interest in neuromorphic engineering and low-latency perception for robotics and autonomous systems.

Technical Contributions

  • Sensor Principles and Available Hardware: Detailed explanation of event camera operation, including pixel circuits and output characteristics, and a review of commercial and research sensors.
  • Algorithmic Taxonomy: Systematic categorization of event processing methods, from event-by-event processing to frame-based aggregation and learning-based approaches.
  • Learning-Based Techniques: Coverage of deep learning methods adapted for event data, including spiking neural networks (SNNs) and conventional neural networks on event representations.
  • Application Domains: Survey of applications in robotics, AR/VR, and automotive scenarios, highlighting tasks like SLAM, object tracking, and recognition.
  • Challenges and Opportunities: Identification of open problems such as noise, calibration, and the need for standardized benchmarks, alongside future research directions.

Results

As a survey, the paper does not present original experimental results. Instead, it reports comparative metrics from the literature: event cameras achieve temporal resolution on the order of microseconds, dynamic range of 140 dB versus 60 dB for frame cameras, and pixel bandwidth in the kHz range, resulting in reduced motion blur. These properties enable performance in challenging scenarios where traditional cameras fail, such as high-speed motion and extreme lighting. The survey also notes that event-based methods have demonstrated success in tasks like high-speed tracking and HDR reconstruction, though quantitative comparisons are referenced from primary studies.

Significance

This survey has significantly shaped the event-based vision community by providing a unified framework and vocabulary. It has guided the development of new algorithms and hardware, and its identification of challenges has spurred research in areas like event-based deep learning and neuromorphic computing. The paper's emphasis on bio-inspired perception aligns with broader trends in energy-efficient AI and edge computing, influencing both academic research and industrial applications in robotics and autonomous systems.

event-camerasneuromorphic-visioncomputer-visionspiking-neural-networksbio-inspired-sensorslow-latency-visionhigh-dynamic-rangesurvey
Computer VisionNeural Networks

Related Papers