Event-Based Vision at the Edge: Neuromorphic Sensors, Spiking Processors, and the Toolchain Gap
At a Glance
- A different kind of camera: Event cameras don't capture frames. Each pixel reports only when its brightness changes, giving microsecond-scale pixel latency and up to 140 dB of dynamic range against roughly 60 dB for frame cameras.
- Sensors are already small and frugal: Prophesee's GenX320 is a 3×4 mm, 320×320 sensor with power-management modes down to 36 µW and a parallel readout mode at 3–4 mW. The Sony/Prophesee IMX636 brings the approach to 1280×720.
- Sensor and spiking processor on one chip: SynSense's Speck integrates an event sensor with a 327,000-neuron spiking CNN and processes each event through nine layers in 3.36 µs.
- The biggest energy wins come from processing events one at a time: On Intel's Loihi, a fully asynchronous robot-dodging network used 1.64% of the energy per inference of an event-frame approach on the same chip, and 4.30% of a Jetson Orin NX baseline.
- The gap is software: A 2024 Nature Communications study moved the same spiking models across seven simulators and four chips. Convolutional models held accuracy within about a percentage point. Recurrent models — the kind that use event timing most — showed significant accuracy differences from platform to platform.
This is the vision counterpart to "Sub-Milliwatt Keyword Spotting," which found that once the classifier falls into microwatts, the sensor front end and the measurement boundary become the real story. It also extends "Spiking Neural Network Accelerators and the Neuromorphic Edge Inference Revival," where event-based vision came up as one of the workloads that suits spiking hardware best. Here we follow the whole chain: the sensor, the processor, and the software that has to connect them.
What an Event Camera Measures
A conventional camera samples every pixel at a fixed rate, whether or not anything in the scene changed. An event camera works the other way. Each pixel responds on its own and emits an event only when its brightness changes past a threshold. The event records when it happened, where, and whether the pixel got brighter or darker.
The 2022 IEEE TPAMI survey "Event-Based Vision," by Guillermo Gallego, Tobi Delbruck, Davide Scaramuzza and colleagues, sets out the advantages:
- Dynamic range: 140 dB versus about 60 dB for frame cameras, so bright and dark regions of the same scene stay usable
- Temporal resolution: high, which reduces motion blur
- Power: low, because a static scene produces few events
The same property creates the processing problem. An event camera's output isn't an image. It's an asynchronous stream of sparse points in space and time, and most computer vision software and hardware expects frames.
The Sensors
Two commercial sensors mark the range of what's available.
Sony/Prophesee IMX636 is the high-resolution option:
- 1280×720, about 0.92 megapixels, with 4.86 µm pixels in a 1/2.5-inch format
- Pixel latency under 100 µs at 1,000 lux and under 1,000 µs at 5 lux
- Dynamic range above 86 dB from 5 lux to 100 klux, and above 120 dB from 80 mlux to 100 klux
- 5 mW standby power and 205 mW maximum
- Embedded event signal processing, per Prophesee's product page, which does not list the specific filters
Prophesee GenX320, launched in October 2023, targets always-on devices:
- 320×320 resolution with 6.3 µm backside-illuminated stacked pixels in a 1/5-inch format, on a 3×4 mm die
- Pixel latency under 150 µs at 1,000 lux, and dynamic range above 140 dB
- Power-management modes down to 36 µW; a low-power monitor mode at 2.9 mW, parallel readout at 3–4 mW depending on event rate, and MIPI CSI-2 streaming at 22.8 mW
The GenX320's on-chip processing matters as much as its pixels. The sensor timestamps and filters events before they leave the chip. An event rate controller caps the output rate, and a spatio-temporal contrast filter detects and removes redundant bursts. Some of the work of deciding what's worth processing has already moved into the sensor.
Note the spread in GenX320's modes: 36 µW to 22.8 mW, more than two orders of magnitude, depending on how the data leaves the chip. As with the front-end measurement boundary in "Sub-Milliwatt Keyword Spotting," a sensor's power figure depends on which mode it's in.
Sensor Plus Spiking Processor
The first end-to-end system
In 2017, Arnon Amir, Brian Taba, and colleagues at IBM presented the first gesture recognition system built end to end on event-based hardware at CVPR. A DVS128 event camera streamed events live into IBM's TrueNorth, a processor with a million spiking neurons, configured as a convolutional network.
- Accuracy: 96.5% out of sample on DvsGesture, a new dataset of 11 hand gestures from 29 subjects under three lighting conditions
- Latency: 105 ms to detect the onset of a gesture
- Power: under 200 mW
One chip: SynSense Speck
SynSense's Speck removes the link between sensor and processor entirely. It's a system-on-chip that integrates an event-based vision sensor — a 128×128 array, per SynSense — with an asynchronous spiking convolutional neural network.
Ole Richter and colleagues described the pipeline at the IEEE ASYNC symposium in 2023:
- Scale: 327,000 neurons across nine spiking CNN layers
- Latency: 3.36 µs per incoming event
- Design principle: highly sparse computation, for edge processing in a compact device
SynSense's product material quotes scene analysis under 10 mW with real-time response under 200 ms, for gesture recognition, face and object detection, and tracking. Those are vendor figures rather than a benchmark measurement.
Because Speck is fully asynchronous, circuits act only when events arrive. There's no frame clock and no point at which the chip gathers a full image. The low resolution is the trade-off: 128×128 suits gestures and presence detection better than detailed scene understanding.
Why per-event processing matters: Loihi on a moving robot
The clearest measurement of what asynchronous processing buys comes from a 2024 robotics study by Junjie Jiang, Delei Kong, Chenming Hu, and Zheng Fang. They built an obstacle-dodging system for a mobile robot from an event camera, a spiking network, and Intel's Loihi.
Many event-vision systems don't process events individually. They collect a batch over a time window and convert it into an event frame or a 3D voxel grid, then run something close to conventional vision on it. Jiang and colleagues processed each event as it arrived.
The energy results:
- 4.30% of the energy per inference of an event-spike-tensor method on an NVIDIA Jetson Orin NX in energy-saving mode
- 1.64% of the energy per inference of an event-frame method running on the same Loihi chip
The second number is the more revealing. Same processor, same sensor type, same task — and batching events into frames cost about 60 times more energy. The authors also report better robustness than frame-based methods across time windows and lighting conditions, and describe the work as the first fully asynchronous neuromorphic system solving sequential tasks on a real mobile robot.
Much of the efficiency, in other words, comes from how the software represents events. Neuromorphic hardware running frame-style processing gives most of it back.
The Conventional Competitor
Event sensors don't require spiking processors, and the lowest-power wearable result in this article runs on neither a neuromorphic chip nor a GPU.
Helios, from Prarthana Bhattacharyya and colleagues in 2024, targets always-on gesture control for smart eyewear. It pairs a 3×4 mm event camera drawing 20 mW with an NXP Nano UltraLite compute platform under 350 mW. It reached 91% accuracy on seven gesture classes, including subtle swipes and pinches, with 60 ms latency across 20 users.
Helios 2.0, in 2025, recognizes lateral thumb swipes and a double pinch on a Qualcomm Snapdragon Hexagon DSP:
- Power: 6–8 mW
- Accuracy: F1 above 80% with a six-channel model and above 70% with two channels
- Against the prior state of the art: 20% better F1 at 25 times lower power
- Training data: synthetic only
That's a conventional mobile DSP reaching single-digit milliwatts for event-based gesture recognition. For a product team, the sensor decision and the processor decision are separable. An event sensor with good on-chip filtering can pair with a DSP already in the phone or glasses. The case for a spiking processor has to rest on what it does better — per-event asynchronous operation, latency, or lower power at the idle floor — not on the sensor alone.
The Toolchain Gap
That brings us to the problem every hardware result above works around: software.
Every neuromorphic platform in this article has its own stack. Intel has Lava for Loihi. SynSense has Rockpool for Xylo and Sinabs for Speck. Research groups use snnTorch, Norse, Nengo, or Spyx. A spiking model trained in one generally doesn't run on another platform's hardware without rework.
A common intermediate representation
In 2024, Jens Pedersen, Jason Eshraghian, and colleagues published the Neuromorphic Intermediate Representation (NIR) in Nature Communications. They framed the problem directly: approaches that blur top-down algorithm design and bottom-up hardware design make it hard to extract clear abstractions that carry across technology stacks, and that holds back incremental progress.
NIR plays the role for spiking systems that intermediate representations play in conventional ML compilers:
- Primitives: 11 computational primitives and 3 higher-order ones, including affine and linear maps, convolution, delay, integrators and leaky integrators, scaling, and spiking
- Formalism: each primitive is defined as a hybrid system of continuous-time dynamics plus discrete events, so the representation isn't tied to one simulator's time step or one chip's constraints
- Coverage: seven simulators (Lava, Nengo, Norse, Rockpool, Sinabs, snnTorch, Spyx) and four hardware platforms (Intel Loihi 2, SpiNNaker 2, SynSense Speck, SynSense Xylo)
What happened when models moved
The team reproduced three models across those platforms: a single leaky integrate-and-fire neuron, a spiking CNN on the event-based N-MNIST dataset, and a spiking recurrent network for Braille letter recognition.
- Spiking CNN: mean test accuracy of 97.7%, with a standard deviation of 0.9% across platforms
- Spiking recurrent network: significant differences in end-task performance between platforms
The authors traced the differences to three sources: how each platform implements the neuron model, quantization, and determinism.
That result has a direct consequence for event vision. Convolutional spiking models, which treat events roughly like sparse images, port reasonably well. Recurrent models, which carry state over time, don't yet — and state over time is how a network uses the precise timing that makes event cameras worth having. The Loihi robot result showed that per-event temporal processing is where the energy savings are. NIR's result shows that it's also where portability is weakest.
NIR's own stated limits point to the next gaps. It doesn't yet cover adaptive threshold mechanisms, gating, resonate-and-fire neurons, or multicompartment neuron models, and it targets digital platforms rather than analog ones.
What This Means for Anyone Building Event-Based Edge Vision
- Choose the sensor and processor separately. A 36 µW-capable sensor with on-chip filtering can feed a spiking SoC, a neuromorphic research chip, or a conventional DSP. Helios 2.0 shows the DSP route already reaches 6–8 mW.
- Keep events as events if power is the goal. On the same Loihi chip, per-event processing used 1.64% of the energy of event frames. Converting to frames early gives most of the advantage away.
- Expect integrated sensor-processor chips to fit narrow tasks. Speck's microsecond per-event latency comes with a 128×128 array suited to gestures and presence.
- Budget for revalidation when you port. Convolutional spiking models moved across platforms within about a point of accuracy. Recurrent ones didn't, so a model tuned for one chip needs retesting on another.
Speculation: The pieces are converging — sensors that filter on-chip, processors that compute per event, and an intermediate representation that several vendors already support. The software gap that remains sits exactly where event cameras are most useful, in stateful models that use timing. If intermediate representations close that gap, the case for spiking processors in always-on vision gets stronger, because it's in timing-based models that per-event hardware separates from a conventional DSP running binned events.
This article completes a three-part run on neuromorphic edge hardware, following "Spiking Neural Network Accelerators and the Neuromorphic Edge Inference Revival" and "Sub-Milliwatt Keyword Spotting." It connects back to "Neuromorphic Chips and the Future of Edge AI." A natural next step is the analog side of the same problem — in-memory computing with resistive devices, where the portability and drift questions look different again.
Stay in the loop. Project Isocline publishes deep-dive technical analysis on AI infrastructure, energy systems, and the engineering decisions shaping the next decade of compute. No noise. No fluff.
You can unsubscribe at any time.