Counting is easy until the evidence disappears

Ask a video model how many times a ball hits a wall and the task sounds almost trivial. It is not. The model first has to see each contact, keep the events separate and carry the total to the end of the clip.

A new preprint turns that simple job into a controlled stress test. The researchers made 2,190 synthetic videos with three kinds of event: a bouncing ball touching a wall, a brief visual blink and a state changing from one category to another.

Every clip lasts 24 seconds. The team varies two things independently: how many events happen and how quickly they arrive. Appearance, geometry and start time change across ten random versions of each setting, while the schedule stays fixed.

That design makes the failure easier to locate. A wrong answer cannot be blamed vaguely on a busy real-world scene.

A final number can hide a bad trace

Most video benchmarks ask whether the final answer is right. This study also asks models to report when each event happened, then matches those timestamps against an executable trace produced by the video generator.

The distinction catches two different mistakes. A model may miss or invent events but land on the correct total by accident. Or it may describe most events correctly and then add them up badly.

Across the main Gemini 3.6 Flash test, the average exact-match score was 21.1% across the three tasks. The average timestamp recall was 31.2%. The result varied sharply with the way an event was shown.

Persistent state changes were much easier to recover than momentary blinks. The visual evidence stayed on screen instead of vanishing between sampled frames.

The reliable region shrank quickly

The authors define a setting as reliable when at least eight of ten randomized clips receive the right count, and when every easier count or slower rate leading to it also meets that bar.

Under that rule, Gemini counted persistent state transitions reliably through 12 events at 0.5 and 1 event per second. For brief blinks, it had no reliable positive-count region at all.

At high count and high frequency, just 0.2% of final counts were correct. The model reported only 28% as many timestamps as actually occurred and recovered 18.1% of the true events.

The paper also tested several Qwen3-VL and InternVL configurations. Their shapes differed, but none kept a broad reliable region once the event load rose. This is a system-level result, though: model, interface and frame sampling are tangled together.

More frames did not solve it

Google's Gemini documentation says the standard video pipeline samples about one frame per second and warns that fast action may lose detail. The researchers therefore tried giving the model more visual evidence.

On the bouncing-ball task, four-frame-per-second sampling lifted final count accuracy from 19.6% to 29.3%. But only 3.7% of the reported event sequence matched the ground truth faithfully.

Event-centred keyframes produced a much larger score, but they are an upper-bound intervention: selecting a frame around each true event can reveal how many events occurred before the model has to count them.

Prompt changes and explicit reasoning brought little improvement. More input can make the final number look better without showing that the system actually followed the sequence.

What is confirmed, found and still open

Confirmed: the eight-author preprint was submitted to arXiv on 6 August 2026 under a CC BY 4.0 licence. It reports controlled tests on 2,190 generated videos and additional interventions across several video-language systems.

The research finding: success depends heavily on event count, frequency and visual persistence. At higher loads, omissions dominate. Dense sampling can improve a final count while leaving the event trace largely unfaithful.

Still open: how well the result transfers to production video systems, other APIs and longer natural scenes. The authors test real repeated-event clips too, but the clearest numerical boundaries come from synthetic tasks. The study is a preprint and has not been independently replicated.

The practical lesson is modest. If an application needs to count fast or repeated events, a plausible answer is not enough. It needs event-level evidence and a tested operating range.

Sources

  1. Baskar et al. - The Low-Frequency TrapPrimary preprint record submitted 6 August 2026. Source for authorship, version date, licence and the main abstracted findings.
  2. Baskar et al. - Full Low-Frequency Trap paperFull primary manuscript. Source for the benchmark design, model configurations, trace metrics, interventions, numerical results and stated limitations.
  3. Google AI for Developers - Video understandingPrimary product documentation confirming the default one-frame-per-second video processing rate and warning that rapid action can lose detail.