BROWSE
Detail
Conference Paper
Dynamic Keyframe Sampling for Efficient Vision-Language Situation Understanding in Real-Time Robotic Surveillance
Cited - time in 
Share 
Share
- Authors
- Jaeyoon Jang, Youngwoo Yoon, Hosub Yoon
- Issue Date
- 2026-08
- Citation
- International Symposium on Robot and Human Interactive Communication (RO-MAN) 2026, pp.1-6
- Publisher
- IEEE
- Language
- English
- Type
- Conference Paper
- Abstract
- Deploying large Vision-Language Models (VLMs) on human-care service robots is limited by the cost of processing video at full frame rate, even though most observation time is uneventful. We propose Dynamic Keyframe Sampling (DKS), a low-overhead, always-on front-end that gates VLM inference: it scores visual change with pixel-wise mean absolute difference and, when a person is present, refines the decision with skeleton-based relative motion, then adaptively contracts or expands the sampling interval. Using a Qwen3-VL-8B backbone, DKS cuts the average input frames by roughly 73–78% relative to dense or fixed-rate sampling while preserving accuracy, which sharply raises Intelligence Density (ID), our accuracy-per-frame efficiency measure (e.g., on MSR-VTT-QA, ID rises from 0.44 to 1.97 with under a one-point accuracy change). Beyond a single operating point, we run a controlled comparison against uniform and learned-feature samplers at matched frame budgets—an analysis rarely made explicit in prior work. It reveals that on standard trimmed benchmarks the benefit of DKS comes from adaptive budget reduction rather than from selecting better frames than uniform thinning, and that LoRA adaptation and DKS are complementary. DKS thus offers a practical, architecture-agnostic way to focus VLM computation on informative moments under tight on-robot compute budgets.
- KSP Keywords
- Always-On, Efficiency Measure, Frame rate, Front-End, Language Models, Low overhead, Operating Point, Service robots, Situation understanding, fixed-rate, mean absolute difference