|国家预印本平台
首页|TAPS : Frustratingly Simple Test Time Active Learning for VLMs

TAPS : Frustratingly Simple Test Time Active Learning for VLMs

TAPS : Frustratingly Simple Test Time Active Learning for VLMs

来源:Arxiv_logoArxiv
英文摘要

Test-Time Optimization enables models to adapt to new data during inference by updating parameters on-the-fly. Recent advances in Vision-Language Models (VLMs) have explored learning prompts at test time to improve performance in downstream tasks. In this work, we extend this idea by addressing a more general and practical challenge: Can we effectively utilize an oracle in a continuous data stream where only one sample is available at a time, requiring an immediate query decision while respecting latency and memory constraints? To tackle this, we propose a novel Test-Time Active Learning (TTAL) framework that adaptively queries uncertain samples and updates prompts dynamically. Unlike prior methods that assume batched data or multiple gradient updates, our approach operates in a real-time streaming scenario with a single test sample per step. We introduce a dynamically adjusted entropy threshold for active querying, a class-balanced replacement strategy for memory efficiency, and a class-aware distribution alignment technique to enhance adaptation. The design choices are justified using careful theoretical analysis. Extensive experiments across 10 cross-dataset transfer benchmarks and 4 domain generalization datasets demonstrate consistent improvements over state-of-the-art methods while maintaining reasonable latency and memory overhead. Our framework provides a practical and effective solution for real-world deployment in safety-critical applications such as autonomous systems and medical diagnostics.

Dhruv Sarkar、Aprameyo Chakrabartty、Bibhudatta Bhanja

计算技术、计算机技术

Dhruv Sarkar,Aprameyo Chakrabartty,Bibhudatta Bhanja.TAPS : Frustratingly Simple Test Time Active Learning for VLMs[EB/OL].(2025-07-26)[2025-08-18].https://arxiv.org/abs/2507.20028.点此复制

评论