|国家预印本平台
首页|QuMATL: Query-based Multi-annotator Tendency Learning

QuMATL: Query-based Multi-annotator Tendency Learning

QuMATL: Query-based Multi-annotator Tendency Learning

来源:Arxiv_logoArxiv
英文摘要

Different annotators often assign different labels to the same sample due to backgrounds or preferences, and such labeling patterns are referred to as tendency. In multi-annotator scenarios, we introduce a novel task called Multi-annotator Tendency Learning (MATL), which aims to capture each annotator tendency. Unlike traditional tasks that prioritize consensus-oriented learning, which averages out annotator differences and leads to tendency information loss, MATL emphasizes learning each annotator tendency, better preserves tendency information. To this end, we propose an efficient baseline method, Query-based Multi-annotator Tendency Learning (QuMATL), which uses lightweight query to represent each annotator for tendency modeling. It saves the costs of building separate conventional models for each annotator, leverages shared learnable queries to capture inter-annotator correlations as an additional hidden supervisory signal to enhance modeling performance. Meanwhile, we provide a new metric, Difference of Inter-annotator Consistency (DIC), to evaluate how effectively models preserve annotators tendency information. Additionally, we contribute two large-scale datasets, STREET and AMER, providing averages of 4300 and 3118 per-annotator labels, respectively. Extensive experiments verified the effectiveness of our QuMATL.

Liyun Zhang、Zheng Lian、Hong Liu、Takanori Takebe、Yuta Nakashima

计算技术、计算机技术

Liyun Zhang,Zheng Lian,Hong Liu,Takanori Takebe,Yuta Nakashima.QuMATL: Query-based Multi-annotator Tendency Learning[EB/OL].(2025-03-19)[2025-05-17].https://arxiv.org/abs/2503.15237.点此复制

评论