What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play
What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play
计算技术、计算机技术
Jordan Boyd-Graber,Shi Feng.What can AI do for me: Evaluating Machine Learning Interpretations in Cooperative Play[EB/OL].(2018-10-22)[2025-10-17].https://arxiv.org/abs/1810.09648.点此复制
Machine learning is an important tool for decision making, but its ethical
and responsible application requires rigorous vetting of its interpretability
and utility: an understudied problem, particularly for natural language
processing models. We propose an evaluation of interpretation on a real task
with real human users, where the effectiveness of interpretation is measured by
how much it improves human performance. We design a grounded, realistic
human-computer cooperative setting using a question answering task, Quizbowl.
We recruit both trivia experts and novices to play this game with computer as
their teammate, who communicates its prediction via three different
interpretations. We also provide design guidance for natural language
processing human-in-the-loop settings.
展开英文信息
评论