Manifesto from Dagstuhl Perspectives Workshop 24352 -- Conversational Agents: A Framework for Evaluation (CAFE)
Manifesto from Dagstuhl Perspectives Workshop 24352 -- Conversational Agents: A Framework for Evaluation (CAFE)
During the workshop, we deeply discussed what CONversational Information ACcess (CONIAC) is and its unique features, proposing a world model abstracting it, and defined the Conversational Agents Framework for Evaluation (CAFE) for the evaluation of CONIAC systems, consisting of six major components: 1) goals of the system's stakeholders, 2) user tasks to be studied in the evaluation, 3) aspects of the users carrying out the tasks, 4) evaluation criteria to be considered, 5) evaluation methodology to be applied, and 6) measures for the quantitative criteria chosen.
Christine Bauer、Li Chen、Nicola Ferro、Norbert Fuhr、Avishek Anand、Timo Breuer、Guglielmo Faggioli、Ophir Frieder、Hideo Joho、Jussi Karlgren、Johannes Kiesel、Bart P. Knijnenburg、Aldo Lipani、Lien Michiels、Andrea Papenmeier、Maria Soledad Pera、Mark Sanderson、Scott Sanner、Benno Stein、Johanne R. Trippas、Karin Verspoor、Martijn C Willemsen
计算技术、计算机技术
Christine Bauer,Li Chen,Nicola Ferro,Norbert Fuhr,Avishek Anand,Timo Breuer,Guglielmo Faggioli,Ophir Frieder,Hideo Joho,Jussi Karlgren,Johannes Kiesel,Bart P. Knijnenburg,Aldo Lipani,Lien Michiels,Andrea Papenmeier,Maria Soledad Pera,Mark Sanderson,Scott Sanner,Benno Stein,Johanne R. Trippas,Karin Verspoor,Martijn C Willemsen.Manifesto from Dagstuhl Perspectives Workshop 24352 -- Conversational Agents: A Framework for Evaluation (CAFE)[EB/OL].(2025-06-08)[2025-06-29].https://arxiv.org/abs/2506.11112.点此复制
评论