Generative Models, Humans, Predictive Models: Who Is Worse at High-Stakes Decision Making?
Generative Models, Humans, Predictive Models: Who Is Worse at High-Stakes Decision Making?
信息传播、知识传播科学、科学研究计算技术、计算机技术
Keri Mallari,Albert Gordo,Martin T. Wells,Sarah Tan,Kori Inkpen,Julius Adebayo.Generative Models, Humans, Predictive Models: Who Is Worse at High-Stakes Decision Making?[EB/OL].(2024-10-20)[2025-09-18].https://arxiv.org/abs/2410.15471.点此复制
Despite strong advisory against it, large generative models (LMs) are already
being used for decision making tasks that were previously done by predictive
models or humans. We put popular LMs to the test in a high-stakes decision
making task: recidivism prediction. Studying three closed-access and
open-source LMs, we analyze the LMs not exclusively in terms of accuracy, but
also in terms of agreement with (imperfect, noisy, and sometimes biased) human
predictions or existing predictive models. We conduct experiments that assess
how providing different types of information, including distractor information
such as photos, can influence LM decisions. We also stress test techniques
designed to either increase accuracy or mitigate bias in LMs, and find that
some to have unintended consequences on LM decisions. Our results provide
additional quantitative evidence to the wisdom that current LMs are not the
right tools for these types of tasks.
展开英文信息
评论