|国家预印本平台
首页|Generative Models, Humans, Predictive Models: Who Is Worse at High-Stakes Decision Making?

Generative Models, Humans, Predictive Models: Who Is Worse at High-Stakes Decision Making?

Generative Models, Humans, Predictive Models: Who Is Worse at High-Stakes Decision Making?

来源:Arxiv_logoArxiv
英文摘要

Despite strong advisory against it, large generative models (LMs) are already being used for decision making tasks that were previously done by predictive models or humans. We put popular LMs to the test in a high-stakes decision making task: recidivism prediction. Studying three closed-access and open-source LMs, we analyze the LMs not exclusively in terms of accuracy, but also in terms of agreement with (imperfect, noisy, and sometimes biased) human predictions or existing predictive models. We conduct experiments that assess how providing different types of information, including distractor information such as photos, can influence LM decisions. We also stress test techniques designed to either increase accuracy or mitigate bias in LMs, and find that some to have unintended consequences on LM decisions. Our results provide additional quantitative evidence to the wisdom that current LMs are not the right tools for these types of tasks.

Keri Mallari、Julius Adebayo、Kori Inkpen、Martin T. Wells、Albert Gordo、Sarah Tan

计算机与自动化

Keri Mallari,Julius Adebayo,Kori Inkpen,Martin T. Wells,Albert Gordo,Sarah Tan.Generative Models, Humans, Predictive Models: Who Is Worse at High-Stakes Decision Making?[EB/OL].(2025-02-14)[2025-03-14].https://arxiv.org/abs/2410.15471.点此复制

评论