Where the really hard choices are: A general framework to quantify decision difficulty

Abstract Current models of decision-making more often than not ignore the level of difficulty of choices or treat it only informally. Yet, difficulty has been shown to affect human decision quality. We propose instance complexity (IC), a measure of computational resource requirements, as a generalisable framework to quantify difficulty of a choice based on a small number of properties of the choice. The main advantage of IC compared to other measures of difficulty is fourfold. Firstly, it is based on the theory of computation, a rigorous mathematical framework. Secondly, our measure captures complexity that is intrinsic to a decision task, that is, it does not depend on a particular solution strategy or algorithm. Thirdly, it does not require knowledge of a decision-maker’s attitudes or preferences. And lastly, it allows computation of difficulty of a decision task ex-ante, that is, without solving the decision task. We tested the relation between IC and (i) decision quality and (ii) effort exerted in a decision using two variants of the 0-1 knapsack problem, a canonical and ubiquitous computational problem. We show that participants exerted more effort on instances with higher IC but that decision quality was lower in those instances. Together, our results suggest that IC can be used as a general framework to measure the inherent complexity of decision tasks and to quantify computational resource requirements of choices. The latter is particularly relevant for models of resource allocation in the brain (meta-decision-making/cognitive control). Our results also suggest that existing models of decision-making that are based on optimisation (rationality) as well as models such as the Bayesian Brain Hypothesis, are computationally implausible.