序贯公共物品困境下的公平高效策略学习

Rational individuals can obtain higher rewards in the short term by defecting in social dilemmas, which, however, leads to low collective utility or even task failure. Many recent works have induced cooperative behaviors in social dilemmas though, they work only in stateless matrix games but fail in sequential social dilemmas. In tasks of sequential social dilemmas involving large number of players and complex states, cooperation is no longer simply one-step action and is hard to learn. Some works take payoffs equality into agents’ reward signals in decentralized multi-agent reinforcement learning to prevent some agents from taking up too much resources and starving others. However, this payoffs equality cannot lead to effective cooperative strategy, because it will force well-learned agents to sacrifice their high efficiency for equality if some agents have extremely low performance. In this work, we consider sequential public goods dilemmas in which group members can donate voluntarily for public welfare. We take fairness into account for training, well-learned agents obtain adequate rewards without being constrained by the policies of others, and meanwhile, the laggards have more access to learning owing to sufficient public goods. We empirically show that our method has excellent performance both in terms of collective efficiency and fairness. Compared to baselines, our agents acquire more universal and sustainable policies in sequential public goods dilemmas.