Joint Optimization of AI Fairness and Utility: A Human-Centered Approach

Today, AI is increasingly being used in many high-stakes decision-making applications in which fairness is an important concern.Already, there are many examples of AI being biased and makingquestionable and unfair decisions. The AI research community hasproposed many methods to measure and mitigate unwanted biases,but few of them involve inputs from human policy makers. Weargue that because different fairness criteria sometimes cannotbe simultaneously satisfied, and because achieving fairness oftenrequires sacrificing other objectives such as model accuracy, it iskey to acquire and adhere to human policy makers’ preferenceson how to make the tradeoff among these objectives. In this paper,we propose a framework and some exemplar methods for elicitingsuch preferences and for optimizing an AI model according to thesepreferences.