From: "Artificial Intelligence: A Modern Approach" by Stuart Russell and Peter Norvig (Chapter 16)
Utility functions are a cornerstone of rational agent design in AI. They provide a mathematical framework for encoding preferences over different states and outcomes, allowing agents to make optimal decisions even under uncertainty.
A utility function U(s) assigns a numerical value to each possible state s, representing the "desirability" of that state to the agent. The agent's goal is to maximize expected utility across all possible actions and their outcomes.
Consider a delivery robot choosing between two routes:
Expected utility calculation:
0.9 × U(10 min) + 0.1 × U(30 min)1.0 × U(15 min)The agent chooses the route with higher expected utility.
Goal-based agents operate on binary logic: either the goal is achieved or it isn't. But real-world scenarios involve:
For any two states s1 and s2, the agent either prefers s1, prefers s2, or is indifferent. No "undefined" preferences.
If an agent prefers s1 to s2 and s2 to s3, it must prefer s1 to s3. This prevents circular preferences.
If outcome A is strictly preferred to outcome B, then any lottery (probabilistic mix) that increases the chance of A should be preferred.
In enterprise AI systems, utility functions translate abstract business goals into quantifiable objectives:
U(state) = w1 × revenue(state)
+ w2 × customer_satisfaction(state)
- w3 × operational_cost(state)
- w4 × risk(state)
Where w1, w2, w3, w4 are weights determined by business priorities. Changing these weights changes agent behavior without rewriting the decision logic.
Utility functions are the bridge between human values and machine decisions. Poorly designed utility functions lead to misalignment—agents that optimize the wrong thing efficiently. The challenge isn't teaching AI to maximize utility; it's defining the right utility function.
"When a measure becomes a target, it ceases to be a good measure."
If you reward an AI for "customer engagement," it might spam users. If you reward "uptime," it might avoid necessary maintenance. Utility functions must capture true intent, not proxy metrics.
Linear utility functions assume that doubling revenue is twice as good. But diminishing returns are real—the 10th million dollars matters less than the first. Non-linear utility functions (logarithmic, sigmoid) better model human preferences.