AI Goals and Rationality
Motivation
Before building or analyzing an AI system, you must specify what the system is trying to do. Without a precise goal there is no basis for choosing between designs, no way to judge whether the system is working, and no guarantee that optimizing the system’s objective actually produces the desired behavior. This article develops the standard AI framework for specifying goals: rationality as maximizing expected performance, performance measures as functions of environment trajectories, and the PEAS decomposition as a tool for making the specification explicit.
Rationality
The standard definition in AI is: a rational agent selects whichever action maximizes its expected performance given its percept history and built-in knowledge (Russell and Norvig 2020). Rationality is not omniscience — an agent cannot know outcomes that depend on unknowable future events — and it is not perfection; it is the best possible behavior given the information available.
Formal rationality requires four things:
- A performance measure that evaluates any sequence of environment states.
- Prior knowledge about the environment.
- The actions the agent can perform.
- The agent’s percept history to date.
Given these, a rational agent takes the action that maximizes the expected value of the performance measure.
Performance Measures
A performance measure assigns a score to the environment’s state trajectory, not to the agent’s internal states or beliefs. Getting this right is harder than it sounds:
- A vacuum cleaner scored on “dirt cleaned at each step” will clean and then dirty its environment to clean it again.
- A robot scored on “reaching the goal” with no penalty for collisions may damage property or people.
- An AI assistant scored on “user approval” may tell users what they want to hear rather than what is true.
Good performance measures capture the task designer’s actual goal — not a proxy that can be gamed. This is an instance of Goodhart’s Law: when a measure becomes a target, it ceases to be a good measure.
PEAS
A task environment is specified by four components (Russell and Norvig 2020):
| Component | Question |
|---|---|
| Performance measure | How is success defined? |
| Environment | What world does the agent live in? |
| Actuators | What actions can the agent take? |
| Sensors | What can the agent perceive? |
PEAS forces designers to be explicit. A taxi-driving agent: performance = safe, fast, legal, comfortable trips; environment = roads, traffic, pedestrians, weather; actuators = steering, gas, brake, signals; sensors = cameras, lidar, GPS, speedometer.
Task Environment Properties
Task environments vary along dimensions that determine which algorithms are applicable:
- Fully vs. partially observable. Can the agent see all relevant state? Partially observable environments require maintaining a belief state.
- Deterministic vs. stochastic. Does an action’s outcome follow deterministically from the current state and action? Stochastic environments require reasoning about probability distributions.
- Episodic vs. sequential. Are decisions independent across episodes, or does today’s action affect future options? Sequential environments require planning over time, as in Markov decision processes.
- Static vs. dynamic. Does the environment change while the agent deliberates? Dynamic environments penalize slow agents.
- Discrete vs. continuous. Are state and action spaces finite sets or continuous manifolds?
- Single-agent vs. multi-agent. Does the environment contain other agents whose goals may align or conflict? Adversarial settings call for minimax search.
Most real-world problems are partially observable, stochastic, sequential, dynamic, continuous, and multi-agent — a combination that makes them hard.