Extensive form game - AbsoluteAstronomy.com

An extensive-form game is a specification of a game in game theory

Game theory

Game theory is a mathematical method for analyzing calculated circumstances, such as in games, where a person’s success is based upon the choices of others...

, allowing (as the name suggests) explicit representation of a number of important aspects, like the sequencing of players' possible moves, their choices at every decision point, the (possibly imperfect) information each player has about the other player's moves when he makes a decision, and his payoffs for all possible game outcomes. Extensive-form games also allow representation of incomplete information in the form of chance events encoded as "moves by nature".

Finite extensive-form games

Some authors, particularly in introductory textbooks, initially define the extensive-form game as being just a game tree

Game tree

In game theory, a game tree is a directed graph whose nodes are positions in a game and whose edges are moves. The complete game tree for a game is the game tree starting at the initial position and containing all possible moves from each position; the complete tree is the same tree as that...

with payoffs (no imperfect or incomplete information), and add the other elements in subsequent chapters as refinements. Whereas the rest of this article follows this gentle approach with motivating examples, we present upfront the finite extensive-form games as (ultimately) constructed here. This general definition was introduced by Harold W. Kuhn

Harold W. Kuhn

Harold William Kuhn is an American mathematician who studied game theory. He won the 1980 John von Neumann Theory Prize along with David Gale and Albert W. Tucker...

in 1953, who extended an earlier definition of von Neumann from 1928. Following the presentation from , an n-player extensive-form game thus consists of the following:

A finite set of n (rational) players
A rooted tree, called the game tree
Each terminal (leaf) node of the game tree has an n-tuple
Tuple
In mathematics and computer science, a tuple is an ordered list of elements. In set theory, an n-tuple is a sequence of n elements, where n is a positive integer. There is also one 0-tuple, an empty sequence. An n-tuple is defined inductively using the construction of an ordered pair...

of payoffs, meaning there is one payoff for each player at the end of every possible play
A partition
Partition of a set
In mathematics, a partition of a set X is a division of X into non-overlapping and non-empty "parts" or "blocks" or "cells" that cover all of X...

of the non-terminal nodes of the game tree in n+1 subsets, one for each (rational) player, and with a special subset for a fictitious player called Chance (or Nature). Each player's subset of nodes is referred to as the "nodes of the player". (A game of complete information thus has an empty set of Chance nodes.)
Each node of the Chance player has a probability distribution
Probability distribution
In probability theory, a probability mass, probability density, or probability distribution is a function that describes the probability of a random variable taking certain values....

over its outgoing edges.
Each set of nodes of a rational player is further partitioned in information set
Information set
In game theory, an information set is a set that, for a particular player, establishes all the possible moves that could have taken place in the game so far, given what that player has observed. If the game has perfect information, every information set contains only one member, namely the point...

s, which make certain choices indistinguishable for the player when making a move, in the sense that:
- there is a one-to-one correspondence between outgoing edges of any two nodes of the same information set—thus the set of all outgoing edges of an information set is partitioned in equivalence classes, each class representing a possible choice for a player's move at some point—, and
- every (directed) path in the tree from the root to a terminal node can cross each information set at most once
the complete description of the game specified by the above parameters is common knowledge
Common knowledge (logic)
Common knowledge is a special kind of knowledge for a group of agents. There is common knowledge of p in a group of agents G when all the agents in G know p, they all know that they know p, they all know that they all know that they know p, and so on ad infinitum.The concept was first introduced in...

among the players

A play is thus a path through the tree from the root to a terminal node. At any given non-terminal node belonging to Chance, an outgoing branch is chosen according to the probability distribution. At any rational player's node, the player must choose one of the equivalence classes for the edges, which determines precisely one outgoing edge except (in general) the player doesn't know which one is being followed. (An outside observer knowing every other player's choices up to that point, and the realization

Realization (probability)

In probability and statistics, a realization, or observed value, of a random variable is the value that is actually observed . The random variable itself should be thought of as the process how the observation comes about...

of Nature's moves, can determine the edge precisely.) A pure strategy for a player thus consists of a selection—choosing precisely one class of outgoing edges for every information set (of his). In a game of perfect information, the information sets are singletons. It's less evident how payoffs should be interpreted in games with Chance nodes. It is assumed that each player has a von Neumann–Morgenstern utility function defined for every game outcome; this assumption entails that every rational player will evaluate an a priori

A priori

A priori is Latin for "from the former" or "from before", and may refer to:* A priori knowledge, justification or arguments. See a priori and a posteriori.* A priori , a type of constructed language...

random outcome by its expected

Expected value

In probability theory, the expected value of a random variable is the weighted average of all possible values that this random variable can take on...

utility.
The above presentation, while precisely defining the mathematical structure over which the game is played, elides however the more technical discussion of formalizing statements about how the game is played like "a player cannot distinguish between nodes in the same information set when making a decision". These can be made precise using epistemic modal logic; see for details.

A perfect information two-player game over a game tree

Game tree

(as defined in combinatorial game theory

Combinatorial game theory

Combinatorial game theory is a branch of applied mathematics and theoretical computer science that studies sequential games with perfect information, that is, two-player games which have a position in which the players take turns changing in defined ways or moves to achieve a defined winning...

and artificial intelligence

Artificial intelligence

Artificial intelligence is the intelligence of machines and the branch of computer science that aims to create it. AI textbooks define the field as "the study and design of intelligent agents" where an intelligent agent is a system that perceives its environment and takes actions that maximize its...

), for instance chess

Chess

Chess is a two-player board game played on a chessboard, a square-checkered board with 64 squares arranged in an eight-by-eight grid. It is one of the world's most popular games, played by millions of people worldwide at home, in clubs, online, by correspondence, and in tournaments.Each player...

, can be represented as an extensive form game as defined with the same game tree and the obvious payoffs for win/lose/draw

Draw (chess)

In chess, a draw is when a game ends in a tie. It is one of the possible outcomes of a game, along with a win for White and a win for Black . Usually, in tournaments a draw is worth a half point to each player, while a win is worth one point to the victor and none to the loser.For the most part,...

outcomes. A game over an expectminimax tree, like that of backgammon

Backgammon

Backgammon is one of the oldest board games for two players. The playing pieces are moved according to the roll of dice, and players win by removing all of their pieces from the board. There are many variants of backgammon, most of which share common traits...

, has no imperfect information (all information sets are singletons) but has Chance moves. As further examples, various variants of poker

Poker

Poker is a family of card games that share betting rules and usually hand rankings. Poker games differ in how the cards are dealt, how hands may be formed, whether the high or low hand wins the pot in a showdown , limits on bet sizes, and how many rounds of betting are allowed.In most modern poker...

have both chance moves (the cards being dealt, initially and possibly subsequently depending on the poker variant, e.g. in draw poker

Draw poker

Draw poker is any poker variant in which each player is dealt a complete hand before the first betting round, and then develops the hand for later rounds by replacing, or "drawing", cards....

there are additional Chance nodes besides the initial one), and also have imperfect information (some or all the cards held by other players, again depending on the Poker variant; see community card poker

Community card poker

Community card poker refers to any game of poker that uses community cards , which are cards dealt face up in the center of the table and shared by all players. In these games, each player is dealt privately an incomplete hand , which are then combined with the community cards to make a complete...

Perfect and complete information

A complete extensive-form representation specifies:

the players of a game
for every player every opportunity they have to move
what each player can do at each of their moves
what each player knows for every move
the payoffs received by every player for every possible combination of moves

The game on the right has two players: 1 and 2. The numbers by every non-terminal node indicate to which player that decision node belongs. The numbers by every terminal node represent the payoffs to the players (e.g. 2,1 represents a payoff of 2 to player 1 and a payoff of 1 to player 2). The labels by every edge of the graph are the name of the action that that edge represents.

The initial node belongs to player 1, indicating that player 1 moves first. Play according to the tree is as follows: player 1 chooses between U and D; player 2 observes player 1's choice and then chooses between U' and D' . The payoffs are as specified in the tree. There are four outcomes represented by the four terminal nodes of the tree: (U,U'), (U,D'), (D,U') and (D,D'). The payoffs associated with each outcome respectively are as follows (0,0), (2,1), (1,2) and (3,1).

If player 1 plays D, player 2 will play U' to maximise his payoff and so player 1 will only receive 1. However, if player 1 plays U, player 2 maximises his payoff by playing D' and player 1 receives 2. Player 1 prefers 2 to 1 and so will play U and player 2 will play D' . This is the subgame perfect equilibrium

Subgame perfect equilibrium

In game theory, a subgame perfect equilibrium is a refinement of a Nash equilibrium used in dynamic games. A strategy profile is a subgame perfect equilibrium if it represents a Nash equilibrium of every subgame of the original game...

Imperfect information

An advantage of representing the game in this way is that it is clear what the order of play is. The tree shows clearly that player 1 moves first and player 2 observes this move. However, in some games play does not occur like this. One player does not always observe the choice of another (for example, moves may be simultaneous or a move may be hidden). An information set is a set of decision nodes such that:

Every node in the set belongs to one player.
When play reaches the information set, the player with the move cannot differentiate between nodes within the information set; i.e. if the information set contains more than one node, the player to whom that set belongs does not know which node in the set has been reached.

In extensive form, an information set is indicated by a dotted line connecting all nodes in that set or sometimes by a loop drawn around all the nodes in that set.

If a game has an information set with more than one member that game is said to have imperfect information. A game with perfect information is such that at any stage of the game, every player knows exactly what has taken place earlier in the game; i.e. every information set is a singleton set. Any game without perfect information has imperfect information.

The game on the left is the same as the above game except that player 2 does not know what player 1 does when he comes to play. The first game described has perfect information; the game on the left does not. If both players are rational and both know that both players are rational and everything that is known by any player is known to be known by every player (i.e. player 1 knows player 2 knows that player 1 is rational and player 2 knows this, etc. ad infinitum), play in the first game will be as follows: player 1 knows that if he plays U, player 2 will play D' (because for player 2 a payoff of 1 is preferable to a payoff of 0) and so player 1 will receive 2. However, if player 1 plays D, player 2 will play U' (because to player 2 a payoff of 2 is better than a payoff of 1) and player 1 will receive 1. Hence, in the first game, the equilibrium will be (U, D' ) because player 1 prefers to receive 2 to 1 and so will play U and so player 2 will play D' .

In the second game it is less clear: player 2 cannot observe player 1's move. Player 1 would like to fool player 2 into thinking he has played U when he has actually played D so that player 2 will play D' and player 1 will receive 3. In fact in the second game there is a perfect Bayesian equilibrium where player 1 plays D and player 2 plays U' and player 2 holds the belief that player 1 will definitely play D. In this equilibrium, every strategy is rational given the beliefs held and every belief is consistent with the strategies played. Notice how the imperfection of information changes the outcome of the game.

In games with infinite action spaces and imperfect information, non-singleton information sets are represented, if necessary, by inserting a dotted line connecting the (non-nodal) endpoints behind the arc described above or by dashing the arc itself. In the Stackelberg game described above, if the second player had not observed the first player's move the game would no longer fit the Stackelberg model; it would be Cournot competition

Cournot competition

Cournot competition is an economic model used to describe an industry structure in which companies compete on the amount of output they will produce, which they decide on independently of each other and at the same time. It is named after Antoine Augustin Cournot who was inspired by observing...

Incomplete information

It may be the case that a player does not know exactly what the payoffs of the game are or of what type his opponents are. This sort of game has incomplete information. In extensive form it is represented as a game with complete but imperfect information using the so called Harsanyi
John Harsanyi
John Charles Harsanyi was a Hungarian-Australian-American economist and Nobel Memorial Prize in Economic Sciences winner....

transformation. This transformation introduces to the game the notion of nature's choice
Move by nature
In game theory a move by nature is a decision or move in an extensive form game made by a player who has no strategic interests in the outcome. The effect is to add a player, 'Nature' whose practical role is to act as a random number generator...

or God's choice. Consider a game consisting of an employer considering whether to hire a job applicant. The job applicant's ability might be one of two things: high or low. His ability level is random; he is low ability with probability 1/3 and high ability with probability 2/3. In this case, it is convenient to model nature as another player of sorts who chooses the applicant's ability according to those probabilities. Nature however does not have any payoffs. Nature's choice is represented in the game tree by a non-filled node. Edges coming from a nature's choice node are labelled with the probability of the event it represents occurring.

The game on the left is one of complete information (all the players and payoffs are known to everyone) but of imperfect information (the employer doesn't know what was nature's move.) The initial node is in the centre and it is not filled, so nature moves first. Nature selects with the same probability the type of player 1 (which in this game is tantamount to selecting the payoffs in the subgame played), either t1 or t2. Player 1 has distinct information sets for these; i.e. player 1 knows what type he is (this need not be the case). However, player 2 does not observe nature's choice. He does not know the type of player 1; however, in this game he does observe player 1's actions; i.e. there is perfect information. Indeed, it is now appropriate to alter the above definition of perfect information: at every stage in the game, every player knows what has been played by the other players. In the case of complete information, every player knows what has been played by nature. Information sets are represented as before by broken lines.

In this game, if nature selects t1 as player 1's type, the game played will be like the very first game described, except that player 2 does not know it (and the very fact that this cuts through his information sets disqualify it from subgame

Subgame

In game theory, a subgame is any part of a game that meets the following criteria :#It has a single initial node that is the only member of that node's information set In game theory, a subgame is any part (a subset) of a game that meets the following criteria (the following terms allude to a game...

status). There is one separating perfect Bayesian equilibrium; i.e. an equilibrium in which different types do different things.

If both types play the same action (pooling), an equilibrium cannot be sustained. If both play D, player 2 can only form the belief that he is on either node in the information set with probability 1/2 (because this is the chance of seeing either type). Player 2 maximises his payoff by playing D' . However, if he plays D' , type 2 would prefer to play U. This cannot be an equilibrium. If both types play U, player 2 again forms the belief that he is at either node with probability 1/2. In this case player 2 plays D' , but then type 1 prefers to play D.

If type 1 plays U and type 2 plays D, player 2 will play D' whatever action he observes, but then type 1 prefers D. The only equilibrium hence is with type 1 playing D, type 2 playing U and player 2 playing U' if he observes D and randomising if he observes U. Through his actions, player 1 has signalled his type to player 2.

Formal definition

Formally, a finite game in extensive form is a structure

where:

is a finite tree with a set of nodes , a unique initial node , a set of terminal nodes (let be a set of decision nodes) and an immediate predecessor function on which the rules of the game are represented,
is a partition of called an information partition,
is a set of actions available for each information set which forms a partition on the set of all actions .
is an action partition corresponding each edge to a single action , fulfilling:

, restriction

is a bijection.

is a finite set of players, is (a special player called) nature, and is a player partition of information set . Let be a single player that makes a move at node .
is a family of probabilities of the actions of nature, and
is a payoff profile function.

Infinite action space

It may be that a player has an infinite number of possible actions to choose from at a particular decision node. The device used to represent this is an arc joining two edges protruding from the decision node in question. If the action space is a continuum between two numbers, the lower and upper delimiting numbers are placed at the bottom and top of the arc respectively, usually with a variable that is used to express the payoffs. The infinite number of decision nodes that could result are represented by a single node placed in the centre of the arc. A similar device is used to represent action spaces that, whilst not infinite, are large enough to prove impractical to represent with an edge for each action.

The tree on the left represents such a game, either with infinite action spaces (any real number

Real number

In mathematics, a real number is a value that represents a quantity along a continuum, such as -5 , 4/3 , 8.6 , √2 and π...

between 0 and 5000) or with very large action spaces (perhaps any integer

Integer

The integers are formed by the natural numbers together with the negatives of the non-zero natural numbers .They are known as Positive and Negative Integers respectively...

between 0 and 5000). This would be specified elsewhere. Here, it will be supposed that it is the latter and, for concreteness, it will be supposed it represents two firms engaged in Stackelberg competition

Stackelberg competition

The Stackelberg leadership model is a strategic game in economics in which the leader firm moves first and then the follower firms move sequentially...

. The payoffs to the firms are represented on the left, with q1 and q2 as the strategy they adopt and c1 and c2 as some constants (here marginal costs to each firm). The subgame perfect Nash equilibria of this game can be found by taking the first partial derivative

Partial derivative

In mathematics, a partial derivative of a function of several variables is its derivative with respect to one of those variables, with the others held constant...

(reference?) of each payoff function with respect to the follower's (firm 2) strategy variable (q2) and finding its best response

Best response

In game theory, the best response is the strategy which produces the most favorable outcome for a player, taking other players' strategies as given...

function,

. The same process can be done for the leader except that in calculating its profit, it knows that firm 2 will play the above response and so this can be substituted into its maximisation problem. It can then solve for q1 by taking the first derivative, yielding

. Feeding this into firm 2's best response function,

and (q1*,q2*) is the subgame perfect Nash equilibrium.