Interactive replicator dynamics

The Interactive Replicator Dynamics

Department of Philosophy, Logic and Scientific Method
London School of Economics and Political Science

The replicator dynamics were first introduced by Taylor and Jonker (1978) to provide continuous dynamics for evolutionary game theory. The basic idea is that the frequency of a strategy in a population will increase, or decrease, depending on whether the fitness conferred by that strategy is better, or worse, than the average fitness of the population.

This program implements the discrete replicator dynamics for 3 strategies. Let s_i denote the proportion of the population which follows strategy i = 1, 2, or 3. (Note that it follows that 0 ≤ s_i ≤ 1 for all i and that s₁ + s₂ + s₃ = 1.)

Let U_i denote the expected utility of strategy i in the population. If we denote the payoff i receives when played against strategy j by σ_ij, then the value of U_i is computed as follows:

U_i = s₁ σ_i1 + s₂ σ_i2 + s₃ σ_i3.

The average utility of the population is simply the weighted average of these expected utilities. The weight used is simply the proportion of the population following a strategy:

UPOP = U₁ s₁ + U₂ s₂ + U₃ s₃.

The discrete replicator dynamics says that the proportion of the population following a strategy will change as follows:

s^’_i = s_i × ((U_i + β) / (UPOP + β)).

The constant β denotes the background fitness of the population (i.e., something like the default fecundity of individuals). Larger values of β cause the strategy frequencies of the population to change more slowly.

Sometimes we want to introduce a small amount of bias into how people choose others to interact with. (The default assumption is that all pairwise combinations are equally likely.) That is, sometimes we want a person’s partner to be correlated with the strategy used by a person. It is possible — although I won't include the details here as the notation is slightly messier — to introduce correlation into the above equations. A correlation coefficient of ε = 1 means that people only interact with people following the same strategy as themselves; a correlation coefficient of ε = 0 means that interactions are not biased at all. Values between 0 and 1 represent bias of increasingly greater degree.

How to use it

Click on "Setup" to initialise the simplex. (If you want to delete all the plotted paths, simply click on "Setup" again.) Once the model has been initialised, you need to click on "Go" to begin plotting paths. As long as the "Go" button is depressed, you can interact with the simplex.

Click anywhere inside the simplex to identify the initial conditions for an evolutionary trajectory. After clicking, the discrete replicator dynamics will be run and the evolutionary trajectory determined. A path will then be drawn inside the simplex showing how the population will change over time.

If you want to change the starting conditions of a path, simply click on the blue dot representing the initial condition and drag it somewhere else. If you want to delete a path, drag the dot outside of the simplex and release it.

All of the values are initialised to reasonable defaults. Here’s what the variables mean:

Parameter	Description
correlation	The correlation coefficient ε. If ε = 0, no bias exists; if ε = 1, players only interact with their own kind.
max-generations	The number of generations to compute before drawing the path. If the value of background-fitness is high, then the strategy frequencies change by a smaller amount each generation, and so it may be necessary to increase the number of generations to see the evolutionary trajectory.
arrow-position	Approximately where along the path to place an arrowhead denote the direction of the trajectory.
background-fitness	The background fitness β. Larger values mean less dramatic change from one generation to the next.
check-for-polymorphism?	This is just an efficiency device. If On, it means that whenever two strategies take up more than 99.99% of the population, it is assumed that the simulation will converge to a mixed state containing just those two strategies. The run will be aborted at that point even if we haven’t reached the maximum number of generations yet.
base-game	A few interesting games are pre-defined. You can enter custom values for the payoff matrix, as well.

System Requirements

The applet requires Java 5 or higher. Java must be enabled in your browser settings. Mac users must have Mac OS X 10.4 or higher. Windows and Linux users may obtain the latest Java from Sun's Java site.

Procedures

extensions [ table ]

globals [
  x1 
  y1
  x2
  y2
  x3
  y3 
  path-table
  previous-correlation
  previous-base-game
  previous-arrow-position
  previous-background-fitness
]

to setup
  clear-all
  set previous-correlation correlation
  set previous-base-game base-game
  set previous-arrow-position arrow-position
  set previous-background-fitness background-fitness
  
  ask patches [ set pcolor [ 230 230 230 ] ]
  set path-table table:make
  set x1 0
  set y1 0
  create-turtles 1 [
    setxy x1 y1
    set size 0.1
    set heading 0
    set color black
    set pen-size 3
    pen-down
    rt 30
    fd 20 * (2 / (sqrt 3))
    set x2 xcor
    set y2 ycor
    rt 120
    fd 20 * (2 / (sqrt 3))
    set x3 xcor 
    set y3 ycor
    rt 120
    fd 20 * (2 / (sqrt 3))
    die
  ]
  create-turtles 1 [
    setxy (x1 - 0.1) (y1 - 0.4)
    set label-color black
    set label "S1"
    set size 0 
  ]
  create-turtles 1 [
    setxy (x2 + 0.4) y2
    set label-color black
    set label "S2"
    set size 0 
  ]
  create-turtles 1 [
    setxy (x3 + 1) (y3 - 0.4)
    set label-color black
    set label "S3"
    set size 0 
  ]
end

to show-initial-conditions
  if mouse-down? [
     show (to-initial-condition mouse-xcor mouse-ycor )
  ]
end

to-report to-initial-condition [ x y ]
  let det (y2 - y3) * (x1 - x3) + (x3 - x2) * (y1 - y3)
  let ic1 ((y2 - y3) * (x - x3) + (x3 - x2) * (y - y3)) / det
  let ic2 ((y3 - y1) * (x - x3) + (x1 - x3) * (y - y3)) / det
  let ic3 (1 - ic1 - ic2)
  
  report (list ic1 ic2 ic3)
end

to-report to-xy [ ic ]
  let ic1 (item 0 ic)
  let ic2 (item 1 ic)
  let ic3 (item 2 ic)
  let x (ic1 * x1 + ic2 * x2 + ic3 * x3)
  let y (ic1 * y1 + ic2 * y2 + ic3 * y3)
  report (list x y)
end

to refresh-display
  clear-drawing
  create-turtles 1 [
    setxy x1 y1
    set size 0
    set heading 0
    set color black
    set pen-size 3
    pen-down
    rt 30
    fd 20 * (2 / (sqrt 3))
    rt 120
    fd 20 * (2 / (sqrt 3))
    rt 120
    fd 20 * (2 / (sqrt 3))
    die
  ]
  let keys table:keys path-table
  foreach keys [
    let path table:get path-table ?
    draw-line path 
  ]
  display
end

to recalculate-paths 
  let keys table:keys path-table
  foreach keys [
    let ic (to-initial-condition (item 0 ?) (item 1 ?))
    let path (replicator-dynamics ic)
    table:put path-table ? path
  ]
  refresh-display
end

to check-background-fitness
  if (previous-background-fitness != background-fitness) [
    set previous-background-fitness background-fitness
    recalculate-paths 
  ]
end

to check-arrow-position
  if (previous-arrow-position != arrow-position) [
    set previous-arrow-position arrow-position
    recalculate-paths 
  ]
end

to check-correlation
  if (previous-correlation != correlation) [
    set previous-correlation correlation
    recalculate-paths
  ]
end

to check-base-game
  if (previous-base-game != base-game) [
    set previous-base-game base-game
    if base-game = "Rock-scissors-paper" [
      set pm0-0 1  set pm0-1 (2 + 1 / background-fitness)  set pm0-2 0
      set pm1-0 0  set pm1-1 1  set pm1-2 (2 + 1 / background-fitness)
      set pm2-0 (2 + 1 / background-fitness)  set pm2-1 0  set pm2-2 1
    ]
    if base-game = "Rock-scissors-paper (spiral in)" [
      set pm0-0 1  set pm0-1 2.5  set pm0-2 0
      set pm1-0 0  set pm1-1 1  set pm1-2 2.5
      set pm2-0 2.5  set pm2-1 0  set pm2-2 1
    ]
    if base-game = "Rock-scissors-paper (spiral out)" [
      set pm0-0 1  set pm0-1 1.5  set pm0-2 0
      set pm1-0 0  set pm1-1 1  set pm1-2 1.5
      set pm2-0 1.5  set pm2-1 0  set pm2-2 1
    ]
    if base-game = "4-5-6 divide the cake" [
      set pm0-0 4  set pm0-1 4  set pm0-2 4
      set pm1-0 5  set pm1-1 5  set pm1-2 0
      set pm2-0 6  set pm2-1 0  set pm2-2 0
    ]
    if base-game = "Preservation of weak dominance" [
      set pm0-0 1  set pm0-1 1  set pm0-2 1
      set pm1-0 1  set pm1-1 1  set pm1-2 0
      set pm2-0 0  set pm2-1 0  set pm2-2 0
    ]
    if base-game = "Selection of Lyapunov unstable strategy" [
      set pm0-0 0  set pm0-1 1  set pm0-2 0
      set pm1-0 0  set pm1-1 0  set pm1-2 2
      set pm2-0 0  set pm2-1 0  set pm2-2 1      
    ]
    if base-game = "Asymptotically stable, but not neutrally stable" [
      set pm0-0 1  set pm0-1 5  set pm0-2 0
      set pm1-0 0  set pm1-1 1  set pm1-2 5
      set pm2-0 5  set pm2-1 0  set pm2-2 4            
    ]
    
    recalculate-paths
    refresh-display
  ]
  
end

to go
  check-correlation
  check-base-game
  check-arrow-position
  check-background-fitness
  
  if mouse-down? [    
    let candidate min-one-of turtles [distancexy mouse-xcor mouse-ycor]
    ifelse candidate != nobody and [distancexy mouse-xcor mouse-ycor] of candidate < 0.25 [
      let key (list [xcor] of candidate [ycor] of candidate)
      
      if (table:has-key? path-table key) [
        table:remove path-table key
        refresh-display
      ]
        
      let xdelta ([xcor] of candidate - mouse-xcor)
      let ydelta ([ycor] of candidate - mouse-ycor)
      
      ;; The WATCH primitive puts a "halo" around the watched turtle.
      ;watch candidate
      let last-xcor 0
      let last-ycor 0
           
      while [mouse-down?] [
        ;; If we don't force the display to update, the user won't
        ;; be able to see the turtle moving around.
        display
        ;; The SUBJECT primitive reports the turtle being watched.     
        ask candidate [ setxy (mouse-xcor + xdelta) (mouse-ycor + ydelta)]
        set last-xcor (mouse-xcor + xdelta)
        set last-ycor (mouse-ycor + ydelta)
      
      ]
      
      let ic (to-initial-condition last-xcor last-ycor)
      ifelse not inside? ic 
        [ ask candidate [ die ] ]
        [ 
          let path (replicator-dynamics ic)
          draw-line path
          table:put path-table (list last-xcor last-ycor) path 
        ]
        
      ;; Undoes the effects of WATCH.  Can be abbreviated RP.
      ;reset-perspective
    ]
    [
      create-turtles 1 [
        setxy mouse-xcor mouse-ycor
        set shape "circle" 
        set color blue
        set size 0.25
      ] 
    ]
  ]  
end

to erase-line [ states ]
  let ic first states
  let pt to-xy ic
  let plotter nobody
  create-turtles 1 [
    set plotter self
    set size 0
    set color black
    set xcor (item 0 pt)
    set ycor (item 1 pt)
    pen-down
    pen-erase
  ]
  
  foreach but-first states [
    ask plotter [
      set pt to-xy ?
      setxy (item 0 pt) (item 1 pt) 
    ]  
  ]
  
  ask plotter [ die ]
end

to draw-line [ states ]
  let ic first states
  let len (length states)
  let stamp-here (round (len * arrow-position))
  
  let pt to-xy ic
  let plotter nobody
  create-turtles 1 [
    set plotter self
    set size 0
    set color black
    set xcor (item 0 pt)
    set ycor (item 1 pt)
    pen-down
  ]
  
  let i 0
  let stamped false
  let previous-pt []
  
  foreach but-first states [
    ask plotter [
      set previous-pt pt
      set pt to-xy ?
      setxy (item 0 pt) (item 1 pt) 
      if (i = stamp-here) 
      [
        set size 0.5
        set heading atan ((item 0 pt) - (item 0 previous-pt)) ((item 1 pt) - (item 1 previous-pt))
        stamp
        set size 0
        set stamped true
      ]
      set i (i + 1)     
    ]  
  ]
  
  ask plotter [ die ]
end

to-report inside? [ ic ]
  let ic1 (item 0 ic)
  let ic2 (item 1 ic)
  let ic3 (item 2 ic)
  report 0 <= ic1 and ic1 <= 1
         and 0 <= ic2 and ic2 <= 1
         and 0 <= ic3 and ic3 <= 1
end

to-report replicator-dynamics [ ic ]
  let P (list (item 0 ic) (item 1 ic) (item 2 ic) )

  let pm (list (list pm0-0 pm0-1 pm0-2)
               (list pm1-0 pm1-1 pm1-2)
               (list pm2-0 pm2-1 pm2-2))
  
  
  let converged false
  let gen 0
  let states (list ic )
  
  while [ gen < max-generations and not converged ] 
  [
    let U [ 0 0 0 ]
    
    let i 0    
    while [ i < 3 ] [
      
      let j 0
      while [ j < 3 ] [
        
        let uvalue (item i U) 
        let pvalue (item j P)
        let pmvalue (item j (item i pm))
        
;        set uvalue (uvalue + pmvalue * pvalue)
;        set U (replace-item i U uvalue)
        ifelse (i = j) 
        [
          set uvalue (uvalue + pmvalue * (pvalue + correlation * (1 - pvalue)) )
          set U (replace-item i U uvalue)
        ]
        [
          set uvalue (uvalue + pmvalue * (pvalue - correlation * pvalue) )
          set U (replace-item i U uvalue)
        ]


        set j (j + 1) 
      ]
      set i (i + 1)  
    ]
        
    let USQ 0
    
    foreach [ 0 1 2 ] [
      set USQ (USQ + (item ? P) * (item ? U))
    ]
    
    foreach [ 0 1 2 ] [
      ;let background-fitness 40
      let pvalue ((item ? P ) * ( ((item ? U) + background-fitness) / (USQ + background-fitness)))
      set P (replace-item ? P pvalue)
    ]
  
    set states (lput P states)
    set gen (gen + 1)
    
    if (item 0 P) = 0.9999 or (item 1 P) = 0.9999 or (item 2 P) = 0.9999 
      [ set converged true ]
      
    if (check-for-polymorphisms? = true) 
    [
      if (item 0 P) + (item 1 P) >= 0.9999
         or
         (item 0 P) + (item 2 P) >= 0.9999
         or
         (item 1 P) + (item 2 P) >= 0.9999 
      [
        set converged true 
      ]
    ]
  ]
  report states
end

The Interactive Replicator Dynamics

Dr J McKenzie Alexander