!! Under construction !!
Model Simulation
Once a formal model is given, it is possible to simulate it. Simulations can
- be informative on model behavior (test predictions, see the influence of parameters etc.)
- give a practical way to assess the invertability of a model (by inverting simulated data)
Different categories of models
Two families of models must be distinguished
- Models for which generated data is further used by the model
- Models for which generated data is not further used by the model
The first category is typical of models of learning from feedback. In such models, a decision is made. A feedback is then recieved followed by an update of hidden states depending on both the decision and the feedback.
The second category corresponds to models where learning (if there is any) is not dependent on a feedback that depends on model output (decisions).
Toolbox functions for model simulation
Important note : In both the simulation and inversion functions, EVOLUTION is performed before OBSERVATION. This is important because for models of learning from feedback, the two steps should be done the other way round (first observing, then updating)
For models with no learning, this has no consequences (hence, there are no hidden states, parameters are assumed to be fixed)
For models of learning from feedback, we use the following trick. The first call of the evolution function is replaced by an identity. Update for time t is performed at time t+1 from input at time t+1. This means that input must be temporally shifted (input for updating states at trial t must be put at position t+1)
The simulation therefore occurs as follows
- Trial 1, Identity EVOLUTION is performed. Which means x1 = x0 or in other words, hidden states are centered on initial states.
- Trial 1, OBSERVATION from initial states
- Trial 2, EVOLUTION from x1 to x2 using input of trial 2.
- Trial 2, OBSERVATION from x2
- Trial 3, etc
An example of use can be found applied to models of delay discounting (Demo_Delay.m)
Simulation, no feedback
Function to use is simulateNLSS.m
It is used as follows [y,x,x0,eta,e] = simulateNLSS(n_t,f_fname,g_fname,theta,phi,u,alpha,sigma,options,x0)
where
- n_t is the number of time samples
- f_fname : name of the evolution function
- g_fname : name of the observation function
- theta : evolution parameters
- phi : observation parameters
- u : predefined experimenter data
- alpha: precision of the stochastic innovations (no need here, we consider deterministic models)
- sigma: precision of the measurement error (only for continuous data)
An example of the use of this simulation function can be found in Demo_Delay.m
Simulation with feedback
Function to use is simulateNLSS_fb.m
It is used as follows [y,x,x0,eta,e,u] = simulateNLSS_fb(n_t,f_fname,g_fname,theta,phi,u,alpha,sigma,options,x0,fb)
It is built upon simulateNLSS.m. Two things change.
- output u is added. It provides experimenter data for further inversion
- input fb is added. This contains the information to apply identity to first trial, information on possible feedbacks at each time and information on where to find feedback from previous trial, and finaly information on how to build experimenter data for inversion
Additional input structure
The structure fb must be set as follows:
- fb.h_fname : handle of the function returning feedback from output
[fb] = h_fname(yt,t,in)
% INPUT
% - yt : generated data at time t
% - t : time of trial
% - in : structure containing potential feedback
% - u0 : potential feebacks
% OUTPUTS
% fb : recieved feedback
- fb.inH : input for function h
- fb.indy : indice of generated data that is used to select feedback through function h
- fb.indfb : ordered indices of where to store feedbacks in experimenter data.
Additional field in options
To implement the 'trick' consisting in applying identity evolution to the first trial, a field must be added to the option structure
Applying identity is equivalent to just skipping the update. As such it is named skipf. This field is a vector of size [1*number of trials]. It is by default set to zeros. Fields where evolution must be skipped is set to value one.
Therefore to just skip the first evolution step in a series on N trials :
options.skipf = zeros(1,n_t);
options.skipf(1) = 1; % apply identity mapping from x0 to x1.
An example of the use of this simulation function can be found in Demo_Qlearning_2Q.m.
Checking the quality of the inversion with simulated data
For synthetic data, posterior on parameters can be confronted to the real parameters that generated the data using :
% Q : simulated Qvalues
% Q0 : initial states of simulated Qvalues
% phi : real observation parameters
% theta : real evolution parameters
displayResults(posterior,out,y,Q,Q0,theta,phi,Inf,Inf)
Legend
- Top-left : Posterior over hidden states (mean and standard deviation)
- Dashed line : real / Full line : estimated
- Blue : first cue / Green : second cue
- Top-right : Probability of choosing action 1
- Dashed : actual subject choices / full : posterior probability of model choice
- Bottom-left : real Qvalues / mean of posterior on Qvalues
- Bottom-right : probability of choice 1 model vs data
- Data points are assembled in four bins based on model probability of choice
Legend
- Top-left : Posterior over the evolution parameter (learning rate)
- Grey Bar : mean of posterior / Read : Standard deviation of posterior
- Green dot : real value (Note that the plot is in the transformed space)
- Top-right : Posterior over the evolution parameter (learning rate)
- Grey Bar : mean of posterior / Read : Standard deviation of posterior
- Green dot : real value (Note that the plot is in the transformed space)
- Bottom-right : Posterior on initial Qvalues
They were here forced to zero.