My favorites | Sign in
Project Home
READ-ONLY: This project has been archived. For more information see this post.
Search
for
Extensions  
Updated Nov 7, 2013 by jean.dau...@gmail.com

!! Under construction !!

Extension to multiple session and multiple updates

Dealing with multiple sessions

Until now we have defined an evolution function to account for a single time series (a single session).

In practice, subjects perform multiple independent sessions of a same task. One could therefore wish to assume that the subjects use the same model for the different sessions with possible variation of some of the used parameters if sessions correspond to different conditions.

Defining a single model generating all sessions makes it possible to use information at best to infer each parameter (wether shared accross sessions or not)

The previous framework is extended to deal with multiple sessions. This is done the following way:

  • An evolution function is created that tracks the hidden states of all sessions together
  • An observation function generates the data for all sessions simultaneously from the extended hidden states leading to an extended data vector

Practically and Extended model (EM) is created from an initial model (IM). To generate data, the evolution and observation functions of the EM use experimenter data to call the IM's functions with the correct arguments (correct indices of hidden states, parameters) and rooting of outputs.

Behavioral data from the different sessions must be concatenated. Experimenter data from the different sessions must also be concatenated. Additional information about sessions is requested.

I here present a script to perform the inversion of a reinforcement learning model on the data of multiple sessions of an operant learning task.

Organizing data

We concatenate separately the experimenter data, the output data and the hidden states of each session in the order of the session rank. No change for the parameters that we assume to be the same for each session. There is a single output data (choice) that does not require any ordering.

Defining the Extended model

To define the EM, one needs

  • definition of the IM
  • definition of the parameter space of the EM
  • definition of the use of the parameters of the EM

This is done as follows

% future dimensions of the extended model
dim_e = struct('n_theta',1,... % specify parameter space
             'n_phi', 1);%      

% Information for sessions
in_sessions = struct();
in_sessions.n_sess = Nsessions; % number of sessions
in_sessions.f_fname = @f_Qlearn_2Q; % handle of the shared evolution function
in_sessions.g_fname = @g_softmax_2Q; % handle of the shared observation function
in_sessions.dim_e = dim_e; % specify extended model s parameter space
in_sessions.binomial = 1;
in_sessions.inF = [];
in_sessions.inG = [];
in_sessions.inH = Rewards; % for simulation only
in_sessions.fb = fb; % for simulation only

in_sessions.ind.theta =  ones(Nsessions,1)*[1]; % specify index of parameter to use for each session. 
%size: Nsessions x Npara for each sessions 
%Here NparaEvolution =1 + the same parameter is used (parameter of index 1)
% for all sessions, if the evolution function had two paramaters and the same
%parameters should be used for all sessions it would be: ones(Nsessions,1)*[1,2]
in_sessions.ind.phi =   ones(Nsessions,1)*[1]; % specify index of parameter to
%use for each session. 

This is enough to create the extended model using the following function

[ f_fname_e,g_fname_e,dim_e,options_e ] = makeExtendedModel(dim,options,in_sessions);

Setting priors for inversion

Priors must be set so to comply with those new dimensions.

% Defining Priors
% Priors on parameters (mean and Covariance matrix)
priors.muPhi = zeros(dim.n_phi,1); 
priors.SigmaPhi = 1e2*eye(dim.n_phi);
% No state noise for deterministic update rules
priors.a_alpha = Inf;
priors.b_alpha = 0;
options.priors = priors;

Inverting the sessions together

Inversion is now performed with the extended model

[posterior,out] = VBA_NLStateSpaceModel(y,u, f_fname_e,g_fname_e,dim_e,options_e);

Results of inversion for multiple sessions

The same example used for reinforcement learning on multiple sessions (with the same simulation parameters)

Here, is plotted the VB_inversion part of the visual output.

  • Top-left : posterior predictive density for all sessions
  • Middle-left : posterior on hidden states for all sessions
  • Bottom : parameter estimaties are accurate. Posterior variance on parameters is lower than on the single session example (there is indeed more information from the multiple sessions)

Dealing with multiple updates within a trial

In some models, many hidden states can be updated during one trial. To do so, one must just define an evolution function that updates several hidden states. The critical point is to define well the indices of which hidden state should be updated either in the evolution function or in experimenter data.

Plotting inversion results for each session individually

Inverting over multiple sessions corresponds to inverting a single model with multiple outputs. As such, standard display functions of the toolbox will display sessions altogether making figures messy and difficult to read.

It is however possible the result of the inversion (predictive density, posterior on hidden states) for each session or for groups of sessions.

This can be done by calling

VBA_ReDisplay_sessions(posterior,out,group_sessions)

Where group_sessions is a cell of arrays of indices of sessions. If group_sessions isn't given or set to , no groups are considered.

Powered by Google Project Hosting