Static models with softmax decision rule
I here focus on a subclass of models: static models with a softmax decision rule. In these models, data is supposed to be generated from binary decisions
Where s is the sigmoid function. We assume the posterior is a Gaussian over a single partition of all parameters. The joint distribution over data and parameters is
In our optimization scheme, the posterior covariance matrix at time t is computed from the mode μt of the joint distribution at time t as:
Here, we make the approximation that the mode of the distribution at time t is equal to the mode of the distribution at time t+1, that is μt+1≈μt. Therefore we approximate the posterior covariance at time t+1 by
The hessian of the joint distribution can be expressed as follows
We here make a further approximation and neglect the hessian term.
By doing so, we do not take into account correlations between the parameters. Note that we only do this approximation while optimizing the design. The main inversion after each decision does not do this approximation.
In our approximation, the posterior covariance is independent of the actual generated data
The criteria to optimize is therefore
To quickly compute the posterior covariance matrix for any ut+1 , we compute a closed form approximation of the posterior covariance matrix at time t+1, using the mode of the distribution at trial t.
Il y avait plein de coquilles dans cette page (pb de notations + oublis de certains termes). D'après moi la formule de H était fausse il manquait un facteur s(gu) dans la somme.