Reinforcement learning has been extensively used to explain how humans solve specific tasks. However, contrary to humans who adapt continuously to known and unknown situations, most reinforcement learning agents struggle when the task changes over time. In this work, we present a new model-based reinforcement learning agent that arbitrates between different models to adapt to changing environments. The agent can detect when the task changes, retroactively update its models depending on when it estimates that the change happened, create new models, re-use past models, merge models if they become similar, and forget unused models. The agent arbitrates between models at the level of local changes. This promotes transfer learning as the agent uses previous local knowledge to adapt faster to new contexts at a small computational and memory cost. We also investigate the role of model reliability in exploration and how co-variation of local changes could permit context detection. We evaluate our computational model's performance in simulation against other theoretical models. Our method provides new insights and predictions about optimal decision-making in changing environments which could be tested by future experiments in humans.
| Type : | : | Poster |
| Thématiques | : | Computational |
| PDF version | : |  PDF version |
