Recherche par auteur > Fontolan Lorenzo

Animals foraging in unpredictable environments must constantly adjust their learning strategies to navigate uncertainty and maximize reward while managing metabolic costs. It has been proposed that animals employ a combination of model-free (MF) reinforcement learning strategies - where decisions are guided by immediate reward feedback - and model-based (MB) strategies - where an internal model of the environment informs decision-making. Clarifying the interplay between these strategies is crucial for understanding how animals make effective decisions in uncertain environments, which directly impacts their survival. However, disambiguating these strategies remains challenging, as both can result in similar behaviors. We investigate this issue using a simulated two-site foraging paradigm where agents navigate between sites with probabilistic rewards. A key feature of the task is the asymmetry between rewarded and unrewarded attempts: while receiving a reward definitively confirms the correct site, an omitted reward does not necessarily imply the wrong site. Additionally, switching between foraging sites carries a travel cost. To examine foraging strategies, we implemented and compared nine artificial agents: three model-free, three model-based with perfect task knowledge, and three model-based agents that must infer the task's structure from experience. We evaluated agent performance across multiple behavioral metrics - such as accuracy, switch frequency, and failure rates, while systematically varying learning rates, discount factors, travel costs, and exploration rates. Our analysis revealed substantial behavioral overlap between MF and MB strategies across a wide range of parameters, challenging the hypothesis that behavioral data alone can reliably differentiate between these decision-making strategies. This result highlights the need for alternative approaches to infer the underlying cognitive dynamics. Despite the behavioral overlap, certain distinctions emerged between MF and MB agents. Specifically, only MB agents exhibited highly consistent switch patterns with minimal variance in consecutive failure counts following state changes, and MF strategies based on the Marginal Value Theorem uniquely showed a decrease in consecutive failures with increasing accumulated rewards, aligning with theoretical predictions. Surprisingly, MF agents that treated failures as definitive evidence of wrong port choice - ignoring the task's asymmetric structure- outperformed those that did not factor in reward omissions for value updates. These findings represent a significant step toward identifying the behavioral signatures of different learning strategies in dynamic foraging tasks, and highlight the inherent challenges in distinguishing MF and MB strategies based solely on behavioral data.