Ideal Query Expansion using Reinforcement Learning

Abstract

Information retrieval (IR) systems often struggle with short, ambiguous, or underspecified queries, leading to suboptimal document retrieval. Traditional query reformulation methods, such as those based on the Rocchio algorithm, rely on heuristic term selection and relevance feedback but typically apply fixed or manually tuned weights to expanded terms. This limits their adaptability and generalization across diverse query-document contexts. In this thesis, we propose a novel reinforcement learning (RL)-based framework to dynamically optimize term weighting in reformulated queries. We model the problem as a Markov Decision Process (MDP), where each state represents a query as a vector of term weights. An RL agent learns a policy to assign optimal weights to terms by maximizing a reward signal based on retrieval performance—specifically precision-based metrics like Mean Average Precision (MAP). Our method is evaluated on benchmark datasets, where it outperforms traditional static approaches by learning query-specific term weighting strategies that generalize well to unseen queries. The approach draws inspiration from earlier optimization techniques such as Dynamic Feedback Optimization in TREC but differs fundamentally by employing a data-driven learning mechanism rather than rule-based reweighting. The results demonstrate that reinforcement learning offers a principled and flexible solution for effective query reformulation in modern IR systems.