Research
Learning to make decisions under uncertainty.
My research focuses on the theory of reinforcement learning and online learning. I study how algorithms gather information, optimize their decisions and adapt to the structure of a problem. The aim is to understand both what makes learning difficult and how to design algorithms with precise performance guarantees.
Exploration and posterior sampling
How should an algorithm explore?
Learning to make good decisions requires gathering information as well as collecting rewards. I study how randomized algorithms can balance these two objectives, with guarantees that reflect the difficulty of the particular problem they face.
My work connects the analysis of Thompson sampling and non-parametric bandit algorithms to posterior sampling in reinforcement learning. A recent direction separates the decision to explore from the choice of the policy used to gather information, taking a step toward instance-dependent regret optimality in episodic Markov decision processes.
Representative papers
- Towards instance-dependent regret optimality in Episodic MDPs with Posterior SamplingNeurIPS 2026
- A General Recipe for the Analysis of Randomized Multi-Armed Bandit AlgorithmsPreprint
- Sub-sampling for Efficient Non-Parametric Bandit ExplorationNeurIPS 2020 Spotlight
- Optimal Thompson Sampling strategies for support-aware CVaR banditsICML 2021
Policy gradients and stochastic approximation
When do gradient-based methods learn efficiently?
Convergence to a good policy does not by itself describe how efficiently an algorithm learns. I study the behaviour of gradient-based methods through their regret, convergence rates and sensitivity to the choice of learning rate.
In stochastic-gradient bandits, our work identifies learning-rate regimes that separate logarithmic from polynomial regret. I also study stochastic approximation with two coupled time scales, quantifying how nonlinear dynamics and constant step sizes affect error and bias. These analyses provide tools for understanding iterative algorithms used in reinforcement learning.
Representative papers
- Does Stochastic Gradient really succeed for bandits?NeurIPS 2025 Oral
- The Bias of Nonlinear Two-Time-scale Stochastic Approximation under Constant Step-SizesPreprint
Ongoing research project
- Improved regret for Stochastic Gradient Bandits Research note · Version 1 · 16 November 2026
Adaptive and structured online learning
How can learning adapt to its environment?
The standard bandit model is a starting point for problems where environments change, observations have a cost, or decisions must satisfy additional requirements. I study how algorithms can use the structure of these problems while retaining meaningful performance guarantees.
Recent work includes bandit convex optimization in non-stationary environments, unconstrained linear bandits, and contextual bandits with paid observations. Related work studies revenue constraints and online auctions. Across these settings, the goal is to understand which information and assumptions are needed for efficient learning.
Representative papers
Connections to applications
I also collaborate on agricultural applications, including adaptive crop-management trials and analyses of coffee varietal performance across environments. These collaborations connect questions about learning and statistical inference to decisions made under environmental uncertainty.