Research

Learning to make decisions under uncertainty.

My research focuses on the theory of reinforcement learning and online learning. I study how algorithms gather information, optimize their decisions and adapt to the structure of a problem. The aim is to understand both what makes learning difficult and how to design algorithms with precise performance guarantees.

Exploration and posterior sampling

How should an algorithm explore?

Learning to make good decisions requires gathering information as well as collecting rewards. I study how randomized algorithms can balance these two objectives, with guarantees that reflect the difficulty of the particular problem they face.

My work connects the analysis of Thompson sampling and non-parametric bandit algorithms to posterior sampling in reinforcement learning. A recent direction separates the decision to explore from the choice of the policy used to gather information, taking a step toward instance-dependent regret optimality in episodic Markov decision processes.

Representative papers

Policy gradients and stochastic approximation

When do gradient-based methods learn efficiently?

Convergence to a good policy does not by itself describe how efficiently an algorithm learns. I study the behaviour of gradient-based methods through their regret, convergence rates and sensitivity to the choice of learning rate.

In stochastic-gradient bandits, our work identifies learning-rate regimes that separate logarithmic from polynomial regret. I also study stochastic approximation with two coupled time scales, quantifying how nonlinear dynamics and constant step sizes affect error and bias. These analyses provide tools for understanding iterative algorithms used in reinforcement learning.

Representative papers

Ongoing research project

Adaptive and structured online learning

How can learning adapt to its environment?

The standard bandit model is a starting point for problems where environments change, observations have a cost, or decisions must satisfy additional requirements. I study how algorithms can use the structure of these problems while retaining meaningful performance guarantees.

Recent work includes bandit convex optimization in non-stationary environments, unconstrained linear bandits, and contextual bandits with paid observations. Related work studies revenue constraints and online auctions. Across these settings, the goal is to understand which information and assumptions are needed for efficient learning.

Representative papers

Connections to applications

I also collaborate on agricultural applications, including adaptive crop-management trials and analyses of coffee varietal performance across environments. These collaborations connect questions about learning and statistical inference to decisions made under environmental uncertainty.

Browse all publications.