统计与数据科学系系列学术报告之四百零二期

时间：2023年7月18日（周二）15:00-16:00

主持人：复旦大学管理学院统计与数据科学系郁文教授

地点：史带楼604室

报告人：Zhimei Ren Assistant Professor

Department of Statistics and Data Science at the Wharton School

University of Pennsylvania

题目：Policy learning “without” overlap: Pessimism and generalized empirical Bernstein’s inequality

摘要：Offline policy learning aims at utilizing observations collected a priori (from either fixed or adaptively evolving behavior policies) to learn the optimal individualized decision rule in a given class. Existing policy learning methods rely on a uniform overlap assumption, i.e., the propensities of exploring all actions for all individual characteristics are lower bounded in the offline dataset. In other words, the performance of these methods depends on the worst-case propensity in the offline dataset. As one has no control over the data collection process, this assumption can be unrealistic in many situations, especially when the behavior policies are allowed to evolve over time with diminishing propensities.

In this talk, I will introduce a new algorithm that optimizes lower confidence bounds (LCBs) — instead of point estimates — of the policy values. The LCBs are constructed by quantifying the estimation uncertainty of the augmented-inverse-propensity-weighted (AIPW)-type estimators using knowledge of the behavior policies for collecting the offline data. Without assuming any uniform overlap condition, we establish a data-dependent upper bound for the suboptimality of our algorithm, which depends only on (i) the overlap for the optimal policy, and (ii) the complexity of the policy class. As an implication, for adaptively collected data, we ensure efficient policy learning as long as the propensities for optimal actions are lower bounded over time, while those for suboptimal ones are allowed to diminish arbitrarily fast. In our theoretical analysis, we develop a new self-normalized concentration inequality for IPW estimators, generalizing the well-known empirical Bernstein’s inequality to unbounded and non-i.i.d. data.

个人简介：Zhimei Ren is an incoming assistant professor in the Department of Statistics and Data Science at the Wharton School, University of Pennsylvania. She was a postdoctoral researcher in the Statistics Department at the University of Chicago, advised by Professor Rina Foygel Barber. Before joining the University of Chicago, she obtained her Ph.D. in Statistics from Stanford University, advised by Professor Emmanuel Candès. Prior to this, she received her Bachelor’s degree in Statistics from Peking University.