Data-Driven Off-Policy Estimator Selection: An Application in User Marketing on An Online Content Delivery ServiceView Publication
Off-policy evaluation (OPE) is the method that attempts to estimate the performance of decision making policies using historical data generated by different policies without conducting costly online A/B tests. Accurate OPE is essential in domains such as healthcare, marketing or recommender systems to avoid deploying poor performing policies, as such policies may hart human lives or destroy the user experience. Thus, many OPE methods with theoretical backgrounds have been proposed. One emerging challenge with this trend is that a suitable estimator can be different for each application setting. It is often unknown for practitioners which estimator to use for their specific applications and purposes. To find out a suitable estimator among many candidates, we use a data-driven estimator selection procedure for off-policy policy performance estimators as a practical solution. As proof of concept, we use our procedure to select the best estimator to evaluate coupon treatment policies on a real-world online content delivery service. In the experiment, we first observe that a suitable estimator might change with different definitions of the outcome variable, and thus the accurate estimator selection is critical in real-world applications of OPE. Then, we demonstrate that, by utilizing the estimator selection procedure, we can easily find out suitable estimators for each purpose.
RecSys Workshop REVEAL 2020: Bandit and Reinforcement Learning from User Interactions
Related PublicationsView All
Policy-Adaptive Estimator Selection for Off-Policy Evaluation
Takuma Udagawa, Haruka Kiyohara*, Yusuke Narita*, Yuta Saito*, Kei TatenoOff-policy evaluation (OPE) aims to accurately evaluate the performance of counterfactual policies using only […]
Evaluating the Robustness of Off-Policy Evaluation
Yuta Saito*, Takuma Udagawa, Haruka Kiyohara*, Kazuki Mogi*, Yusuke Narita*, Kei TatenoOff-policy Evaluation (OPE), or offline evaluation in general, evaluates the performance of hypothetical polic […]