Evaluating the Robustness of Off-Policy EvaluationView Publication
Off-policy Evaluation (OPE), or offline evaluation in general, evaluates the performance of hypothetical policies leveraging only offline log data. It is particularly useful in applications where the online interaction involves high stakes and expensive setting such as precision medicine and recommender systems. Since many OPE estimators have been proposed and some of them have hyperparameters to be tuned, there is an emerging challenge for practitioners to select and tune OPE estimators for their specific application. Unfortunately, identifying a reliable estimator from results reported in research papers is often difficult because the current experimental procedure evaluates and compares the estimators’ performance on a narrow set of hyperparameters and evaluation policies. Therefore, it is difficult to know which estimator is safe and reliable to use. In this work, we develop Interpretable Evaluation for Offline Evaluation (IEOE), an experimental procedure to evaluate OPE estimators’ robustness to changes in hyperparameters and/or evaluation policies in an interpretable manner. Then, using the IEOE procedure,we perform extensive evaluation of a wide variety of existing estimators on Open Bandit Dataset, a large-scale public real-world dataset for OPE. We demonstrate that our procedure can evaluate the estimators’ robustness to the hyperparamter choice, helping us avoid using unsafe estimators. Finally, we apply IEOE to realworld e-commerce platform data and demonstrate how to use our protocol in practice.
Related PublicationsView All
Policy-Adaptive Estimator Selection for Off-Policy Evaluation
Takuma Udagawa, Haruka Kiyohara*, Yusuke Narita*, Yuta Saito*, Kei TatenoOff-policy evaluation (OPE) aims to accurately evaluate the performance of counterfactual policies using only […]
Data-Driven Off-Policy Estimator Selection: An Application in User Marketing on An Online Content Delivery Service
Yuta Saito, Takuma Udagawa, Kei TatenoOff-policy evaluation (OPE) is the method that attempts to estimate the performance of decision making policie […]