Locally Optimal Design for A/B Tests in the Presence of Covariates and Network Dependence
Description
A/B test, a simple type of controlled experiment, refers to the statistical procedure of experimenting to compare two treatments applied to test subjects. For example, many IT companies frequently conduct A/B tests on their users who are connected and form social networks. Often, the users’ responses could be related to the network connection. In this article, we assume that the users, or the test subjects of the experiments, are connected on an undirected network, and the responses of two connected users are correlated. We include the treatment assignment, covariate features, and network connection in a conditional autoregressive model. Based on this model, we propose a design criterion that measures the variance of the estimated treatment effect and allocate the treatment settings to the test subjects by minimizing the criterion. Since the design criterion depends on an unknown network correlation parameter, we adopt the locally optimal design method and develop a hybrid optimization approach to obtain the optimal design. Through synthetic and real social network examples, we demonstrate the value of including network dependence in designing A/B experiments and validate that the proposed locally optimal design is robust to the choices of parameters. Supplementary materials for this article are available online.
Publication Date
1-1-2022
Publisher
figshare Academic Research System
DOI
10.6084/m9.figshare.19230111.v1
Document Type
Data Set
Recommended Citation
Zhang, Qiong; Kang, Lulu (2022), "Locally Optimal Design for A/B Tests in the Presence of Covariates and Network Dependence", figshare Academic Research System, doi: 10.6084/m9.figshare.19230111.v1
https://doi.org/10.6084/m9.figshare.19230111.v1
Identifier
10.6084/m9.figshare.19230111.v1
Embargo Date
1-1-2022