Date of Award

5-2024

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Mathematical Sciences

Committee Chair/Advisor

Qiong Zhang

Committee Member

Xiaoqian Sun

Committee Member

Christopher McMahan

Committee Member

Colin Gallagher

Abstract

Online controlled experiments, primarily used on digital platforms like websites or apps, involve varying certain variables while keeping others constant to determine their effects on specific outcomes. This method allows researchers to compare results with a control group, gaining reliable insights. These experiments have grown popular among companies for assessing product impacts and guiding decision-making.

This dissertation presents a group Sequential Probability Ratio Testing (group SPRT) algorithm optimized for online experiments to balance sample number and accuracy by minimizing expected costs. It contrasts group SPRT with traditional SPRT under normal distributions, revealing differences in test power, sample size, and cost efficiency, thus guiding experimental design optimization. In addition, the work delves into the model for network sampling design in A/B testing, which involves selecting a sample from a network to assign users to different versions of a product or feature. The responses of these users are then analyzed to evaluate the impact of variations in the experimental design, building under the conditional auto-regressive model to manage large-scale data effectively. The efficacy of this approach is validated through simulations on both synthetic and real-world networks, testing its robustness in practical applications.

Available for download on Saturday, May 31, 2025

Share

COinS