Date of Award
12-2022
Document Type
Dissertation
Degree Name
Doctor of Philosophy (PhD)
Department
Mathematical Sciences
Committee Chair/Advisor
Yuyuan Ouyang
Committee Member
William Bridges Jr
Committee Member
Kai Liu
Committee Member
Boshi Yang
Abstract
In this dissertation, our main focus is to design and analyze first-order methods for computing approximate solutions to convex, smooth optimization problems over certain feasible sets. Specifically, our goal in this dissertation is to explore some variants of sliding and Frank-Wolfe (FW) type algorithms, analyze their convergence complexity, and examine their performance in numerical experiments. We achieve three accomplishments in our research results throughout this dissertation. First, we incorporate a linesearch technique to a well-known projection-free sliding algorithm, namely the conditional gradient sliding (CGS) method. Our proposed algorithm, called the conditional gradient sliding with linesearch (CGSls), does not require the knowledge of Lipschitz constant of the gradient of objective function, which is critical in the numerical implementation of the CGS method. Second, we explore the possibility of designing a bundle level type version of the CGS method, which to the best of our knowledge has not yet appeared in the literature. Our proposed sliding APL (SAPL) method achieves the same complexity to the CGS method. Third, we study numerical algorithms for solving the image co-localization problem. For this problem, we propose new variants of the Frank-Wolfe (FW) method and compare their empirical performance with other existing methods. The dissertation is organized as follows. In the first chapter, we review some projection-based and projection-free algorithms, their variants, and their respective advantages and disadvantages. Several useful definitions, theorems, and lemmas are also introduced in this chapter that will be utilized throughout the dissertation. For completeness, we prove most of the known results listed in this chapter (proof deferred to the appendix). In the second chapter, we incorporate a linesearch technique to the well-known CGS method and propose the CGSls method. We show that the proposed CGSls method converges with similar complexity to the CGS method. We also examine the performance of the proposed algorithm by comparing it to the CGS method and other projection-free algorithms. In the third chapter, we explore the possibility of designing a bundle level type variant of the CGS method. The proposed SAPL method is inspired by previous literature on bundle level type method. Such bundle level type method has not yet appeared in any literature on sliding algorithms. We show that the proposed SAPL method converge with the same order of complexity as the CGS and CGSls methods. In the fourth chapter, we apply the algorithms studied in previous chapters to the well-known video co-localization problem. We also propose new variants of the FW method and compare their empirical performance with other numerical methods.
Recommended Citation
Nazari, Seyed Hamid, "On Variants of Sliding and Frank-Wolfe Type Methods and Their Applications in Video Co-localization" (2022). All Dissertations. 3192.
https://open.clemson.edu/all_dissertations/3192