Understanding Policy Transfer Under Decomposition and Coordination: Sequential Reinforcement Learning With Relaxation-Based Feasibility Control

Author

Aannand Lal, Clemson UniversityFollow

Date of Award

12-2025

Document Type

Thesis

Degree Name

Master of Science (MS)

Department

Mechanical Engineering

Committee Chair/Advisor

Dr. Cameron Turner

Committee Member

Dr. John Wagner

Committee Member

Dr. Laura Redmond

Abstract

This thesis investigates how decomposition and coordination (D&C) choices influence exploration, learning, and transfer in reinforcement learning (RL) based design frameworks. A unified methodology is developed to evaluate generality under decomposition and to demonstrate sequential coordination on a representative physical task. The generality study holds the underlying physics constant while varying problem presentation - specifically scalarization weights, objective orientations, and feasibility handling. A train-evaluate matrix is constructed in which each trained policy is assessed across all alternative decompositions. Set-based diagnostics quantify generality through exact-match identity, Jaccard similarity of distinct feasible states, total feasible coverage, and Pareto-front quality. Results show that different decompositions can yield comparable hypervolume yet diverge in identity, revealing that presentation alters exploration paths. Adjusting or reorienting select objectives improves alignment in some cases, indicating that presentation grammar and objective pairing are critical coordination and design choices. Building on this foundation, a sequential demonstration applies the framework to Mars terrain navigation using MOLA data. A relaxation variable ρ is introduced as an explicit coordination channel controlling permissible deviations from slope constraints. Deterministic evaluations across increasing ρmax values show a transition from stalled trajectories at ρmax = 0 to efficient motion with higher reach and modest hazard at moderate ρmax. The behavior exhibits consistent temporal patterns of relaxation and recovery, indicating structured coordination rather than memorization. The research contributes: (1) a reproducible protocol for assessing presentation sensitivity via a train- evaluate matrix and complementary set metrics; (2) a constructive mapping from D&C constructs to RL and constrained-MDP elements that clarifies how presentation induces distributional shifts; and (3) a sequential relaxation mechanism that operationalizes coordination within RL while yielding planner- usable option sets. These results advance understanding of how problem formulation governs learning behavior and establish a basis for coordination-aware, presentation-robust decision frameworks.

Recommended Citation

Lal, Aannand, "Understanding Policy Transfer Under Decomposition and Coordination: Sequential Reinforcement Learning With Relaxation-Based Feasibility Control" (2025). All Theses. 4675.
https://open.clemson.edu/all_theses/4675