Control, Decision-Making, and Learning Approaches for Connected and Autonomous Driving Systems with Humans-in-the-loop

Author

Fangjian LiFollow

Date of Award

5-2022

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Mechanical Engineering

Committee Chair/Advisor

Dr. Yue Wang

Committee Member

Dr. John R. Wagner

Committee Member

Dr. Mashrur Chowdhury

Committee Member

Dr. Ardalan Vahidi

Abstract

By virtue of vehicular connectivity and automation, the vehicle becomes increasingly intelligent and self-driving capable. However, no matter what automation level the vehicle can achieve, humans will still be in the loop despite their roles. First, considering the manual driving car as a disturbance to the connected and autonomous vehicles (CAVs), a novel string stability is proposed for mixed traffic platoons consisting of both autonomous and manual driving cars to guarantee acceptable motion fluctuation and platoon safety. Furthermore, humans are naturally considered as the rider in the passenger vehicle. A human-centered cooperative adaptive cruise control (CACC) is designed to improve physical and psychological comfort with the guarantee of string stability. Compared with the benchmark CACC, the human-centered CACC essentially enhances driving comfort.

In emergencies and adversarial scenarios, the human operator can act as a supervisor of the autonomous driving systems. Correspondingly, a human-robot interaction framework to reduce the vulnerability of a CAV platoon under cyber attacks is proposed. To mitigate the effects of cyber-attacks, an observer-based resilient controller is first designed. The corresponding platoon safety conditions are also derived. Next, to facilitate human supervision, a decision-making aid system is developed, which consists of an anomaly reporting system (ARS), a trust-based information management system (TIMS), and a graphical user interface (GUI). Representative humans-in-the-loop experiment demonstrates that the proposed framework can effectively guide human operators when working with CAV platoons under cyber attacks.

Moreover, as an expert, an experienced driver can even teach autonomous systems to drive safely. To fulfill this imitation learning task, safety awareness is injected into the adversarial inverse reinforcement learning (AIRL) algorithm, forming a novel safe inverse reinforcement learning (SAIRL) algorithm. First, the control barrier function (CBF) analysis is used to guide the training process of a safety critic. Second, the safety critic is integrated with the AIRL discriminator to impose the safety consideration. To further enforce the importance of safety, a safety regulator is also introduced to discourage the recovered reward function from assigning high rewards to the risky state-action pairs. In the simulated highway-driving scenario, the proposed S-AIRL outperformed the original AIRL algorithm with a much lower collision rate.

Finally, to accommodate the maximum adversaries of the neighbor manual driving car, its interaction with the subject autonomous vehicle is modeled as a two-player zero-sum Markov game (TZMG). The TZMG Q learning algorithm is then adopted to train a safe policy corresponding to the Nash Equilibrium. In the human subject test, compared to the conventional policy learning algorithm, the TZMG-based algorithm can produce a much safer driving policy with a lower collision rate to the adversaries of real manual driving cars.

Recommended Citation

Li, Fangjian, "Control, Decision-Making, and Learning Approaches for Connected and Autonomous Driving Systems with Humans-in-the-loop" (2022). All Dissertations. 3028.
https://open.clemson.edu/all_dissertations/3028

Author ORCID Identifier

0000-0002-0099-5728