Date of Award

12-2023

Document Type

Dissertation

Degree Name

Doctor of Philosophy (PhD)

Department

Psychology

Committee Chair/Advisor

Dr. Marissa Shuffler

Committee Member

Dr. Jennifer Ogle

Committee Member

Dr. Allison Traylor

Committee Member

Dr. Patrick Rosopa

Abstract

Peer Evaluation Systems (PESs) allow members of student teams to provide one another with computer-mediated feedback in the form of qualitative, open-ended comments. The current research leverages unsupervised Natural Language Processing (NLP), namely Biterm Topic Modeling (BTM) and sentiment analysis, to uncover latent topics and degree of positivity and negativity expressed in peer feedback, respectively. BTM results revealed a 6-topic model that was reliably replicated over 10 Gibbs initializations 80% of the time. Topics were labeled Timely Communication, Idea Generation, Coordination & Adaptation, Work Quality, Team Support & Focusing, and Work Accountability. Qualitative comparison suggests that these topics demonstrate significant overlap with concepts detailed within existing teamwork and feedback frameworks. Sentiment analysis indicated that peer feedback had a predominantly neutral to positive valence orientation, but that the analysis had limited accuracy. Significance testing evaluating the impact of the topic of feedback, feedback valence, and feedback length on outcome measures of students’ learning of teamwork skills were entirely non-significant. These results are discussed, along with discussions of NLP’s potential to expand existing theories and frameworks with data-driven techniques, and to provide educators with rapid, high-level insight from PESs that support student learning outcomes.

Author ORCID Identifier

C16140353

Share

COinS
 
 

To view the content in your browser, please download Adobe Reader or, alternately,
you may Download the file to your hard drive.

NOTE: The latest versions of Adobe Reader do not support viewing PDF files within Firefox on Mac OS and if you are using a modern (Intel) Mac, there is no official plugin for viewing PDF files within the browser window.