Technological and Classical Pedagogical Agents in Action: How Design Influences Learning in Kenyan Higher Education

Abstract

This study explores whether classical (classic) or technological pedagogical agents perform better in promoting student interaction and learning outcomes in the Kenyan context.  The study used a mixture of both qualitative and quantitative methods. A total of 200 university students from varying socio-economic backgrounds across Kisumu, Kisii and Homabay counties in Kenya were targeted as the population. A multi-stage stratified random sampling approach was used in order to obtain cases from a range of socio-economic statuses, geographical locations and school types. In addition to pre- and post-test data to measure learning outcomes, focus group data was collected to gain further qualitative insights into student preferences and experiences with the agents. Cognitive Load Theory (CLT) and Social Presence Theory provided the theoretical framework for the study, with an emphasis on emotional involvement and the social aspects of learning. Didactic data from the pre- and post-tests were analyzed through paired-samples t-tests to compare the learning outcomes of the experimental and control groups, while measurement of engagement amount was analyzed with an independent t-test to determine the difference between the engagement of those students who interacted with more classical Agents against those who interacted with Technological (Abstract) Agents. Means and standard deviations were calculated using descriptive statistics. Focus group qualitative data were transcribed, coded, and analyzed using thematic analysis to extract common themes and insights regarding student preferences and agent efficacy. The findings demonstrate that students engaging with classical agents showed dramatically improved engagement levels (Mean: 4.3, SD = 0.5) and learning outcomes (25% improvement on post-test scores) compared with students using Technological (Abstract) Agents (Mean: 3.5, SD = 0.6; 15% improvement on post-test scores). Similarly, results from inferential statistics bolster these conclusions with a t-test identifying a significant difference in engagement scores (t(198) = 4.82, p < 0.05) and a paired-samples t-test indicating significant gains in learning outcomes associated with classical agents (t(99) = 8.75, p < 0.01). Focus group quantitative data indicated a strong overall preference for classical agents, with emotionally relatable feedback prevalent from qualitative analyses. 70% of urban students preferred classical agents, but rural students preferred both types of agents equally. The results of the study, along with their implications for learning environments in Kenya, suggest there is some promise of classical pedagogical agents enhancing engagement, and hence learning. Further studies are needed to fully understand their long-lasting effects and to better fit them to diverse educational settings to optimize their impact.