Debugging with an AI Tutor: Investigating Novice Help-seeking Behaviors and Perceived Learning
Debugging is a crucial skill for programmers, yet it poses significant challenges for novice learners. The introduction of large language models (LLMs) has opened up new possibilities for providing personalized debugging support to students. However, concerns have been raised about potential student over-reliance on LLM-based tools. This mixed-methods study investigates how a pedagogically-designed LLM-based chatbot supports students’ debugging efforts in an introductory programming course. We conducted interviews and debugging think-aloud tasks with 20 students at three points throughout the semester. We specifically focused on characterizing when students initiate help from the chatbot during debugging, how they engage with the chatbot’s responses, and how they describe their learning experiences with the chatbot. By analyzing data from the debugging tasks, we identified varying help-seeking behaviors and levels of engagement with the chatbot’s responses, depending on students’ familiarity with the suggested strategies. Interviews revealed that students appreciated the content and experiential knowledge provided by the chatbot, but did not view it as a primary source for learning debugging strategies. Additionally, students self-identified certain chatbot usage behaviors as negative, non-ideal'' engagement and others as positive,
learning-oriented'' usage. Based on our findings, we discuss pedagogical implications and future directions for designing pedagogical chatbots to support debugging.
Tue 13 AugDisplayed time zone: Brisbane change
13:15 - 14:15 | |||
13:15 20mTalk | Debugging with an AI Tutor: Investigating Novice Help-seeking Behaviors and Perceived Learning Research Papers Stephanie Yang Harvard University, Hanzhang Zhao Harvard Graduate School of Education, Yudian Xu Harvard Graduate School of Education, Karen Brennan Harvard Graduate School of Education, Bertrand Schneider Harvard Graduate School of Education | ||
13:35 20mTalk | Evaluating Contextually Personalized Programming Exercises Created with Generative AI Research Papers Evanfiya Logacheva Aalto University, Arto Hellas Aalto University, James Prather Abilene Christian University, Sami Sarsa University of Jyväskylä, Juho Leinonen Aalto University Link to publication DOI Pre-print | ||
13:55 20mTalk | Insights from Social Shaping Theory: The Appropriation of Large Language Models in an Undergraduate Programming Course Research Papers Aadarsh Padiyath University of Michigan, Xinying Hou University of Michigan, Amy Pang University of Michigan, Diego Viramontes Vargas University of Michigan, Xingjian Gu University of Michigan, Tamara Nelson-Fromm University of Michigan, Zihan Wu University of Michigan, Mark Guzdial University of Michigan, Barbara Ericson University of Michigan Pre-print |