ICER 2024
Mon 12 - Thu 15 August 2024 Melbourne, Victoria, Australia
Thu 15 Aug 2024 13:35 - 13:55 - GenAI and Computing Education (II) Chair(s): Kathryn Cunningham

Novice programmers often struggle through programming problem solving due to a lack of metacognitive awareness and strategies. Previous research has shown that novices can encounter multiple metacognitive difficulties while programming, such as forming incorrect conceptual models of the problem and having a false sense of progress after arriving at and testing a solution. Novices are typically unaware of how these difficulties are hindering their progress toward a solution. Meanwhile, many novices are now programming with generative AI (GenAI), which can provide complete solutions to most introductory programming problems, code suggestions, hints for next steps when stuck, and explain cryptic error messages. Its impact on novice metacognition has only started to be explored. In this study, we replicate a previous study that examined novice programming problem solving behavior and extend it by incorporating GenAI tools. Through 21 lab sessions consisting of participant observation, interview, and eye tracking, we explore how novices are coding with GenAI tools. Although 20 of 21 students completed the assigned programming problem, our findings show a clear divide in the use of GenAI tools between students who accelerated and students who struggled. For students in the latter group, our findings indicate that previously known metacognitive difficulties persist, and that GenAI unfortunately can compound them and even introduce new metacognitive difficulties. Our findings show there is a clear divide between students using GenAI to accelerate and students hindered by these tools. We argue that these findings indicate a need for GenAI scaffolding through new tools and pedagogies to combat this concerning problem.

Thu 15 Aug

Displayed time zone: Brisbane change

13:15 - 14:15
GenAI and Computing Education (II)Research Papers
Chair(s): Kathryn Cunningham University of Illinois Urbana-Champaign
13:15
20m
Talk
Using Benchmarking Infrastructure to Evaluate LLM Performance on CS Concept Inventories: Challenges, Opportunities, and Critiques
Research Papers
Murtaza Ali University of Washington, Prerna Rao University of Washington, Yifan Mai Stanford University, Benjamin Xie Stanford University
DOI Pre-print
13:35
20m
Talk
The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers
Research Papers
James Prather Abilene Christian University, Brent Reeves Abilene Christian University, Juho Leinonen Aalto University, Stephen MacNeil Temple University, Arisoa Randrianasolo Abilene Christian University, Brett Becker University College Dublin, Bailey Kimmel Abilene Christian University, Jared Wright Abilene Christian University, Ben Briggs Abilene Christian University
Link to publication DOI Pre-print
13:55
20m
Talk
An Investigation of the Drivers of Novice Programmers’ Intentions to Use Web Search and GenAI
Research Papers
James Skripchuk North Carolina State University, John Bacher North Carolina State University, Thomas Price North Carolina State University