ICER 2024
Mon 12 - Thu 15 August 2024 Melbourne, Victoria, Australia
Thu 15 Aug 2024 13:55 - 14:15 - GenAI and Computing Education (II) Chair(s): Kathryn Cunningham

External help resources are frequently used by novice programmers solving classwork in undergraduate computing courses. Traditionally, these tools consisted of web resources such as tutorial websites and Q&A forums. With the rise of Generative AI (GenAI), there has been increasing concern and research about how external resources should be used in the classroom. However, little work has directly contrasted student beliefs and perceptions of web resources with GenAI, has grounded these beliefs in prior psychological theory, and has investigated how demographic factors and student backgrounds influence these beliefs and intentions. We administered a vignette-style survey across two courses required for a CS major at an R1 University, a freshman (n=152) and senior capstone course (n=44). Students responded to likert questions aiming to measure behavioral factors related to these tools, such as intention to use, perceived attitudes, peer perceptions, and their own perceived tool competency. We primarily investigate the results of an introductory course, finding that novices have a wide range of opinions on both resources, but overall find them slightly useful and have a tendency to prefer web-search. We compare this with seniors, who have more positive perceptions of these tools, and discuss possible reasons and implications for this difference. We constructed two path models to investigate which factors strongly influence novices’ intention to use resources and find the primary factor to be their general attitudes in how these tools will result in a positive or negative outcome (e.g. perceived benefits, justifiability). We also measure the effects of student background on intention to use these resources. Finally, we discuss implications and suggestions on how instructors can use this information to approach, address, and influence resource usage in their classrooms.

Thu 15 Aug

Displayed time zone: Brisbane change

13:15 - 14:15
GenAI and Computing Education (II)Research Papers
Chair(s): Kathryn Cunningham University of Illinois Urbana-Champaign
13:15
20m
Talk
Using Benchmarking Infrastructure to Evaluate LLM Performance on CS Concept Inventories: Challenges, Opportunities, and Critiques
Research Papers
Murtaza Ali University of Washington, Prerna Rao University of Washington, Yifan Mai Stanford University, Benjamin Xie Stanford University
DOI Pre-print
13:35
20m
Talk
The Widening Gap: The Benefits and Harms of Generative AI for Novice Programmers
Research Papers
James Prather Abilene Christian University, Brent Reeves Abilene Christian University, Juho Leinonen Aalto University, Stephen MacNeil Temple University, Arisoa Randrianasolo Abilene Christian University, Brett Becker University College Dublin, Bailey Kimmel Abilene Christian University, Jared Wright Abilene Christian University, Ben Briggs Abilene Christian University
Link to publication DOI Pre-print
13:55
20m
Talk
An Investigation of the Drivers of Novice Programmers’ Intentions to Use Web Search and GenAI
Research Papers
James Skripchuk North Carolina State University, John Bacher North Carolina State University, Thomas Price North Carolina State University