We propose and evaluate a lightweight strategy for tracing code that can be efficiently taught to novice programmers, building off of recent findings on “sketching” when tracing. This strategy helps novices apply the syntactic and semantic knowledge they are learning by encouraging line-by-line tracing and providing an external representation of memory for them to update. To evaluate the effect of teaching this strategy, we conducted a block-randomized experiment with 24 novices enrolled in a university-level CS1 course. We spent only 5-10 minutes introducing the strategy to the experimental condition. We then asked both conditions to think-aloud as they predicted the output of short programs. Students using this strategy scored on average 15% higher than students in the control group for the tracing problems used the study (p<0.05). Qualitative analysis of think-aloud and interview data showed that tracing systematically (line-by-line and “sketching” intermediate values) led to better performance and that the strategy scaffolded and encouraged systematic tracing. Students who learned the strategy also scored on average 7% higher on the course midterm. These findings suggest that in <1 hour and without computer-based tools, we can improve CS1 students’ tracing abilities by explicitly teaching a strategy.
Integrated with Code.org CS Principles (2019): In collaboration with GT Wrobel at Code.org, we integrated strategies and ideas from this paper into Code.org’s Computer Scince Principles 2020-‘21 curriculum to help improve the experiences of the 50,000 students a year who use it to learn computing!
When I was teaching CS A, I always tried to get students to trace their code on scrap paper (don’t try to keep it all in your head!). This research actually taught a specific tracing strategy, even better!
Pamela Fox, CS instructor and Khan Academy curriculum creator