Testing Scratch Programs Automatically

Abstract

Block-based programming environments like Scratch foster engagement with computer programming and are used by millions of young learners. Scratch allows learners to quickly create entertaining programs and games, while eliminating syntactical program errors that could interfere with progress. However, functional programming errors may still lead to incorrect programs, and learners and their teachers need to identify and understand these errors. This is currently an entirely manual process. We introduce a formal testing framework that describes the problem of Scratch testing in detail, and instantiate this formal framework with the Whisker tool, which provides automated and property-based testing functionality for Scratch programs. Empirical evaluation on real student and teacher programs demonstrates that Whisker can successfully test Scratch programs, and automatically achieves an average of 95.25 % code coverage. This opens up new possibilities to support learners of programming in their struggles. This summary refers to the article \"Testing Scratch Programs Automatically\" published at the 27th ACM SIGSOFT International Symposium on Foundations of Software Engineering 2019.