Speaker: Christian Johannes Tomaschitz
Abstract
This master thesis presents a comprehensive experimental analysis aimed at assessing the effectiveness of ChatGPT compared to traditional teaching methods using a textbook. The study was conducted in the context of computer science classes at a secondary school in Austria, where the programming language Python was taught to students in the 10th and 11th grades, aged 16 to 18 years. Over a period of eight weeks, 35 learners who had no prior knowledge of programming were included in the experiment and assigned the task of enhancing their programming skills using Python and TigerJython as the development platform. Two groups were formed, and the allocation of students to the two groups was carried out randomly. Group A primarily used the textbook "Programming in Python by Tobias Kohn, while Group B worked with ChatGPT. To assess the participants baseline level, an entrance test was conducted at the beginning of the study to evaluate their prior knowledge.
The investigation ultimately included two tests and a final project to assess the level of knowledge and allow for an immediate comparison of learning outcomes between the groups. A special focus was placed on acquiring programming skills, whether the use of ChatGPT contributed to a deeper understanding of programming concepts compared to traditional E-Books. Furthermore, the study analyzed the development and application of self-learning skills among the students within their respective groups, which is essential for autonomously tackling new challenges and expanding their knowledge. The traditional data analysis was expanded through analysis of the learner’s diary of the final project after the investigation to gain insights into their individual learning experiences, preferences, and perceived challenges.
The findings of this study provide insights into the effectiveness of AI-supported learning compared to traditional methods and emphasize the need for a differentiated application of various teaching methods in the field of computer science.
Both learning methods had their own advantages:
- Students who used the textbook gained a deeper understanding of programming principles,
- while those who worked with ChatGPT demonstrated improved and accelerated problem-solving skills, strategies, and a broader overview of the tasks at hand.
These results emphasize the importance of merging traditional and contemporary teaching methods to meet the diverse learning needs of students and to optimally prepare them for future challenges in an increasingly digitized world.
