The aim of this master thesis is to analyze the task characteristics that influence students' performance in solving tasks in the Pišek environment. To this end, we analyzed student performance in the third ACM School Pišek competition using machine learning methods.
In the theoretical part of the master's thesis, we presented the development of students' computational thinking skills in programming. We have described the advantages of programming and listed the computational, psychological and pedagogical skills that students develop while learning programming. We have presented the features of block-based visual programming environments and compared them with those of text-based programming environments. We listed and described programming concepts because knowledge of these concepts is critical to analyzing the data we collected. Programming concepts are basic building blocks that are linked together in various ways to create a program. Examples of programming concepts are: Sequence, Loop, Condition, Variable. We have described the competition tasks using programming concepts based on the expected solutions. For this purpose, we have previously designed a set of concepts suitable for describing the tasks of the third Pišek Experimental Competition, based on the set of programming concepts developed by the Pišek Competition Committee. Below, we looked at programming code at a higher taxonomic level and found basic examples of programming patterns for iterations. Programming patterns are software solutions to recurring algorithmic problems.
To describe the current state of programming education in Slovenia, we examined the inclusion of programming content in Slovenian curricula and in the K-12 framework for computer science education, which forms the basis of most established and accepted computer science curricula. We focused on the importance of competitions and described the programming competitions currently held in Slovenia. We have described in more detail the block-based Pišek Programming Competition, which aims to stimulate students' interest in programming and develop their programming skills, and to close the gap in the number of participants between the Bebras Competition in Computational Thinking and the RTK Competition in Programing. Using specific examples of the competition tasks, we described the basic features of the tasks, namely programming concepts, task representations, complexity of the programming environment, and required approaches to solving programming problems. In the empirical part, we tested the influence of the first three features on students' performance in the competition.
In the empirical part, we evaluated which tasks students solved best and worst by calculating a difficulty index and a discriminability index for each competition group, and which tasks discriminated best between students' performances. We then created machine learning models, Random Forests, for each competition group. In the predictive models, we used student performance as the outcome variable. As independent predictor variables, we used the derived concept variables indicating the presence of the selected concept in the successfully solved tasks, the opsini covariates on the task formats, and the difficulty of the programming environment. The task formats are: Moving a Figure on a Grid, Performing Tasks, Turtle Graphics, and Input-Output Tasks. Factors that increase the difficulty are the limited number of blocks and the use of tests in the solutions. In evaluating the importance of the predictor variables in the predictive models, we found that the task formats and difficulty factors have a similar influence on student performance in the third Pišek competition. In analyzing the influence of mastery of programming concepts, we found that two tasks whose solutions contain identical concepts can be of different difficulty.
In future work, we will investigate the influence of expected programming patterns on predicting student performance in the Pišek competition.
|
