Subjective user evaluation of CF PASCAL programming tools

This study investigated subjective evaluations of two programming environments: 1) SUPPORT, an interactive programming environment with a syntax directed editor on a personal computer and 2) a batch run environment on a large mainframe computer. Participants were students in a 15 week introductory computer science course. In Part 1, one group of 128 first used SUPPORT, while another group of 85 programmed on a mainframe environment. After 6 weeks they were given an evaluative questionnaire and then switched programming environments. In Part 2, 68 used SUPPORT and 60 used the mainframe. At the twelfth week of the course, they were given two questionnaires, one evaluating the environment they had used in the last 6 weeks and one comparing both enviro nments. A measure of programming performance (exam and programming project grades) was also collected. SUPPORT was predicted to reduce the burden of remembering syntactic details resulting in better performance and higher subjective evaluations. Unexpectedly, the SUPPORT users did not earn statistically significantly higher grades. Furthermore, participants expressed a preference for the mainframe over SUPPORT. Specific items on the questionnaires were used to diagnose the strengths and weaknesses of each environment. Designers of syntax directed editors should focus on reducing the syntactic burden not only in programming , but also in the user interface of these tools.