Task-oriented assessment for authentic learning in an introductory programming unit

Title

Task-oriented assessment for authentic learning in an introductory programming unit

Author(s)

James Montgomery, School of Engineering & ICT

Subtheme

Making a Difference for Students

Presentation Type

Showcase Presentation

Room

Social Sciences 209

Time

13.15-13.35

Abstract

This paper describes the implementation of a competency- and portfolio-based introductory programming unit KIT101 Programming Fundamentals at the University of Tasmania. Introductory programming units worldwide, including KIT101, are notorious for having relatively high failure rates, because programming requires students to apply new skills in an unfamiliar domain (Teague, 2015). The unit’s redesign was driven by making a difference for students through the delivery of frequent formative assessment through portfolios, authentic learning experiences, and the opportunity for students to demonstrate pass-level competency or to extend themselves to attain a higher grade. The changes were modelled on recent work at Swinburne University of Technology, who devised an approach to portfolio-based assessment based on constructivist principles (Biggs, 1999) that could scale to large class sizes (Cain & Barbar, 2016; Cain, Grundy & Woodward, 2017; Renzella et al., 2016).

Prior to 2017 KIT101 employed a ‘traditional’ teaching pattern of formative computer lab tasks, two major assignments and a 60% final written exam. This approach was not effective for many students; assignments were daunting (despite provided guidance) and a written exam is an inauthentic way of measuring student programming ability. The new competency model presents students with a collection of smaller portfolio tasks which students complete in an authentic environment, with access to code editing tools and documentation, and then submit via MyLO. Tutors provide feedback on submitted work which students may revise until it is of an acceptable standard. Each task contributes to a particular grade, and students qualify for a grade by completing all tasks for that grade and those below. Pass-level tasks, which comprise two-thirds of all tasks, are highly scaffolded, while Credit and Distinction tasks hand greater responsibility to students, and the two HD tasks have students completing work of their own devising, with teaching staff guidance. By stratifying portfolio tasks struggling students can take more time to master fundamental skills without falling further behind. At the same time, the competency model gives confidence in the abilities of passing students.

The set of portfolio tasks was developed using Swinburne’s unit as a guide and incorporating other recent changes to KIT101. Skills at the CR and DN levels were selected based on observations of student performance in different areas of the 2016 exam. Students submit code, screenshots and other documents to MyLO, where tutors assign a status indicating progress toward completion: Redo/Resubmit if further work is required, Discuss, or Completed when it has met the required standard. A suite of supporting documents and MyLO customisations were developed, including MyLO grading schemes that incorporate a glyph (text-based icon), status text and colour. Thus students focus on the work rather than the mark.

Repeating and new students appreciated the new design’s ongoing feedback, opportunity to correct their mistakes, clear progression of tasks and accompanying course materials. Tutors with experience of the unit’s earlier design observed improved student progress. Within the core BICT cohort, the 63% pass rate was the second highest in the last eleven Semester 1 deliveries (median 55%), while the 21% failure (NN) rate was the third lowest (median 26%); and the 14% absentee rate was equivalent to the long-term median. This first delivery also identified targets for improvement, including additional mechanisms to keep students on track (enabling more students to pass), more efficient feedback from tutors, and an improved ePortfolio system that composes work submitted during semester into a single document.

While well-suited to teaching introductory programming, this competency-based approach would also benefit other skills-based, first-year and foundation units. The lessons learned in task selection and pacing, and in adapting MyLO to support this delivery, will help others wanting to implement a similar scheme in their unit.

References

Biggs, J. (1999). What the student does: Teaching for enhanced learning. Higher Education Research & Development, 18(1), pp.57-75. DOI: 10.1080/0729436990180105

Cain, A., & Babar, M.A. (2016). Reflections on applying constructive alignment with formative feedback for teaching introductory programming and software architecture. Proceedings of the 38th International Conference on Software Engineering Companion, pp.336-345.

Cain, A., Grundy, J., & Woodward, C.J. (2017) Focusing on learning through constructive alignment with task-oriented portfolio assessment. European Journal of Engineering Education, pp.1-16. DOI: 10.1080/03043797.2017.1299693

Renzella, J., Cummaudo, A., Cain, A., & Wilson, R. (2016). Doubtfire: A modern, lightweight learning management system. Online resource: https://github.com/doubtfire-lms

Teague, D. (2015). Neo-Piagetian Theory and the Novice Programmer. PhD by Publication, Queensland University of Technology.