A scholarly approach to course review

Title

A scholarly approach to course review: developing a teaching cost to revenue metric and quality score

Author(s)

Justin Walls, Faculty of Health
Jo-Anne Kelder, Faculty of Health
Andrea Carr, University College

Subtheme

Advancing the Scholarship of Learning and Teaching

Presentation Type

Showcase Presentation

Room

Social Sciences 209

Time

14.00-14.20

Abstract

The University of Tasmania operates within a national context that has experienced significant change and uncertainty over the last decade. The Faculty of Health is the largest faculty at the University, with approximately 11,000 students, with over 20 bachelor degree courses. The student enrolment associated with these courses contributes substantial revenue to the University through a combination of student fees and government funding.

This presentation details a faculty-level response to the external pressures faced by the University of Tasmania and presents an approach that shows promise for optimising course quality whilst maximising course efficiency. The dimensions of cost, revenue and quality were selected as the three core components that impact efficiency and effectiveness of curriculum and teaching. Efficiency is aligned to cost and revenue while effectiveness is a quality-related construct. An assumption of the research was that these potentially antagonistic elements – cost, revenue and quality – must be kept in balance if courses are to be attractive to students (high quality) and financially viable.

Data collection and analysis spanned micro (unit), meso (course) and macro (Faculty) level. Revenue data consisted of annual teaching student enrolment numbers by unit of study, aggregated to course level. Cost data included academic and administrative staff costs; and student professional experience placement expenditure. A Teaching Revenue to Cost of delivery Ratio (TRCR) metric and course Quality Score metric was developed to enable comparisons of units and course performance.

The TRCR metric can be used to describe overall course profitability and allow comparison of units and courses. In addition, the TRCR can be used to identify course characteristics associated with revenue and cost patterns. The Quality Score is comprised of three sub-measures that are student focused (student progress, student satisfaction, graduate satisfaction). The method for calculating the Quality Score means that different sub-measures can be selected and used to derive a single score that can be plotted against the TRCR metric. Choice with respect to Quality Score sub-measures is important, as literature summarised from the “Student-Evaluation-of-Teaching” cluster (Steinhardt et al., 2017) raised methodological issues with the use of student evaluations of teaching, analysing problems of bias and validity and criticising their use as a “steering instrument” on the grounds of fairness (p. 227). Likewise, understanding the limitations of the sub-measures embedded in the Quality Score, means it is useful to consider trends over time, or that a low score can be used as a trigger for quality improvement.

Plotting the TRCR against a Quality Score allows a combined picture of efficiency and quality to emerge in an Efficiency-Quality Matrix. The longitudinal pattern of how related courses move through the Efficiency-Quality Matrix from year to year promises to give valuable insights into how one course is linked with another.

An outcome of the course review process, including the development of a method for constructing unitised course review reports, supported by explanatory narratives of course and unit coordinators, built the capacity of the Faculty for strategic planning. Further, the review identified issues with units and courses that could be immediately addressed and identified that included course design and course delivery characteristics that optimise both efficiency and effectiveness. The method is useful for evidence-based planning in an uncertain and resource-constrained operating context, thereby minimising risk and maximising opportunities.