PS9 R2b Interpreting eVALUate results: Seeing the big picture

Title

Interpreting eVALUate results: Seeing the big picture

Author(s)

• Sonya Stanford, School of Social Sciences, CALE*
• Vaughan Higgins, School of Social Sciences, CALE*
• Jo-Ann Kelder, Tasmanian Institute Learning and Teaching, Academic Division

Subtheme

Connectedness

Presentation Type

Lightning Presentation

Room

Room 2

Time

11.40-12.05

Abstract

Student evaluations of their learning experiences provide powerful insights into the effectiveness of teaching strategies and practices (Mart, 2017). Yet it is questionable whether teaching effectiveness correlates with how much or how well students learn (Hornstein, 2017; Fan et al., 2019). Interpreting the utility of student evaluations of teaching (SET) and determining how to respond to these, can be challenging; especially when judgements about personal characteristics are evident.

To date, the University of Tasmania has not provided explicit guidelines for interpreting SET – ‘eVALUate’ – results. This is a problem as reports are sent to individuals who may not have an analytic frame to understand their meaning. Emotional reactions to results (disappointment, humiliation, shame) may prevent staff seeking support from colleagues, which risks them becoming isolated and disenfranchised.

We present a framework, that was developed by Social Sciences and TILT, to analyse SET results and situate the data in their relevant contexts. Using a reflective approach, two main questions are asked:

1. Does the numerical eVALUate data provide enough information to understand what could be improved or does this need to be investigated further through another forum or strategy?
2. Do your reflections provide clues about what kind of changes could be implemented and who needs to ‘own’ these?

These questions help staff situate ratings and comments in the broader context in which the whole program of learning is provided. It offers new ways of thinking about learning improvement strategies that can be achieved in a collegial environment.

References

Fan, Y., Shepherd, E., Slavich, D., Waters, M., Stone, R., Abel, E., & Johnston, L. (2019). Gender and cultural bias in student evaluations: Why representation matters. PLOS ONE, 14(2), e0209749. https://dx.doi.org/10.1371/journal.pone.0209749

Hornstein, H. A. (2017). Student evaluations of teaching are an inadequate assessment tool for evaluating faculty performance. Cogent Education, 4(1), 1304016. https://dx.doi.org/10.1080/2331186X.2017.1304016

Mart, C. T. (2017). Student evaluations of teaching effectiveness in Higher Education. International Journal of Academic Research in Business and Social Sciences, 7(10), 57-61. http://dx.doi.org/10.1080/2331186X.2017.1304016