Appendix: Tools to assist developers evaluate computer-based courseware
‘Checklists’ are useful diagnostic or evaluative tools. They can take a variety of forms — lists of criteria to consider, series of questions inviting a ratings response, etc. Other evaluation tools are covered in Section 8.6:Choosing data gathering techniques.
Checklists also differ in their level of focus — from holistic or ‘coarse-grained’ to the particular. We present four such checklists to illustrate these differences.
Multimedia/online unit quality checklist
A series of checklists mostly geared towards multi-media products. In terms of the Integrated Evaluation Framework (see Table 2.1), they relate mostly to focus 2.1 and focus 2.2 — formative evaluation.
User impressions are generally rated on a 1-5 Likert scale for each item on the checklist.
Download the Multimedia/online unit quality checklist [Word].
Note: It would be wise to use other evaluation methods in conjunction with this checklist series when evaluating during the development phase.
‘ Heuristic’ tables — Peter Albion
A more holistic checklist that considers interface and educational design. In the form of three heuristics tables (i.e. guidelines):
- Interface design
- Educational design
- Content
Each heuristic can be judged on a 1-5/Not applicable Likert scale.
Tables and explanations can be found in Albion, P. (1999). ‘Heuristic Evaluation of Educational Multimedia: From Theory to Practice’. Paper presented at ASCILITE Conference, Brisbane 1999. See http://www.ascilite.org.au/conferences/brisbane99/papers/albion.pdf
University of Alberta checklist of 'Principles for Effective Instructional Multimedia'
http://www.atl.ualberta.ca/dleweb/production/evaluating.htm
Evaluation and Redevelopment Framework (ERF)
Three evaluation instruments for ICT-based learning designs are provided. From the homepage to the AUTC funded project, follow the "Activities" link to these instruments.
http://www.learningdesigns.uow.edu.au/Activities/evalinstrument.html