Module 10: Evaluation
Explain with what evaluation criteria and measurement instruments usability testing can be carried out.
"Usability Testing" is an empirical method for evaluating the degree of usefulness, usability and user friendliness of learning. Evaluation criteria: error rate, kind of error, speed of response or way of operations Measuring Instruments: observation, loud thinking, logging software, video surveillance, time measurement, eye movement registration, pupillometry, physiological measurements
Describe what the terms "usability" and "usability testing" mean.
"Usability" of a product is the extent by which a defined product can be used by a defined user for a defined objective in the most effective way (e.g. the product supports users to achieve objectives), efficient (e.g. measure supports the users to achieve the objectives with minimal effort) and satisfactory (e.g. perceived pleasant use). https://www.youtube.com/watch?v=BrVnBdW6_rE
Define "Evaluation" and describe its purpose.
EVALUATION is the assessment of products or processes. For the social sciences, evaluation means the review of the value and effectiveness of an intervention (such as therapy measure, awareness campaign) with the means of the empirical research. The evaluation is focused/goal-oriented and follows a systematic, objective approach (i.e. the evaluation has to be understandable for others and verifiable).
Explain the difference between formative and summative evaluation. When is either form useful?
The FORMATIVE EVALUATION consists of a continuous review of learning materials during the design/development process. The interim results will be used for quality assurance and quality improvement during the implementation (process-quality assurance). This way it is possible to make improvements during the implementation of the application and to avoid costly mistakes. The SUMMATIVE EVALUATION assesses the quality/effectiveness of the finished product. Since the outcomes of the summative evaluation are not available until after the end of the operation, it is not possible to incorporate the improvements in the product's development. SUMMATIVE EVALUATION is useful when a number of applications are to be compared in its quality, or if the quality of an application is to be reviewed to decide whether the application should be used further. https://www.youtube.com/watch?v=mjmM1iN-m-E
Describe the four steps of the pyramid model, according to Kirkpatrick. What levels of the model should be covered by formative and which by summative evaluation?
The pyramid model of Kirkpatrick clarifies the subject area of the evaluation. Evaluation takes into account ... -REACTIONS of the students on the learning application or learning material through attitude tests. The acceptance and the attitudes of the students concerning the learning material are the main prerequisites for successful learning (level 1). -The extent of the LEARNING gain (what / how much has been learned) by way of a pre - and post- test comparison (level 2). -BEHAVIOR: The level of learning transfers achieved through observation of the participants in application situations. It should be ascertained whether the learning enables the users to better cope with problems and barriers of their everyday life (e.g. in the work environment) (level 3). -RESULTS of the activity in terms of structural and organizational aspects, (e.g. has the achieved behavioural change resulted in a better working environment, more performance by the employees, etc.) (level 4). LEVELS 1 AND 2 CAN BE COVERED WITH FORMATIVE EVALUATION. LEVELS 3 AND 4 ARE BETTER IN THE FORM OF A SUMMATIVE EVALUATION. Weird video describing it: https://www.youtube.com/watch?v=oBpI0B5PtaA
Identify a specific procedure for quality evaluation. Explain the connection between evaluation, quality management, and quality assurance.
With the help of evaluation, the quality of a product can be secured, improved or adapted to new requirements. Quality assurance is more extensive than the evaluation, but also uses approaches and methods of evaluation research (quality evaluation). Quality assurance is achieved through quality management. Quality management is a systematic procedure for the collection, protection and improvement of quality of a product. A special form of quality management is the "Total Quality Management".
