6. Training Evaluation (Chapter 6)

Logic model

Process used to identify the relationship between training resources, training activities, and program outcomes

Net Promoter Score (NPS)

Score designed to measure employees' satisfaction with a course offering by asking them to rate the likelihood of recommending it to a peer by using a 0-10 point scale

Training effectiveness

The benefits that the company and the trainees receive from training

Practicality

The ease with which the outcome measures can be collected

Skill-based outcomes

Used to assess the level of technical or motor skills and behaviors

Cognitive outcomes

Used to determine the degree to which trainees are familiar with the principles, facts, techniques, procedures, and processes emphasized in the training program

Results

Used to determine the training program's payoff for the company

Workforce analytics

Using quantitative methods and scientific methods to analyze data from HR databases, corporate financial statements, employee surveys, and other data sources to make evidence-based decision and show that HR practices influence important company metrics

Benefits

Value a company gains from the training program

Criterion contamination

Extent that training outcomes measure inappropriate capabilities or are affected by extraneous conditions

Criteria relevance

Extent to which training outcomes are related to the learned capabilities emphasized in the training program

Threats to validity

Factors that will lead an evaluator to question either (1) the believability of the study results or (2) the extent to which the evaluation results are generalizable to other groups of trainees and situations

Criterion deficiency

Failure to measure training outcomes that were emphasized in the training objectives

External validity

Generalizability of the study results to other groups and situations

Affective outcomes

Include attitudes and motivation

Direct costs

Include salaries and benefits for all employees involved in training, including trainees, instructors, consultants, and employees who design the program; materials and supplies; equipment or classroom rentals or purchases; and travel costs

Self-assessments

Learners' estimates of how much they know or have learned from training

Post-training measure

Measure of outcomes taken after training

Training outcomes/criteria

Measures that the trainer and the company use to evaluate training programs

Training evaluation

Process of collecting the outcomes needed to determine whether training is effective

Cost-benefit analysis

Process of determining the economic benefits of a training program using accounting methods that look at training costs and benefits

Pilot testing

Process of previewing the training program with potential trainees and managers or with other customers (persons who are paying for the development of the program)

Return on expectations (ROE)

Process through which evaluation demonstrates to key business stakeholders that their expectations about training have been satisfied

Pretraining measure

A baseline measure taken before training begins

Dashboard

A computer interface designed to receive and analyze the data from departments within the company to provide information to managers an other decision makers

Utility analysis

A cost-benefit analysis method that involves assessing the dollar value of training based on estimates of the difference in job performance between trained and untrained employees, the number of individuals trained, the length of time a training program is expected to influence performance, and the variability in job performance in the untrained group of employees

Comparison group

A group of employees who participate in the evaluation study but do not attend the training program

Reversal

A time period in which participants no longer receive the training intervention

Summative evaluation

An evaluation conducted to determine the extent to which trainees have changed as result of participating in the training program

Pretest/post-test

An evaluation design in which both pretraining and post-training outcome measures are collected

Post-test only

An evaluation design in which only post-training outcomes are collected

Time series

An evaluation design in which training outcomes are collected at periodic intervals both before and after training

Pretest/post-test with comparison group

An evaluation design that includes trainees and ac comparison group

Random assignment

Assigning employees to the training or comparison group on the basis of chance alone

Internal validity

Believability of study results

Evaluation design

Collection of information - including what , when, how, and from whom - that will be used to determine the effectiveness of the training program

Solomon four-group

Combines the pretest/post-test comparison group and the post-test-only control group design

Return on Investment (ROI)

Comparing the training's monetary benefits with the cost of training

Big data

Complex data sets developed by compiling data across different organizational systems, including marketing and sales, HR, finance, etc.

Success cases or stories

Concrete examples of the impact of training that show how learning has led to results that the company finds worthwhile and the managers find credible

Indirect costs

Costs that are not related directly to design, development or delivery of the training program

Reliability

Degree to which outcomes can be measured consistently over time

Discrimination

Degree to which trainees' performance on the outcome actually reflects true differences in performance

Hawthorne effect

Employees in an evaluation study performing at a high level simply because of the attention they are receiving

Formative evaluation

Evaluation of training that takes place during program design and development