Chapter 6: Training Evaluation

What are the steps in the evaluation process?

1. Conduct a needs analysis 2. Develop measurable learning objectives and analyze transfer of training 3. Develop outcome measures 4. Choose an evaluation strategy 5. Plan and execute the evaluation

What are threats to validity?

Factors that will lead an evaluator to questions the believability of the results or the extent to which the results are generalizable to other groups

Outcome measures are often perfectly related to each other. True False

False

Once the costs and benefits of the training program are determined, ROI is calculated by dividing costs by benefits. True False

False; Net Benefits by Net Cost

A pencil-and-paper test is the best means for measuring skill-based outcomes. True False

False; cognitive

Which of the following is true of random assignment? It assigns employees to a training program without considering their individual differences. It results in an unequal distribution of individual characteristics such as age, gender, and motivation. It increases the differences between the training group and comparison group. It increases the effects of employees dropping out of the study

It assigns employees to a training program without considering their individual differences.

What is the goal of big data?

Make decisions about human capital based on data rather than intuition or conventional wisdom, which likely lead to incorrect conclusions and recommendations.

Behavior and skill-based outcomes are best measured by which of the following? Observations Surveys Interviews Focus groups

Observations

What is training effectiveness?

The benefits that the company and the trainees receive from training. Benefits are learning new skills or behaviors, and increased sales, more satisfied customers

What is evaluation design?

The collection of information including what, when, how, and from whom that will be used to determine the effectiveness of the training program.

What is reliability?

The degree to which outcomes can be measured consistently over time.

What is criterion contamination?

The extent that training outcomes measure inappropriate capabilities or are affected by extraneous conditions.

What is criterion deficiency?

The failure to measure training outcomes that were emphasized in the training objectives.

What are results?

Used to determine the training program's payoff for the company. e.g. increased production, reduced turnover, customer service.

In the context of big data, which of the following refers to the huge amount of data that is being generated and the speed with which it must be evaluated, captured, and made useful? Variety Volume Valence Velocity

Velocity

What is summative evaluation?

An evaluation conducted to determine the extent to which trainees have changed as a result of participating in the training.

Determining ROI begins with what?

An understanding of the objectives of the training programs

What are affective outcomes and how can they be measures?

Attitudes and motivation can be measured using surveys

The evaluation process ideally begins with which of the following? Developing outcome measures Choosing an evaluation strategy Developing measureable learning objectives Conducting a needs analysis

Conducting a needs analysis

What is formative evaluation?

Refers to the evaluation of training that takes place during program design and development. Helps to ensure that the program is well organized and trainees learn

If a firm measures its sales volume before and after a training program, which training outcome is it focused on? Return on investment (ROI) Results Return on expectations (ROE) Reactions

Results

Return on investment (ROI) analysis is best suited for training programs that are which of the following? Attended by few employees One-time events Inexpensive and have limited visibility Significant financial investments

Significant financial investments

Which of the following refers to concrete examples of the impact of training that show how learning has led to results that a company finds worthwhile and credible? Success cases Return on expectations (ROE) Outcome practicality Utility analysis

Success cases

When should a rigorous evaluation design be used?

The results can be used to change the program Training is ongoing Large number of trainees Cost justification is based on numerical indicators. Cost creates a need to show it works Comparing two or more programs

A comparison group refers to a group of employees who participate in the evaluation study but do not attend the training program. True False

True

Pilot testing refers to the process of previewing the training program with potential trainees and managers. True False

True

Research has found that reactions are significantly related to changes in declarative and procedural knowledge. True False

True

What is pilot testing?

the process of previewing the training program with potential trainees and managers or with other customers. a dress rehearsal.