P&T Ch 6

Summative evaluation refers to

-evaluation conducted to determine the extent to which trainees have changed as a result of training. That is, summative evaluation examines whether trainees have improved or acquired knowledge, skills attitudes, behaviors, or other outcomes. Summative evaluation may also include examining the business impact of training.

Formative evaluation refers to

-the evaluation of training that takes place during program design. -helps to ensure that the training program is well organized and runs smoothly and that trainees learn and are satisfied. - provides information about how to make a program better.

The evaluation process involves five key components:

1. Needs assessment 2. Develop measurable learning objectives and analyze transfer of training 3. Development outcome measures 4. Choose an evaluation strategy 5. Plan and execute the evaluation

________ _________ refers to the generalizability of the evaluation results to other groups and other situations.

External validity refers to the generalizability of the evaluation results to other groups and other situations.

____________ is the ease with which the outcome measures can be collected.

Practicality is the ease with which the outcome measures can be collected. One reason companies give for not including learning, performance, and behavior outcomes is that collecting them is too burdensome.

How is skill transfer assessed?

assessed by observing trainees on the job or via managerial and peer ratings.

Affective outcomes include... Affective outcomes that might be collected include...

attitudes and motivation self-efficacy, employee engagement, motivation to learn, tolerance for diversity, safety attitudes, and customer service orientation.

____ ______ refer to complex datasets developed by compiling data across different organizational systems, including .....

Big data marketing and sales, HR, finance, accounting, customer service, and operations.

How is skill learning assessed?

By observing performance in work samples

_______ __________ refers to the extent to which training outcomes are related to the learned capabilities emphasized in the program.

Criteria relevance

__________ _________ refer to the trainees' perceptions of the training experience relating to the content, facilities, trainer, and methods of delivery.

Reaction Outcomes

_________ is whether outcomes can be measured consistently over time.

Reliability Reliability is whether outcomes can be measured consistently over time. Predominantly, we are concerned with consistency over time, such that a reliable test contains items that do not change in meaning or interpretation over time.

________ ____ ________ involves comparing the training program's benefits in monetary terms to the program's costs.

Return on Investment involves comparing the training program's benefits in monetary terms to the program's costs.

reaction are often referred to as a measure of "_________ ________"

creature comfort

Use pre-tests and post-tests to

determine the extent to which trainees have changed from pre-training to post-training measures. The pre-training measure essentially establishes a baseline.

_____________ refers to whether performance on the outcome reflects true differences in performance.

discrimination For example, we want tests that can discriminate between high and low performance. A test that may not discriminate is one that is too easy. In this instance, both high and low performers would do well on the test. Both groups would appear "good" even though they are not.

Random assignment of employees to control and training groups. Randomization helps to

ensure that members of the control group and training group are of similar makeup prior to the training.

Learning or cognitive outcomes relate to

familiarity with information, including principles, facts, techniques, procedures, and processes. Paper and pencil tests or self-assessments are used to assess cognitive outcomes.

__________ __________ is the believability of the study.

internal validity An evaluation needs internal validity to provide confidence that the findings are due to training and not another factor.

formative evaluation involves _______ ________ which involves

pilot testing the process of previewing a training program with potential trainees and their managers. The pilot testing group is then asked to provide feedback about the content and the methods of delivery. This feedback enables the trainer to make any needed improvements.

Behavior and skill-based outcomes relate to

proficiency with technical or motor skills and behavior. These outcomes include both the learning of skills and their transfer.

The attitude of interest depends on training objectives. Affective outcomes can be measured using _______.

surveys.

• Volume refers to

the amount of available data.

• Velocity refers to

the huge amount of data that is being generated and the speed with which it must be evaluated, captured, and made useful.

• Variety includes

the large number of sources and types of data.

reaction outcome is collected when?

the program's conclusion through a questionnaire

Results are Examples include

those outcomes used to determine the benefits of the training program to the company. Examples include reduced costs, increased employee retention, increased sales, increased production, and improved quality or customer service.

Use a control group (i.e., a group that participates in the evaluation study, but does not receive the training) to

to rule out factors other than training as the cause of changes in the trainees.

Three dimensions characterize big data:

volume, variety, and velocity.

key question to consider for reaction outcomes are:

• Did the trainees like the program? • Did the environment help learning? • Was material meaningful?

There are multiple reasons to evaluate training effectiveness:

• To identify the program's strengths and weaknesses, including whether the program is meeting the learning objectives, the quality of the learning environment, and if transfer is occurring. • To assess whether the various features of the training context and content contribute to learning and transfer. • To identify which trainees benefited most or least from the program and why. • To gather information, such as testimonials, to use for marketing training. • To determine financial benefits and costs of the program. • To compare the costs and benefits of training versus other HRM investments. • To compare the costs and benefits of various training programs in order to choose the most effective programs.

3 Methods to control for threats to validity:

• Use pre-tests and post-tests to determine the extent to which trainees have changed from pre-training to post-training measures. The pre-training measure essentially establishes a baseline. • Use a control group (i.e., a group that participates in the evaluation study, but does not receive the training) to rule out factors other than training as the cause of changes in the trainees. • Random assignment of employees to control and training groups. Randomization helps to ensure that members of the control group and training group are of similar makeup prior to the training.