Chapter 6
Pilot testing refers to the process of previewing the training program with potential trainees and managers.
True
Formative evaluation usually involves collecting quantitative data through tests or objective measures of performance
False
It is important to conduct a return on investment (ROI) analysis for nearly all training programs.
False
Once the costs and benefits of the training program are determined, ROI is calculated by dividing costs by benefits.
False
Outcome measures are often perfectly related to each other.
False
Results are more frequently examined than reactions.
False
The Hawthorne effect refers to employees performing at a low level because of the attention they receive.
False
The believability of study results refers to external validity.
False
A comparison group refers to a group of employees who participate in the evaluation study but do not attend the training program.
True
A reliable test includes items for which the meaning or interpretation does not change over time.
True
It is necessary to limit return on investment (ROI) analysis to certain training programs as it can be costly.
True
Negative transfer is evident when learning occurs, but on-the-job behavior is lower than pre-training levels.
True
Pilot testing is more useful for formative evaluation than summative evaluation.
True
The Solomon four-group design combines the pre-test/post-test comparison group and the post-test-only control group design.
True
_____ relate to trainees' attitudes toward training content and motivation to transfer. a. Reactions b. Affective outcomes c. Cognitive outcomes d. Behavior and skill-based outcomes
b. Affective outcomes
Which of the following is an affective learning outcome? a. Learners' satisfaction with training b. Greater appreciation of diversity c. Increased production d. Reduced costs of production
b. Greater appreciation of diversity
Which of the following statements is most true of comparison groups? a. The use of a comparison group poses a threat to internal validity. b. Comparison group consists of a group of employees who do not attend the training program. c. Employees in a comparison group have personal characteristics that are different from other trainees. d. Use of a comparison group in training evaluation increases the possibility that changes found in the outcome measures are due to factors other than training.
b. Comparison group consists of a group of employees who do not attend the training program.
Behavior and skill-based outcomes are best measured by _____. a. surveys b. interviews c. focus groups d. observations
d. observations
_____ refers to the ease with which training outcomes can be collected. a. Reliability b. Practicality c. Acceptability d. Relevance
b. Practicality
Return on investment (ROI) analysis is best suited for training programs that are _____. a. attended by few employees b. inexpensive and have limited visibility c. significant financial investments d. one-time events
c. significant financial investments
Front-line supervisors are likely most concerned with which training outcomes? a. Reactions b. Return on expectations (ROE) c. Cognitive outcomes d. Behavior and skill-based outcomes
d. Behavior and skill-based outcomes
The evaluation process ideally begins with _____. a. developing outcome measures b. choosing an evaluation strategy c. developing measurable learning objectives d. conducting a needs analysis
d. conducting a needs analysis
Typically, _____ are used to assess cognitive outcomes. a. attitude surveys b. observations c. focus groups d. pencil-and-paper tests
d. pencil-and-paper tests
A firm that aims at improving readily observable outcomes such as productivity by collecting related data at periodic intervals before and after training is typically applying the _____ evaluation design. a. Solomon four-group b. pre-test/post-test c. return on investment d. time series
d. time series
Which of the following statements is true of random assignment? a. It assigns employees to a training program without considering their individual differences. b. It results in an unequal distribution of individual characteristics such as age, gender, and motivation. c. It increases the effects of employees dropping out of the study. d. It increases the differences between the training group and comparison group.
a. It assigns employees to a training program without considering their individual differences.
_____ demonstrates to key business stakeholders, such as top-level managers, that their expectations about training have been satisfied. a. Reactions b. Return on expectations (ROE) c. Results d. Return on investment (ROI)
b. Return on expectations (ROE)
_____ refers to concrete examples of the impact of training that show how learning has led to results that a company finds worthwhile and credible. a. Utility analysis b. Success cases c. Return on expectations (ROE) d. Outcome practicality
b. Success cases
A pencil-and-paper test is the best means for measuring skill-based outcomes.
False
Among all training outcomes, cognitive outcomes are most frequently assessed.
False
Evaluation designs without pre-test or comparison groups are most appropriate when companies are interested in determining how much change has occurred in trainees.
False
Favorable trainee reactions are strongly connected with transfer.
False
Which of the following statements best differentiates formative evaluation from summative evaluation? a. Formative evaluation mainly involves collecting quantitative data, whereas summative evaluation involves collecting qualitative data. b. Formative evaluation takes place on the completion of training, whereas summative evaluation takes place during program design and development. c. Formative evaluation focuses on how to make a training program better, whereas summative evaluation helps to determine the extent to which trainees have changed after training. d. Formative evaluation includes measuring the monetary benefits that a company receives from a training program, whereas summative evaluation measures beliefs and opinions of the trainees.
Formative evaluation focuses on how to make a training program better, whereas summative evaluation helps to determine the extent to which trainees have changed after training.
Which of the following statements is true of Solomon four-group evaluation design? a. It is mainly used by companies that are uncomfortable with excluding certain employees or that intend to train only a small group of employees. b. It is an evaluation design in which pre-training outcomes are completely ignored. c. It combines both pre-test/post-test comparison group and post-test-only control group design. d. Relative to the other evaluation designs, it is more economical and takes less time to conduct.
c. It combines both pre-test/post-test comparison group and post-test-only control group design.
Which training outcome relates to trainees providing feedback about their satisfaction with a trainer? a. Results b. Cognitive c. Reactions d. Behavior and skill-based
c. Reactions
_____ is a time period in which participants no longer receive training intervention. a. Regression b. Mortality c. Reversal d. Maturation
c. Reversal
If trainers are interested in the generalizability of a study's results to other groups of employees, then they are said to be interested in the _____ of the study. a. outcome practicality b. criterion relevance c. external validity d. outcome believability
c. external validity
_____ refers to the process of collecting the outcomes needed to determine whether training is effective. a. Training evaluation b. Program design c. Performance appraisal d. Needs assessment
a. Training evaluation
Formative evaluation involves collecting data about a training program from trainees mainly through _____. a. their opinions and feelings about the program b. measures of performance such as volume of sales c. tests and ratings of their behavior d. return on investment (ROI)
a. their opinions and feelings about the program
Cognitive outcomes generally do not help determine how much trainees use training content on the job.
True
Improved self-efficacy beliefs relate to an affective learning outcome.
True
Appropriate training outcomes need to discriminate. This means that _____. a. tests given to employees before and after a training program should differ b. trainees should be asked to take a reliable test that includes items for which the meaning or interpretation change over time c. trainees' performance on the outcome should actually reflect true differences in performance d. different employees should be given different tests for measuring their performance on the same outcome
c. trainees' performance on the outcome should actually reflect true differences in performance
In the context of big data, _____ refers to the large number of sources and types of data. a. valence b. volume c. variety d. velocity
c. variety
_____ relate to whether trainees are using training content back on the job. a. Reactions b. Learning c. Cognitive outcomes d. Behavior and skill-based outcomes
d. Behavior and skill-based outcomes
If a firm measures its sales volume before and after a training program, which training outcome is it focused on? a. Reactions b. Return on expectations (ROE) c. Return on investment (ROI) d. Results
d. Results
Evaluation designs without pre-test or comparison groups are most appropriate when _____. a. information regarding training effectiveness is not needed immediately b. companies are interested in determining how much change has occurred c. a company has a strong orientation toward evaluation d. a company is only interested in whether trainees have achieved a certain proficiency level
d. a company is only interested in whether trainees have achieved a certain proficiency level
In the context of big data, _____ refers to the huge amount of data that is being generated and the speed with which it must be evaluated, captured, and made useful. a. valence b. volume c. variety d. velocity
d. velocity
