Research Exam 2 Ch. 7, 11, 14, & 15

In a paragraph, explain the purpose and procedures of a meta-analysis to someone who has never heard of the research technique.

-definitive methodology is reported concerning the decisions in literature analysis.-results of many studies are quantified and to effect size, that allows statistical techniques as a means of analysis.

What are three critical things that could be checked by doing a pilot study involving an electronic questionnaire survey?

-send a practice round of the survey out to classmates/professour-confirm email address if sampling is needed-make sure to send out the correct link to the survey

In an item analysis, 14 of the 20 high-ability students answered an item correctly, whereas only 8 of the 20 low-ability students did. What is the discrimination index for this item? a. .30 b. .40 c. .50 d. .60 e. .70

a. .30

Which hypothesis is stated in the null form? a. There is no difference between the vocabulary scores of average- and high-ability students. b. The math achievement scores from School A are significantly higher than the scores from School B. c. The perceptual-motor development of first-grade girls is higher than that of first-grade boys. d. There is a positive relationship between attitude toward school and achievement scores.

a. There is no difference between the vocabulary scores of average- and high-ability students.

The inclusion criteria in a meta-analysis are very much like a. delimitations b. limitations c. basic assumptions d. research hypotheses

a. delimitations

Explain how a single study can have more than one ES. Give an example.

if you are comparing men and women in a study you would have an ES for males and another for females

Contrast the electronic questionnaire with the personal interview in terms of advantages or disadvantages in sampling, validity of responses, and administrative considerations (time, expense, helpers).

in person interviews are time consuming-pilot study must make sure the vocabuary is appropriate and the questions are equal to all being interviewed.-do not inject your own bias or argue with the respondent-disadvantage could be how easy it is to lose data..

Explain what is meant by the statement that a statistical test may be statistically significant but not meaningful. Give an example of such a circumstance.

take average road speed that two different samples drove. There is a statistically different because the speed limits could be different from where each car is from. It's not meaningful because the cars because the areas have different speed limits set.

Explain the basis for the statement that representativeness of the sample is more important than size of the sample.

the sample should be as specific and accurate as possible. a sample that calls for high school students but 50% that take the survey are freshman will skew the results.

What are three things a researcher could do to increase the power if the study were done a second time?

1. Use larger sample 2. Improve your process 3. Use a higher significance level

Discuss how the concept of standard error of measurement relates to one's interpretation of a student's score on a test.

The standard error of measurement estimates how repeated measures of a person on the same instrument tend to be distributed around his or her "true" score. The true score is always an unknown because no measure can be constructed that provides a perfect reflection of the true score.

Comparing test items with the course objectives (course topics) checks which type of validity? a. content b. predictive c. concurrent d. construct

a. content

A physical education teacher develops a skill test in volleyball. After administering the test to 50 students, she asks the volleyball coach to rate the students on volleyball skills. She then correlates the students' test scores with the coach's rating. This is an example of what type of validity? a. criterion validity b. logical validity c. content validity d. face validity

a. criterion validity

Choose the correct sequence for developing a questionnaire. a. determining objectives, delimiting the sample, writing items, pilot testing b. pilot testing, determining objectives, writing cover letter, writing items c. delimiting the sample, pilot testing, determining objectives, writing items d. delimiting the sample, determining objectives, writing items, pilot testing

a. determining objectives, delimiting the sample, writing items, pilot testing

A major difference in the purpose of a meta-analysis and a review of literature for a research paper is that the meta-analysis uses the literature a. for empirical and theoretical conclusions b. to document a need for a particular study c. to determine what methods to use to conduct the meta-analysis d. to formulate the research hypothesis for the meta-analysis

a. for empirical and theoretical conclusions

When a tester divides the number of students who answered a test item correctly by the number of students who attempted the item, the result is called a. index of difficulty b. index of discrimination c. item variance d. KR-20 index

a. index of difficulty

A researcher tests the knowledge that 10,000 school children from the southwestern United States have about nutrition, and the scores are transformed into percentiles by grade level. This type of research is a ________ survey study. a. normative b. developmental c. correlational d. longitudinal e. multiple-case

a. normative

Which of the following are most difficult to analyze in a questionnaire study? a. open-ended items b. yes/no items c. Likert-type items d. semantic differential it

a. open-ended items

When a national poll reports that its survey has a margin of error of 3%, it is referring to error due to a. sampling b. respondents not understanding questions c. misinterpreting the results d. recording the response

a. sampling

In calculating ES for a treatment effect between a pretest and a posttest, the text recommends the use of which standard deviation? a. the pretest standard deviation b. the posttest standard deviation c. the pooled standard deviation

a. the pretest standard deviation

When a researcher claims that there is a difference between treatments (i.e., rejects the null hypothesis) when there really is no difference, what type of error is this? a. type I error b. type II error

a. type I error

When an experimenter states that the level of significance is the .05 level, he is setting the probability of committing which type of error? a. type I error b. type II error

a. type I error

The maximum value of item discrimination in norm-referenced test items occurs when the item difficulty is a. .05 b. .50 c. .75 d. 1.00

b. .50

Scaled responses to questionnaire items such as strongly agree, agree, no opinion, and so on are referred to as ________ scale items. a. Gulick b. Likert-type c. semantic differential d. Binet

b. Likert-type

Which of the following is a technique for providing information about the reliability of an instrument? a. bivariate prediction b. coefficient alpha c. factor analysis d. chi square

b. coefficient alpha

The first step in conducting a questionnaire survey is to a. select the sample b. list specific objectives to be achieved c. construct the questionnaire items d. write the cover letter

b. list specific objectives to be achieved

It is common when looking at a distribution to encounter extreme or unusual scores that could have a misleading effect on the results. These extreme scores are termed a. residuals b. outliers c. curve busters d. deviants

b. outliers

In a meta-analysis, the ES for the experimental versus control groups was 0.68, which represents a percentile equivalent of 75. This means that a. the control group's average performance was 68 out of 100 compared with 75 out of 100 for the experimental group b. the average score of the experimental group was better than 75% of the control group's scores c. the probability is 75% that the use of the experimental treatment will improve the average student's score from the 50th to the 68th percentile d. the control group's mean was only 75% as high as that of the experimental group

b. the average score of the experimental group was better than 75% of the control group's scores

Consider the following questionnaire item (which uses a yes/no format): "Do you use objective test items or essay items?" What rule regarding item construction does it violate? a. Avoid using negative items. b. Do not lead the respondent to answer in a certain way. c. Do not use items that have two or more separate ideas in the same item. d. Short questions are easier to understand.

c. Do not use items that have two or more separate ideas in the same item.

Validity is determined by finding the correlation between scores on a. the even-numbered items and the odd-numbered items on a test b. one form of a test and another form of that same test c. a test and some independent criterion d. two administrations of the same test

c. a test and some independent criterion

A predictive validity technique involving the construction of a two-way grid of percentages of students at different levels of an aptitude test who achieved different grades is called a. cross validation b. the known group differences c. an expectancy table d. index of discrimination

c. an expectancy table

The practice of selecting a small random sample of a survey's nonrespondents is primarily intended to a. try to raise the percentage of returns to at least 50% b. identify outliers c. determine whether the nonrespondents are a different population from the respondents d. ensure anonymity of respons

c. determine whether the nonrespondents are a different population from the respondents

A researcher decides to use an alpha of .01 and a power of .80. To determine the needed sample size the researcher must ascertain the expected a. beta b. stem-and-leaf distribution c. effect size d. skewness

c. effect size

In a meta-analysis, the results of various studies on a topic are quantified using a standard measure (ES) that is called a. error squared b. estimate of significance c. effect size d. efficiency standard

c. effect size

A threat to the validity of rating scales is that the rater may tend to let her previous knowledge of the student in another capacity (e.g., another course) influence the rating. This threat is called the a. Hawthorne effect b. personal bias effect c. halo effect d. reminiscence effect

c. halo effect

If a thermometer measured the temperature in an oven as 400° five days in a row when the temperature was actually 337°, this measuring instrument would be considered a. reliable and valid b. valid but not reliable c. reliable but not valid d. unreliable and invalid

c. reliable but not valid

The type of scale that uses items such as "Worthless-Valuable" is called a a. Thorndike scale b. Thurstone scale c. semantic differential scale d. Likert scale

c. semantic differential scale

If a researcher finds a small difference in average test scores between a large sample (over 700) of experimental participants and a large sample (same size) of control participants, it is very likely that the difference is a. statistically significant and has a high degree of meaningfulness b. not statistically significant but has a high degree of meaningfulness c. statistically significant but does not have a high degree of meaningfulness d. neither statistically significant nor meaningful

c. statistically significant but does not have a high degree of meaningfulness

When a researcher states that a result is significant, this means that a. the effect is an especially important one b. the scores are not correlated c. the result is unlikely to be a chance occurrence d. the scores are correlated

c. the result is unlikely to be a chance occurrence

A survey technique that asks a jury of experts to respond to a series of questionnaires in order to reach a consensus about important issues is the a. normative survey method b. obtrusive survey method c. roundtable survey method d. Delphi survey method

d. Delphi survey method

The use of intraclass correlation to estimate reliability in motor performance instead of Pearson r has the advantage(s) of a. allowing analysis of trial-to-trial change b. estimating reliability for more than two trials c. adjusting for differences among individuals d. a and b e. a, b, and c

d. a and b

Reliability of a measure may be established by a. giving the test to the same people on two different occasions and correlating the two sets of scores b. correlating scores from alternate forms of the same test c. correlating the test with some independent criterion d. a and b only e. a, b, and c

d. a and b only

The author of a test of anxiety proceeds on the premise, based on the literature, that the performance of people with high anxiety suffers when they are under stress. In an experiment, the author finds that people who scored as highly anxious on the test performed more poorly on a stressful task. The test author maintained that this finding was evidence of what kind of validity? a. concurrent b. content c. predictive d. construct

d. construct

A teacher wishes to determine the reliability of three trials on a performance test. He uses ANOVA to obtain the reliability coefficient. This technique is called a. interclass correlation b. partial correlation c. equivalence correlation d. intraclass correlation

d. intraclass correlation

The most frequent and severe criticism of meta-analysis is that a. the number of studies involved is generally too small b. the sources used in the literature search are not identified c. meta-analysis does not allow the use of dissertations and theses d. the studies represent wide differences in methodologies, designs, and measurements

d. the studies represent wide differences in methodologies, designs, and measurements

A measure of meaningfulness that expresses the difference between the experimental and control group in standard deviation units is the a. effect size b. r squared c. omega squared d. chi square

effect size

The fact that a meta-analysis quantifies findings from many studies is a frequently mentioned advantage. Another strong advantage of the meta-analysis over the normal review paper is a. the use of the computer in the literature search b. the description of the coding of study characteristics and the criteria for excluding studies c. the inclusion of theses and dissertations as well as journals d. the use of the "scorecard" count of the number of studies that share common findings (such as those that found statistically significant differences)

b. the description of the coding of study characteristics and the criteria for excluding studies

In a meta-analysis, the researcher found that the mean for the experimental group in a study was 14, and the mean for the control group was 10. The standard deviation of the control group was 5. Based on these data, what is the ES? a. 5.0 b. 4.0 c. 0.80 d. 0.50 e. 0.20

c. 0.80

In a meta-analysis of 100 studies that compared boys' and girls' knowledge of AIDS, an average ES of 0.50 was found. This number represents a. a correlation of 0.50 b. a mean square (variance) of 0.50 c. a t ratio of 0.50 d. a difference of 0.50 standard deviation units

d. a difference of 0.50 standard deviation units

A null hypothesis is a. nearly always the same as the research hypothesis b. a statistical hypothesis that assumes that the participants in two treatment groups were unequal before treatment began c. a statistical hypothesis that assumes that there is a difference among the effects of treatments d. a statistical hypothesis that assumes that there is no difference among the effects of treatments

d. a statistical hypothesis that assumes that there is no difference among the effects of treatments

For scores from a test to have good content validity, the following statement(s) must be true: a. The test is highly correlated with the final grade in the course. b. The test adequately samples what was covered in the course. c. The percentage of points for each topic area reflects the amount of emphasis given that topic. d. b and c only e. a, b, and c

d. b and c only

To help ensure an acceptable return rate in a questionnaire study, the researcher should a. make liberal use of open-ended questions b. use follow-up letters or e-mails c. enclose or attach another copy of the survey d. b and c only e. a, b, and c

d. b and c only

The accuracy with which a 12-min run estimates maximal oxygen consumption in a group of male high school seniors represents a. logical validity b. content validity c. construct validity d. concurrent validity

d. concurrent validity

If the researcher fails to reject the null hypothesis when there really is a difference, this is an example of a a. one-tailed test b. two-tailed test c. type I error d. type II error

d. type II error

One form of descriptive research uses surveys to gather data. These surveys may use different types of questions (e.g., fill-in, Likert-type, short answers) in the same instrument. Assume a survey has a number of Likert-type questions. List the steps an author should take to make certain the instrument is reliable and valid. Be specific and provide examples to explain your answer.

intervals between responses should be equal.example: strongly disagree and disagree should be the same as strongly agree and agree.

Discuss the ramifications of the "file-drawer" effect on meta-analyses.

papers with significant findings that are not peer reviewed are constantly being used in research. the meta-analysis should get a total of all peer reviewed and non. they should then get the ES of each and calculate the ratio of peer reviewed to non peer reviewed.