Research: Exam 2

3. internal reliability

(also called internal consistency), a study participant gives a consistent pattern of answers, no matter how the researcher has phrased the question

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) An individual's gender

Nominal/categorical

"If an observational study suffers from '"observer effects", then the researcher might be measuring ____ instead of ________.

conformed behavior or outcome; the actual behavior or outcome

2. interratter reliability

consistent scores are obtained no matter who measures the variable

conceptual variables

construct, is the researcher's definition of the variable in question at a theoretical (theory) level

survey/poll

both mean the same thing: a method of posing questions to people on the phone, in personal interviews, on written questionnaires, or online

Would the correlation coefficient (r) be expected to be high (above 0.7) or low (below 0.7) if we are measuring a stable trait like intelligence?

high (above 0.7)

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) IQ score

interval

how to avoid "reactivity"?

-Another way to avoid reactivity is to use unobtrusive data. Instead of observing behavior directly, researchers measure the traces a particular behavior leaves behind -unobtrusive observations, wait it out, & measure the behavior's results

how to avoid "observer effects"?

-Solution 1: Blend In. One way to avoid observer effects is to make unobtrusive observations—that is, make yourself less noticeable -Masked or "blind" research designs, train observers well, multiple observers

what is used to see if measurement scales has good internal reliability (0.7 or higher). This measures the consistency of responses across items in self-report scales (for example, a well-being scale). The different items on the scale need to be strongly correlated with one another.

Cronbach's alpha

what approach do researchers take minimize problems when using observer bias?

Instead of rating behaviors objectively, observers rate behaviors according to their own expectations or hypotheses

Which Operationalization type is best?

Many people erroneously believe physiological measures are the most accurate, but even their results have to be validated by using other measures

how does social desirability bias/ faking good weaken contract validity

Most of us want to look good in the eyes of others, but when survey respondents give answers that make them look better than they really are, these responses decrease the survey's construct validity

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) Measuring academic majors for a group of college students (i.e. art, biology, psychology)

Nominal/categorical

Identify as self-report/ observational/physiological: Measuring stressful events by examining public records of divorce

Observational

Identify as self-report/ observational/physiological: Heart rate

Physiological

how do response sets/non differentiation weaken construct validity?

Response sets weaken construct validity because these survey respondents are not saying what they really think

1. social desirability bias/ faking good

The idea is that because respondents are embarrassed, shy, or worried about giving an unpopular opinion, they will not tell the truth on a survey or other self-report measure.

How can the question order affect survey responses? How can you control for this?

The order in which questions are asked can also affect the responses to a survey. The earlier questions can change the way respondents understand and answer the later questions

observational research

When a researcher watches people or animals and systematically records how they behave or what they are doing

advantages and disadvantages of forced-choices questions?

advantage: get the wanted anwser quicker with less room for error disadvantage: may still get an unwanted answer

advantages and disadvantages of open-ended questions?

advantage: gives people a chance to voice exactly how they perceived or understood the question without any interaction disadvantage: may get the wrong answer completely because there was no guiding

how can you avoid a double-barreled question?

ask each question separately

how can you avoid a negatively worded question?

asking questions two different ways (helps with internal consistency)

Which of the following indicates divergent/discriminant validity? a. A strong correlation between scores on tests for depression and perceived physical health b. A weak correlation between scores on tests for autism and language delays.

b. A weak correlation between scores on tests for autism and language delays.

Dr. Papp is trying to decide whether he should use SAT scores to decide on accepting students into college. If he looks at the correlation between high SAT scores and high GPA, he is trying to establish________. a. Convergent validity b. Criterion validity c. Face validity d. Divergent validity

b. Criterion validity

Educational psychologists use teacher ratings of classroom shyness to measure children's temperament. Which kind of reliability would have to be evaluated in this scenario? a. Test-retest b. Interrater c. Internal d. a & b

d. a & b

John's doctor checked his blood pressure at 1:30 pm and then again at 1:35 pm and found it to be the same at both times. What type of reliability is this? a. Test-retest b. Interrater c. Internal d. a & c

d. a & c

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) In golf, "par for the course" is a predetermined number of strokes that a golfer should require to complete 18 holes of golf. For most golf courses, this number is 72. A golfer is measuring her or his golf score as the amount above or below par (e.g., a score of +1, or 1 above par, would mean that it took 73 strokes to complete 18 holes).

interval

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) Scores on a personality test assessing whether an individual is an extrovert or an introvert, with a score of "0" indicating a complete introvert and "10" indicating a complete extrovert

interval

1. ordinal scale

measurement applies when the numerals of a quantitative variable represent a ranked order

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) In a study on perception of facial expressions, participants must classify the emotions displayed in photographs of people as anger, sadness, joy, disgust, fear, or surprise

nominal

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) Political party affiliation

nominal/ categorical

Identify as self-report/ observational/physiological: Intelligence tests

observational

Identify as self-report/ observational/physiological: Recording jaywalking

observational

Identify as self-report/ observational/physiological: Tracking footprints in the snow

observational

observer bias

occurs when observers' expectations influence their interpretation of the participants' behaviors or the outcome of the study

1. leading question

one whose wording leads people to a particular response because it explains why some people oppose the law

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) An individual's time in completing the Peachtree Road Race

ratio

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) The number of questions a student completes correctly on this exercise

ratio

Operational variables

represents a researcher's specific decision about how to measure or manipulate the conceptual variable

Identify as self-report/ observational/physiological: Asking a teacher to rate a child's shyness

self-report

"If an observational study suffers from "observer bias", then the researcher might be measuring "the observers' expectations" instead of ________.

the actual behavior or outcome

"If an observational study suffers from "reactivity", then the researcher might be measuring ____ instead of "the typical behavior".

the best/worst behavior

Operationalization

the process of turning a construct of interest into a measured or manipulated variable 1. conceptual variables 2. Operational variables

1. test-retest reliability

the researcher gets consistent scores every time he or she uses the measure

what is the connection between surveys/polls and self-report data?

they both use personal interviews or written questionnaires

A researcher's population of interest is NYC dog owners. Which of the following samples is most likely to generalize to this population of interest? a) a sampling of 25 dog owners selected at random from NYC pet registration records b) a sampling of 25 dog owners visiting dog-friendly parks in NY c) a sampling of 25 dog owners who visit NYC's ASPCA website d) a sampling of 25 dog owners who have appointments for their dogs at vet clinics in the NYC area

a) a sampling of 25 dog owners selected at random from NYC pet registration records

Externally valid samples are more important for some research questions than for others. For which of the following research questions will it be most important to use an externally valid sampling technique? a) estimating the proportion of the US teens who are depressed b) testing the association between depression and illegal drug use in US teens c) testing the effectiveness of support groups for teens with depression

a) estimating the proportion of the US teens who are depressed

Two observers are categorising juvenile gorilla behaviours as "play fight" versus "aggressive fight". Which kind of reliability will need to be evaluated ? a) test-retest reliability b) internal reliability c) interrater reliability

a) test-retest reliability

Educational psychologists use teacher ratings of classroom shyness to measure children's temperament on a nine-point scale. How would you establish criterion validity for this rating? What about convergent validity? Divergent validity?

-Establish criterion validity by identifying relevant behaviors (how many times kids raise hands in class to answer questions, etc.) and seeing if there is a strong correlation. -Convergent validity: Traits that correlate strongly with shyness? Introversion? Parent and therapist ratings? Divergent validity: Traits that correlate weakly with shyness? Anxiety, Autism Spectrum

advantages of observational research?

-Some scientists believe observing behavior is better than collecting self-reports through surveys, because people cannot always report on their behavior or past events accurately, as we've discussed -Given the potential for the effect of question order, response sets, socially desirable responding, and other problems, many psychologists trust behavioral data more than survey data, at least for some variables

2. response sets/ non differentiation

-are a type of shortcut respondents can take when answering survey questions. -Ex. acquiescence: or yeasaying; this occurs when people say "yes" or "strongly agree" to every item instead of thinking carefully about each one -Ex. Fence sitting: playing it safe by answering in the middle of the scale, especially when survey items are controversial

3. negatively worded question

-are another way survey items can be unnecessarily complicated. Whenever a question contains negative phrasing, it can cause confusion, thereby reducing the construct validity of a survey or poll -Ex.does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened?" v. s. "Does it seem possible to you that the Nazi extermination of the Jews never happened, or do you feel certain that it happened?'

Categorical variables

-categories -Also called nominal variables -Ex. sex, whose levels are male and female; and species, whose levels in a study might be rhesus macaque, chimpanzee, and bonobo

Observer Effects

-expectancy effects, this phenomenon can occur even in seemingly objective observations -Ex. "Bright and Dull Rats": In a classic study of observer effects, researchers Rosenthal and Fode (1963) gave each student in an advanced psychology course five rats to test as part of a final lab experience in the course -Ex. "Clever Hans": A horse nicknamed Clever Hans provides another classic example of how observers' subtle behavior changed a subject's behavior, and how scientifically minded observers corrected the problem

2. forced-choice question

-in which people give their opinion by picking the best of two or more options. Forced Choice questions are often used in political polls, such as asking which of two or three candidates respondents plan to vote for. -Ex. An example of a psychology measure that uses forced-choice questions is the Narcissistic Personality Inventory (NPI; Raskin & Terry, 1988). This instrument asks people to choose one statement from each of 40 pairs of items, such as the following: -1. I really like to be the center of attention. !!!It makes me uncomfortable to be the center of attention. -2. I am going to be a great person. !!!I hope I am going to be successful

Which of the following four terms does not belong? a) generalizable sample b) externally valid sample c) representative sample d) biased sample

d) biased sample

4. semantic differential format

-instead of degree of agreement, respondents might be asked to rate a target object using a numeric scale that is anchored with adjectives -Ex. on the Internet site RateMyProfessors, students assign ratings to a professor using the following adjective phrases:

Reactivity

-is a change in behavior when study participants know another person is watching. They might react by being on their best behavior—or in some cases, their worst—rather than displaying their typical behavior. Occurs in humans and animals. -Ex. in a museum, wear-and-tear on the flooring can signal which areas of the museum are the most popular, and the height of smudges on the windows can indicate the age of visitors. The number of empty liquor bottles in residential garbage cans indicates how much alcohol is being consumed in a community (Webb, Campbell, Schwartz, & Sechrest, 1966). Researchers can measure behavior without doing any direct participant observation.

2. double-barreled question

-it asks two questions in one. The wording of a question is sometimes so complicated that respondents have trouble answering in a way that accurately reflects their opinions. -Double-barreled questions have poor construct validity because people might be responding to the first half of the question, the second half, or both.

Quantitative variables

-meaningful numbers -Ex. Height and weight because they are measured in numbers, such as 170 centimeters or 65 kilograms -Ex. Diener's scale of subjective well-being -Ex. IQ score, level of brain activity, and amount of salivary cortisol

2. interval scale

-measurement applies to the numerals of a quantitative variable that meet two conditions: -First, the numerals represent equal intervals (distances) between levels, and second, there is no "true zero" (a person can get a score of 0, but the 0 does not really mean "nothing"). -An IQ test is an interval scale

3. ratio scale

-measurement applies when the numerals of a quantitative variable have equal intervals and when the value of 0 truly means "none" or "nothing" of the variable being measured. -Ex. A researcher might measure how frequently people blink their eyes in a stressful situation; number of eyeblinks is a ratio scale because 0 would represent zero eyeblinks

3. Physiological Measure

-operationalizes a variable by recording biological data, such as brain activity, hormone levels, or heart rate -usually require the use of equipment to amplify, record, and analyze biological data -Ex. moment-to-moment happiness has been measured using facial electromyography (EMG)—a way of electronically recording tiny movements in the muscles in the face -Other constructs might be measured using a brain scanning technique called functional magnetic resonance imaging, or fMRI -A physiological way to operationalize stress might be to measure the amount of the hormone cortisol released in saliva because people under stress show higher levels of cortisol

1. self-report measure

-operationalizes a variable by recording people's answers to questions about themselves in a questionnaire or interview -Ex. Diener's five-item scale and the Ladder of Life question -Ex. asking people how much they appreciate their partner and asking about gender identity are both self-report measures -If stress was the variable being studied, researchers might ask people to self-report on the frequency of specific events they've experienced in the past year, such as marriage, divorce, or moving -In research on children, self-reports may be replaced with parent reports or teacher reports

3. likert scale

-people are presented with a statement and are asked to use a rating scale to indicate their degree of agreement. When such a scale contains more than one item and each response value is labeled with the specific terms *strongly agree, agree, neither agree nor disagree, disagree, and strongly disagree*

What are the advantages of self-report data/surveys?

-self-reports often provide the most meaningful information you can get -Ex. researchers who study dreaming can monitor brain activity to identify when someone is dreaming, but they need to use self-reports to find out the content of the person's dreams because only the dreamer experiences the dream

2. observational measure

-sometimes called a behavioral measure, operationalizes a variable by recording observable behaviors or physical traces of behaviors -Ex. a researcher could operationalize happiness by observing how many times a person smiles. -Ex. Intelligence tests can be considered observational measures, because the people who administer such tests in person are observing people's intelligent behaviors -may record physical traces of behavior -stress behaviors could be measured by counting the number of tooth marks left on a person's pencil, or a researcher could measure stressful events by consulting public legal records to document whether people have recently married, divorced, or moved

1. open-ended

-that allow respondents to answer any way they like. They might ask people to name the public figure they admire the most, or ask a sample of people to describe their views on immigration -Ex. Departing overnight guests might be asked to submit comments about their experience at a hotel. Their various responses to open-ended questions provide researchers with spontaneous, rich information.

how can you avoid a leading question?

-the survey writers might attempt to word every question as neutrally as possible -ask a simple question. When people can understand the question, they can give a clear, direct, and meaningful answer

what 2 levels do all variables have?

1. categorical & 2. quanitative

What are three potential problems related to the wording of survey questions?

1. leading question 2. double-barreled question 3. negatively worded question

what are 3 ways survey questions can be formatted?

1. open-ended 2. forced-choice question 3. likert scale

3 SCALES of measurement?

1. ordinal scale 2. interval scale 3. ratio scale

what are 3 TYPES of measures?

1. self-report measure 2. observational measure 3. physiological measure

what are 3 different things that people might do that weaken the construct validity of a survey?

1. social desirability bias/ faking good 2. response sets 3. acquiescence / yea-saying

3 forms of reliability?

1. test-retest reliability 2. interratter reliability 3. internal reliability

Dr. Easton conducted a study to determine if there is a relationship between open mindedness and political ideology. She created and administered a new test of open mindedness and found a strong correlation between the results from her test and the results from an established measure of open mindedness. What form of validity was Dr. Easton attempting to establish? a. Convergent validity b. Content validity c. Face validity d. Divergent validity

a. Convergent validity

Dr. X found an r of - 0.65 between scores on a depression scale and scores on a psychological well-being scale. This strong negative correlation indicates _______. a. Convergent validity b. Content validity c. Face validity d. Divergent validity

a. Convergent validity

A researcher determines a person's annual income based on the length of a person's arm-span. This test would most likely have ____ and ____. a. high reliability; low validity b. high reliability; high validity c. low reliability; low validity d. low reliability; high validity

a. high reliability; low validity

Dr. McCollum prepared a test for his study of computer knowledge in older adults. Before administering the test, he consulted with a group of computer science professors to make sure that the test contained enough information to assess all the major areas of computer knowledge. What type of validity was Dr. McCollum attempting to establish? a. Convergent validity b. Content validity c. Face validity d. Divergent validity

b. Content validity

Sarah looks through her math test and finds that there are vocabulary questions on it. The test may not be an accurate representation of learned material due to issues with ______ validity. a. Criterion b. Face c. Content d. Inter-rater

b. Face

At the gymnastics meet two of the three judges gave Natalie a score of 7.6 and the third judge gave her a score of 2. This is an issue with what type of reliability? a. Test-retest b. Interrater c. Internal

b. Interrater

A test can be _____________________________. a. Valid without being reliable b. Reliable without being valid c. Both are correct d. Neither is correct

b. Reliable without being valid

A scale or test is said to be _________ if it is measuring what it is intended to measure and ______ if you obtain consistent results every time you use it. a. reliable; valid b. valid; reliable

b. valid; reliable

Which of the following samples is most likely to generalize to its population of interest? a) a convenience sample of 12,000 b) a quota sample of 120 c) a cluster sample of 120 d) a self- selected sample of 120,000

c) a cluster sample of 120 (cluster: a group of similar things or people positioned or occurring closely together)

If a scale or measure has good test-retest reliability it....? a) means that a measure is valid b) will also have a good internal reliability c) means that you'll get similar results every time you use the test

c) means that you'll get similar results every time you use the test

In order to make sure that both observers are looking for and reporting the same thing, Dr. Mark will compare the number of times both observers agreed on their observations. Dr. Mark is measuring_____. a. Validity b. Test-retest reliability c. Internal reliability d. Interrater reliability

d. Interrater reliability

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) An individual's preference for an essay format, with the options of "strongly prefer", "prefer", "don't care one way or the other", "dislike", and "strongly dislike"

ordinal

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) Placement in the Peachtree Road Race (e.g., 1st, 2nd, etc.)

ordinal

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) Ranking of student's class performance

ordinal

Select which scale describes each of the following measurements: (interval, Nominal/categorical, ordinal, ratio) The local fast-food restaurant offers small, medium, and large soft drinks

ordinal

Identify as self-report/ observational/physiological: Brain activity

physiological

Identify as self-report/ observational/physiological: Hormone levels

physiological