Check Yourself Acitivities - Lessons 1-3

Determine whether different questions within the same IQ test correlate with each other.

Convergent validity study

Determine whether a new way of quickly diagnosing diabetes from one blood test agrees with the widely accepted fasting glucose tolerance test diagnostic protocols.

Gold-standard validity study

Determine whether a measure of Hispanic identity correlates with nation of birth (we expect this measure to be positively correlated with birth in Latin-American countries).

Test-retest reliability study

The following two statements are similar, but not identical. What is the implication of the added phrase, "on average", in the second statement? People who slept in the red room had higher sleep quality than people who slept in the blue room; People who slept in the red room had higher sleep quality, on average, than people who slept in the blue room. a. The first statement implies that there is no overlap, but the second allows for some red room sleepers to sleep longer than some blue room sleepers b. Both statements essentially mean the same thing, so the difference is not important c. Adding "on average" means that there was no variability in measured sleep quality

The first statement implies that there is no overlap, but the second allows for some red room sleepers to sleep longer than some blue room sleepers

Logic

What conclusions can we draw form these premises? ex. What does the free radical theory of aging predict would happen?

Epistemology

What should be our standards for believing something? ex. Can we accept expert opinion that this treatment works, or do we need to run a clinical trial?

Ethics

What's a morally good action here? ex. Now that we know the medicine works, should we give it to children whose parents cannot afford to pay for it?

Phenomenology

What's it like to experience this? ex. What were the authors thinking when they wrote this research article?

Ontology

What's the definition of this thing? ex. Should we consider male pattern baldness to be a disease?

empiricism

a body of facts that support an argument

assertion

a short story/example of a limited case

anecdote

a statement or a claim

evidence

a way of knowing based on sensory observation

Someone who holds very strong opinions on a topic. This person is unwilling to read scientific studies that have collected data directly addressing that topic. This person's opinions appear to be based more on ____ then ____. a. Dogma, empiricism b. Empiricism, dogma

a. Dogma, empiricism NOTE: we are greatly simplifying things to emphasize the distinction. In the messy real world, and there are systems of knowledge that are neither dogmatic nor empirical. And sometimes empirical folks can get pretty dogmatic.

Which type of study is the only one that can quantify systematic measurement error? a. Gold-standard validity b. Test-retest reliability c. Convergent validity d. Inter-item reliability

a. Gold-standard validity NOTE: Gold standard is the only one that tells you what is truth, and you need that to assess systematic error.

A monk says: "All sentient beings have Buddha consciousness, but it is sleeping. However, reading the Diamond Sutra awakens Buddha consciousness." A scientist is quite curious about this claim and would like to study it, but doubts whether the relevant variables can be measured to verify this statement empirically. Which is the biggest, most immediate barrier to measurement? a. He can't get a clear theoretical or operational definition of the sleeping-awake variation in Buddha consciousness (ontological challenges) b. He can't get access to any sentient being or a copy of the Diamond Sutra (inaccessible items)

a. He can't get a clear theoretical or operational definition of the sleeping-awake variation in Buddha consciousness (ontological challenges) NOTE: Yeah, spiritual topics usually suffer from definitions first, and often have accessibility issues second. We consider A the better answer here, because you have to know what the thing is before you can say for sure if it's accessible or not. Anyway, as the Diamond Sutra teaches, there is no Buddha consciousness. (That was a joke for all you religion scholars.)

Which type of logic is based on statistical probabilities (rather than decoding the implications of theories)? a. Inductive logic b. Deductive logic

a. Inductive logic

Imagine we rolled dice 10,000 times and then took the average of the score. Dice have 6 sides numbered 1 through 6, so the expected average is 3.5. The observed average is going to be: a. Probably a little higher or a little lower than 3.5, but it could be exactly right b. Probably a little higher or a little lower than 3.5, but never exactly right c. Probably a lot higher or a lot lower than 3.5, but it could be exactly right d. Probably a lot higher or a lot lower than 3.5, but never exactly right

a. Probably a little higher or a little lower than 3.5, but it could be exactly right

A researcher is interested in measuring skin color for studies on stress response to racism in daily life. The researcher wants to know whether a computer program that analyzes the level of brightness of face skin from Facebook profile photos is going to give inconsistent results. He downloads 2 pictures of each person and runs the program to see if the skin darkness score from picture 1 is the same as picture 2. It seems like this person is doing a study of: a. Reliability b. Validity c. Neither

a. Reliability NOTE: We run the same program twice on two different pictures of the same person. We're working on the assumption here that someone who has a darker complexion in one picture is still going to have a dark complexion in the next picture. But of course we're skeptical this will always work out because of lighting and tanning, and thus we propose doing a test-retest reliability study to see if the results are consistent. It would be a validity study if we were comparing Facebook pictures to some other measure of complexion that we thought had some truth to it, but instead we're just using same measure again to see self-consistency, which is reliability.

If everyone in the sampling frame, no matter their characteristics, all have the same probability of being invited to participate, the study design is: a. Simple random sampling (often just called "random sampling") b. Stratified random sampling (often just called "stratified sampling")

a. Simple random sampling (often just called "random sampling") NOTE: Equal probabilities, simple random. Different probabilities for different types of people, then stratified random sampling.

The measure of central tendency that is most vulnerable to extreme values is: a. The mean b. The median c. The mode

a. The mean Adding an outlier to a data set can be expected to distort the mean, but not the median or mode.

If a finding "generalizes" it means: a. What we learn from the sample can be applied to the larger population b. What we learn from the sample is only applicable to the sample population

a. What we learn from the sample can be applied to the larger population NOTE: A study is generalizable when the results can be applied to the larger population represented by the sample.

Putting a histogram on a log scale makes it ____ to see unusual/infrequent/rare responses. a. easier b. harder

a. easier NOTE: Log scales make little numbers more obvious, but compresses big numbers.

There's a statistically significant difference in birth weight of babies between those who had mothers who smoked meth during pregnancy and mothers who did not, therefore, smoking meth probably causes low birth weight. a. induction b. deduction

a. induction

Suppose you are an educational psychologist and are interested in understanding why some kids do better in school than others. You look at the grades of the students in a class that every student in the class got a 'D'. How much variability is there int he letter grades of this class? a. none b. a little c. a lot

a. none NOTE: If everyone got the same letter grade, then there's no variability in letter grades-- just a constant.

What's the log base 10 of a billion? A billion is 1,000,000,000. a. 10 b. 9 c. 3 d. 1

b. 9

If a study has both males and females in it, then the approach to sampling: a. Must be stratified sampling, because gender categories are strata b. Could be convenience, simple random, or stratified sampling

b. Could be convenience, simple random, or stratified sampling NOTE: It's only stratified sampling if we need the probability of recruitment to be different for men and women. Just because the study includes variables that could be stratified doesn't mean we actually are stratifying them. It will only be stratified in we say that we are oversampling a minority group or adjusting for probabilities to compensate for another sampling bias. Simply including people of different types is done in all sampling methods.

Which of the following is a nominal variable, not an ordinal variable? a. A measure of happiness: very happy, somewhat happy, neutral, somewhat unhappy, very unhappy b. Ethnic background c. T-shirt size

b. Ethnic background NOTE: Ethnic background is nominal-- each ethnicity is its own thing, not more or less of "ethnicness". However, the categories from happiness do have an order that expresses how much more or less happiness a person has, just like T-shirt size has an order to it too.

Sampling bias (a lack of representative sample) can sometimes be bad for both kinds of validity, but it is especially and most consistently bad for: a. Internal validity b. External validity

b. External validity NOTE: If I do a study on extroverted people, then it's fair to make an assumption that other extroverted people will be similar to the extroverts in my study. However, it doesn't make any sense to think that a study done on extroverts will necessarily apply to introverts. For this reason, we should be careful to not extrapolate from one study to population to another. in the case of sampling bias, if we tried to do a study representative of all humans, but we only collected data on American undergraduates, that bias in the sample really casts some doubts on whether your results will apply to everyone, doesn't it?! Restrictions on the ability to generalize are called external validity problems--we'll deal with internal validity later.

Sampling bias is different from the biases we speak about with gold-standard validity studies (Lesson 2) because: a. Gold standard validity deals with how well your selection of individuals represents the population of interest, while sampling bias deals with how accurately you can measure some variable for interest on the individuals in your study b. Gold standard validity deals with how accurately you can measure some variable of interest on the individuals in your study, while sampling bias deals with how well your selection of individuals represents the population of interest

b. Gold standard validity deals with how accurately you can measure some variable of interest on the individuals in your study, while sampling bias deals with how well your selection of individuals represents the population of interest NOTE: Gold standard validity was taught in a less called Measurement. It is designed to detect measurement bias, which concerns how well you measure a variable in each individual.If your measure matches the gold standard measure, your measurement validity is high and you don't have measurement bias. Sampling bias, on the other hand, is about who you choose for the study. If you choose a sample that represents the population, then you have a valid sample, and you dong have any sampling bias.

A researcher is studying the number of natural killer white blood cells in tonsil tissues from tonsillectomy patients. She wants to know if the counting method that she has invented (based on sectioning, straining, photography, and computerized cell counting) works the same way as a more expensive, widely accepted method that most other researchers are currently using (dissociation followed by magnetic-antibody sorting). She hopes that the results she gets with her method agree perfectly (exactly same number of NK cells) with the widely accepted method: a. Convergent validity b. Gold-standard validity c. Test-retest reliability d. Inter-observer reliability e. Inter-item reliability

b. Gold-standard validity NOTE: Whenever you hear us talking about comparing two different ways of measuring the construct (as long as those aren't two questions inside the exact same survey instrument), you know it's going to be one of the validity tests. Then when you hear us going on about how the other method is so excellent ("widely-accepted"), it's going to be gold standard.

A researcher is studying the number of natural killer white blood cells in tonsil tissues from tonsillectomy patients. She wants to know if the counting method that she has invented (based on sectioning, straining, photography, and computerized cell counting) works the same way as a more expensive, widely accepted method that most other researchers are currently using (dissociation followed by magnetic-antibody sorting). She hopes that the results she gets with her method agree perfectly (exactly same number of NK cells) with the widely accepted method: a. Convergent validity b. Gold-standard validity c. Test-retest reliability d. Inter-observer reliability e. Inter-item reliability

b. Gold-standard validity NOTE: Whenever you hear us talking about comparing two different ways of measuring the construct (as long as those aren't two questions inside the exact same survey instrument), you know it's going to be one of the validity tests. Then when you hear us going on about how the other method is so excellent ("widely-accepted"), it's going to be gold standard.

For doing experiments, stable personal characteristics are ____ to study as factors that influence exercise behavior outcomes than situational factors. a. Easier b. Harder c. Equally challenging

b. Harder

Who is responsible for making sure animal research at Penn State does not cause excessive suffering in animals? a. IRB b. IACUC c. Belmont Committee

b. IACUC

Which pair of terms is not synonymous with one another in a clinical trial? a. Dependent variable; Outcome b. Independent variable; Outcome c. Independent variable; Predictor d. Dependent variable; Health outcome of interest e. Independent variable; Manipulation

b. Independent variable; Outcome NOTE: IV and outcome never go hand in hand. All the other pairs of terms can be synonymous.

Control condition contains subjects that are different from the experimental condition subjects in terms of: a. Demographic variables b. Manipulated variables

b. Manipulated variables Control groups and experimental treatment groups differ in terms of whatever the researcher is manipulating. We hope that both groups are pretty much the same in terms of everything else, including age, gender, etc.

Imagine a histogram is showing the distribution of IQ scores from a large sample of students. What does the height of the bars represent on the graph? (Hint: what would be an appropriate label for the Y-axis?) a. IQ (how high is the IQ score) b. Number of people (how many had a particular IQ score) c. Standard of deviation (how high is the SD in the given interval)

b. Number of people (how many had a particular IQ score) NOTE: When showing a histogram, the variable units will be on the X-axis and the frequency will be on the Y-axis.

As researchers, we are interested in psychosocial factors in heart disease. Specifically, we are interested in the effect of optimism about a grandchild's future on blood pressure. We sample from a nearby hospital serving an economically disadvantaged neighborhood. In our study, we do find a link between this optimism and blood pressure. What conclusions might we be able to draw from this study? a. Optimism about a grandchild's future effects blood pressure for everyone b. Optimism about a grandchild's future effects blood pressure for those living in low SES communities

b. Optimism about a grandchild's future effects blood pressure for those living in low SES communities NOTE: It could be that if we sampled from a hospital serving an affluent community we could have gotten different results. Perhaps the nature of optimism in the two communities is different? If we did not think about possible differences in populations, we wouldn't know that the relationship of optimism to blood pressure was different than if we had happened to study a different group.

When the CDC wants to see how common a disease is in the whole USA, they usually make sure of a large nationally representative study like the National Health Interview Survey or the National Health and Nutrition Examination Survey. These studies were originally designed to recruit people in the same proportion as society at large. For example, if 72.4% of Americans were white, they would try to recruit 72.4% white people for their sample, and be similarly consistent for all the minority groups. Every ethnic group got recruited in the same proportions as the population. a. Switched from doing stratified sampling to doing simple random sampling b. Switched from doing simple random sampling to doing stratified sampling

b. Switched from doing simple random sampling to doing stratified sampling NOTE: In this simplified scenario (the real history of sampling for NHANES is more complicated than explained above), in the beginning they tried to get a balanced sample for everyone, which is what a simple random sample is designed for. Later, they try to over-sample minorities, and for this oversampling ( higher probability of recruitment if yo're in a minority group), they need to do stratified sampling.

Medical research programs tend to start out with simpler and smaller studies and then progress up to doing large and more proper clinical trials. The early studies are just testing if the intervention works at all (for anyone), the later studies are testing if the promising interventions actually work consistently in the real world (for most everyone). In other words, the basic studies at the beginning are investigating if the intervention has any effect, while the applied studies at the end are investigating if the findings translate to the population at large. a. The early studies are concerned with external validity, which focuses on causality; the first priority in what is called basic research. The latter studies are concerned with internal validity, focusing on generalizability, which is the priority for applied research b. The early studies are concerned with internal validity, which focuses on causality; the first priority in what is called basic research. The latter studies are concerned with external validity, focusing on generalizability, which is the priority for applied research

b. The early studies are concerned with internal validity, which focuses on causality; the first priority in what is called basic research. The latter studies are concerned with external validity, focusing on generalizability, which is the priority for applied research NOTE: Once again, external validity concerns the issue of generalizability, and this is of the greatest concern for applied research. Internal validity is more focused on determining if observed relationships inside a study are truly causal, and these are of utmost concern in basic research (but obviously matters a lot for applied research too).

If a mother's meth smoking causes low birth weight in the offspring, then we would expect to see a statistically significant difference in birth weight or babies between those who had mothers who smoked meth during pregnancy and mothers who did not. a. induction b. deduction

b. deduction

For variables with normal distributions, about 5% of observations are more than ___ standard deviation(s) away from the mean: a. 0 b. 1 c. 2 d. 3 e. It depends on the sample size

c. 2 NOTE: We want you to remember that 95% of observations are within 2 stdev of the mean in a normal distribution. 100% - 95% = 5%. Thus if 95% are within 2 stdev, then the remaining percent are more than 2 stdev.

Which of the following distributions would be characterized as a nominal categorical variable? a. Score on a measure of narcissism b. GPA c. Left or right handedness d. Number of siblings

c. Left or right handedness NOTE: GPA and narcissism are scores, which implies that they are quantitative variables. Number of siblings is a number, so that's obviously quantitative too. Left vs Right handedness is two equal categories, and there's no obviously implied order here, so we'll place it as nominal. Maybe if it were 3 categories with left ambidextrous, and right then it would make more sense to call it ordinal, but just because one nominal response is more common than the other (more righties than lefties) doesn't mean we should call it ordinal. I usually only consider variables as ordinal if the options are yes/no, high/low, fast/slow or some other way like that directly implies more or less of the construct of interest.

The median and the mean will be highly similar when: a. The sample size is large enough b. Categorical data are used c. The distribution is symmetric

c. The distribution is symmetric NOTE: If the data set is not symmetric, adding a larger sample will not make the mean and median similar. When data is categorical instead of numerical, there is no mean (you need numbers to calculate sums), so that answer doesn't make sense.

What is a sampling frame? a. A mathematical process for selecting people at random b. A theoretical definition of who you want to study c. The group of people who actually complete the whole study d. A list of potentially eligible people that you could recruit for your study

d. A list of potentially eligible people that you could recruit for your study NOTE: Look at the multiple choice options again. Can you see which answer is a population, which answer is the randomization method, which answer is called the final sample? Populations are theoretical, while the other issues (sampling frames, sampling methods, recruitment, enrollment, and data collection on the final sample) are all very practical/operational. It's kind of like the distinction we made in the measurement section between a construct and a measure-- the construct is the theoretical definition of the variable, while the measurement instrument/process/protocol is the operational definition of the variable. Here in the methods of finding research subjects, the population is the theoretical definition, and the whole sampling process (from frame to enrollment) is the operational definition of who you study.

Let us say we are interested in seeing if the color of one's bedroom affects sleep quality. You hypothesize that a very strong, bright colors (e.g., red) will have a greater disturbing effect on sleep quality that more "restful" colors (e.g., blue). you decide to measure "sleep quality" by the number of minutes slept. Your subjects made a careful note as to what time they fell asleep and what time they woke up. And sure enough, the "red" group slept an average of 7 hours, while the "blue" group slept an average of 7.5 hours; a difference that is large enough to be significant. So, room color seems to make a difference. In this scenario, which of the following is a reasonable conclusion? a. Room color explains some of the variability in sleep duration b. Room color covaries with sleep duration c. Room color is associated with sleep duration d. All of the above

d. All of the above NOTE: All of these statements are basically synonymous. Later in the course we will deal with whether we should interpret a casual relationship between the variables (which none of these statements imply).

What are the key differences between ratio and interval scales? a. Interval scales have a tune zero and work properly for addition/subtraction, but ratio scales do not Ratio scales have a true zero and work properly in multiplication/division, but interval scales do not b. Interval scales have true zero and work properly in multiplication/division but ratio scales do not c. Ratio scales have a true zero and work properly for addition/subtraction, but interval scales do not d. Ratio scales have a true zero and work properly in multiplication/division, but interval scales do not

d. Ratio scales have a true zero and work properly in multiplication/division, but interval scales do not NOTE: Interval scales might have a zero in them, but it's not a true zero because that zero doesn't represent some kind of nothingness. And when we do multiplication and division, we're usually trying to express how many times more or times lower the data is-- and that is all framed in reference to distance from a meaningful zero. Addition and subtraction also get weird on logarithmic scales, but they're normal on interval and ratio scales.

Many people debate the causal relationship between gun ownership rates and murder rates in society. Jim argues that when more people own guns, more people kill people, because tools of any kind are more likely to be put to use the more they are available. Jack argues that when more people own guns, fewer people kill people, because attackers are deterred by armed resistance. Both are arguing for gun ownership rates directly influencing murder rates. Jane, on the other hand, believes that social inequality influences both murder rates and gun ownership rates, and that both of the causal paths that Jim and Jack believe in are false. Jane is arguing for: a. Regular causality b. Reverse causality c. Bidirectional causality d. Third variable causality

d. Third variable causality

Empiricism means a. To be familiar with theories about the thing in which you are interested b. To draw a conclusion based on what you were told by your teacher c. To draw a conclusion based on what you read in your textbook d. To look and see based on scientifically conducted test

d. To look and see based on scientifically conducted test Being empirical means doing the test-- it's not about the theories or the received wisdom, but instead, empiricism emphasizes data and observations. An empiricist will only trust theories, teachers, and textbooks if those sources are basing their claims on observations, where scientists really went to "look and see" if their claims were supported by that kind of evidence.

Determine whether repeated measurements on the same behavioral assay agree when applied twice.

inter-observer agreement