Exam 1 (Research Methods & Statistics I)

Bar Graph

- Bars DO NOT touch. > Used in discrete variables

Continuous Variable

- Fractions of units are possible. > Ex. Height in inches (68.2 inches is possible)

What does SS stand for?

- Sum of Squared Deviations

Define Cronbach's Alpha.

- The most common measure of internal consistency, used by researchers in psychology. > Conceptually, alpha is the mean of all possible split-half correlations for a set of items. *A value of +.80 or greater is generally considered good internal consistency.

Sample Statistics

- The value obtained from the sample. - Used to estimate population parameters.

The Standard Normal Curve and z Scores

68-95-99 rule

Population

A group of all things that share a set of characteristics.

Sample

A subset of the population intended to represent the population.

What does df stand for?

Degrees of Freedom

Independent Variable Levels

Different values for the independent variable.

Samples are less variable than populations. True False

True

Which of the following refers to the extent to which the measurement accurately predicts behavior related to that measurement? a. Predictive validity b. Concurrent validity c. Content validity d. Convergent validity

a. Predictive validity

Using sample statistics to make conjectures about population parameters is _______. a. unethical b. called inferential statistics c. not advised d. called descriptive statistics

b. called inferential statistics

The height of the horizontal line inside a boxplot indicates the _______. a. mode b. median c. mean d. lowest score

b. median

Histograms

- Bars DO touch. > Used on continuous variables.

What is the difference between a Platykurtic & Leptokurtic distribution?

- Platykurtic - FLAT > The distribution is flatter than normal. >> Has a relatively LOW probability of extreme events. - Leptokurtic - LEAP / TALL > The distribution LEAPS-UP in the middle, relative to a normal curve. >> Has a relatively HIGH probability of extreme events.

Types of Validity - Construct Validity

- The degree to which inferences can legitimately be made from the operationalizations in your study, to the theoretical constructs on which those operationalizations were based. > Example(s): The degree to which your "survey" of self-esteem which consists of 10 questions, actually represents the type of information that is relevant to capturing a person's self-esteem (the construct you can theorize exists, but cannot directly be observed).

Quadrupling the sample size would cut sampling error by _______.

50%

Computing Standard Deviation

Standard Deviation - The typical difference between all the scores and the mean

Greek letters are used to symbolize population data. True False

True

Whether your data is discrete or continuous determines how the data are graphed. True False

True

M represents the _______. a. sample mean b. sum of the scores c. population mean d. deviation score

a. sample mean

According to the reading, a correlation between your scale's items that is greater than _______ indicates good reliability. a. +.70 b. +.80 c. +.60 d. +.50

b. +.80

Computing the standard deviation consists of _______ steps. a. 6 b. 5 c. 4 d. 3

b. 5

What is the appropriate description of the distribution of this graph? a. leptokurtic b. positively skewed c. normal d. negatively skewed

b. positively skewed

Distribution

- A group of scores. > Can be a list, table, or graph. - Normal Distribution - shaped like a normal bell curve; symmetrical - Positively Skewed - tail points to the RIGHT on the x-axis - Negatively Skewed - tail points to the LEFT on the x-axis

Confidence Interval (CI)

- An estimated range of plausible values for a population parameter. - Determine the Point Estimate - Compute the 95% Margin of Error (MOE) - Compute the Upper Bound (UB) and Lower Bound (LB).

✅Threats to Internal Validity - History

- An event occurs between repeated measures of the DV or between the IV and DV that affects scores on the DV. History effects mean that the DV will reflect the manipulation of the IV and the influence of the outside event. > This is particularly problematic for within-participant designs (aka repeated measures), in which a pre-test and post-test is used. > This may also be problematic if something occurs between the manipulation of the IV and measure of the DV. > Example: Suppose you were interested in the effect of massage on academic performance. Participants show up at different locations in a large building. You give some people a 10-minute massage and ask others to wait patiently for 10 minutes (IV). Then, everyone walks to another room in the building to take a test. >> In the time between the manipulation of the IV and the test (DV), there happened to be a gathering of circus clowns practicing juggling throughout the building. Participants test performance in this case would reflect both the effect of circus clowns and the IV.

✅Threats to Internal Validity - Experimenter Bias

- An experimenter's expectations and / or behaviors influence participants. > Different scores on the DV by condition may be due to the manipulation of the IV or due to the experimenter bias. > Notes, this would be less of an issue if the experimenter's behavior was consistent across conditions. - Example: Suppose an attractive research assistant is involved in a gambling study, and it is her responsibility to ask participants if they are interested in gambling. If she asks male participants in 1 way, but asks female participants in a different voice or tone, the differences between male and female participants' gambling behavior might be due to experimenter bias, and not due to the manipulation of the IV.

Types of Validity - Divergent Validity *Used to demonstrate that your measurement has Construct Validity.

- Demonstrates that we are measuring 1 specific construct and not combining 2 different constructs in the same measurement process. > Examples(s): You must demonstrate that the construct "aggression", and the construct "activity level" are separate and distinct. - Concern(s): How do you know that your measure of aggression is measuring "aggression" as opposed to something else that could also produce similar scores on the measurement (such as activity level)? Maybe very active children appear to be more aggressive than less active children.

Types of Validity - Concurrent Validity *Used to demonstrate that your measurement has Construct Validity.

- Determined by whether or not scores on a measurement are directly related to scores on a well-established measurement of the same variable (consistency between 2 measurements of the same variable). > Example(s): "Dr. Emer's depression measure" should result in similar scores as "Beck's Depression Inventory". - Concern(s): Measure of 2 different constructs could be highly correlated, but they obviously measure different things. > Another example: Taller people tend to weight more, but stepping on a scale to measure heigh doesn't make sense, yet the 2 scores would be correlated.

✅Types of Validity - Predictive Validity *Used to demonstrate that your measurement has Construct Validity.

- Determined by whether or not scores on the measurement of a construct accurately predict behavior related to that construct. (This is important in the clinical world - e.g., lethality assessment, recidivism for sex offenders). > Example(s): A person who scores high on a measure of aggression should be aggressive in the circumstances predicted by the score. - Concern(s): Can be controversial - e.g., predictive validity of SAT scores regarding college success - some colleges have removed the SAT from the required application process for this reason.

Threats to Internal Validity - Regression to The Mean

- Extreme scores on a measure tend to get close to the mean during further administrations. - Example: If a pre/post-test measure is administered and an IV is manipulated in the interim, then the difference (particularly if there are extreme scores during time-1) could be due to the IV or regression to the mean effects.

Discrete Variable

- Measured in WHOLE UNITS ONLY. > Fractions are not possible. > Ex. Number of siblings (2.5 is not possible)

What is the difference between a 1-tail test & a 2-tail test?

- One-Tail test is done when you are testing specific direction (greater than, smaller than) - Two-Tail tests are the standard but theoretically done when you are testing for differences but not sure which direction you would see the effects.

✅Threats to Internal Validity - Mortality (Attrition)

- People may drop out of a study at some point before the final DV is measured. > If the people who drop out share some individual difference variable (they are all lazy, they are all intelligent), then differences in your DV may be due to the manipulations of the IV or due to disproportionate number of people left in the experiment that share some similarity (the remaining people are all active or dull). - Example: Imagine that I am interested in the benefits of a new exercise program. Over the course of the experiment, many people drop out of the new exercise program condition. At the end of the experiment, I weigh everyone and find that those in the new program lost more weight than those in the old program. The additional weight loss could be due to the new program, or it could be because all the people left in that condition are persistent (or active, or whatever).

Define Standard Error of the Mean (SEM).

- Represents the typical (or standard) difference a sample mean, of a given size N, will deviate from the population mean. > It measures the standard (or typical) amount of sampling error expected in a given study. *Use the SEM(p) whenever you can because it will provide a more accurate estimate of sampling error. - As the standard deviation increases, the SEM also increases. > As the sample size (N) increases, the SEM decreases. > In some cases, you could decrease the standard deviation of scores, by improving the measurement procedures used in a study.

✅Threats to Internal Validity - Maturation

- Some change that occurs over the course of an experiment due to time. (It is important that if time is an issue in an experiment, the passage of time should remain constant across different levels or conditions of the IV). > Example: In a longitudinal study, participants will change as a result of biological growth or experience. The dependent measure will reflect differences caused by the IV and differences that were caused by biological growth or experiences outside of the study environment. > Example: Participants might experience fatigue or boredom over the course of a procedure. (This is particularly important if your DV might be influenced by fatigue, boredom, or biological change.)

Central Limit Theorem (CLT)

- The distribution of sample means will always be equal to the population mean. - The distribution of sample means' standard deviation will always be equal to the standard error of the mean (population standard deviation divided by the square root of the sample size). - The distribution of sample means will tend to be normal as N increases; in practice, if N > 30, it will be normal unless the original population is very skewed.

Type of Validity - Content Validity *Used to demonstrate that your measurement has Construct Validity.

- The extent to which a test represents the universe of items from which it is drawn. (This is important for tests of particular types of knowledge). > Example(s): Consider a test in a class that you feel does not represent well what was covered in class, or a measure that assesses sensation seeking but only asks about outdoor activities.

✅Types of Validity - Face Validity *Used to demonstrate that your measurement has Construct Validity.

- The extent to which it appears that this technique measures what it claims to measure. > Example(s): For measuring self-esteem - "I think I am as good as other people" (high face validity) vs. "I get angry when people do not tell me the truth". (low face validity) - Concern(s): Face validity is subjective, and can be a problem because it cues participants to what the study is about and may cause them to change their behavior (self-serving bias, social desirability bias).

Types of Validity - Internal Validity

- The extent to which you can draw causal inferences between your variables. - Concern(s): There are 9 threats to internal validity.

✅Types of Validity - External Validity

- The extent to which you can generalize your study's results and conclusions to other situations, people, stimuli, and time. > Example(s): Your results' findings about how TAMUCC freshmen use visual search to find items can be generalized to other populations (i.e., other universities' students, people in other states, other countries, and so on) because it is unlikely that there will be perceptual differences in how this sample searches for items compared to other samples. >> The findings might not generalize to other populations like children or older adults who are typically not TAMUCC students, because there might be important differences in perceptual ability for these populations.

Distribution of Sample Means

- The population of all possible random sample means for a study conducted with a given sample size. > Example: In other words, if a study was planned to have a sample of 25 people drawn from a large population of 100,000, the distribution of sample means for this study would include the mean from every possible combination of 25 people from the population. - When you gather sample data, you are conceptually taking one of these samples from this distribution of sample means. > So, it would be helpful to know what this distribution of sample means looks like so you can estimate your study's expected sampling error.

✅Threats to Internal Validity - Selection

- The procedure used to select participants from a sample, bias the sample with regard to a specific, individual difference variable. > Example: Let's suppose you sample from an all-girls school, but you wish to generalize your results to all children. In this case, the fact that you sampled no boys might be problematic. > Example: Imagine that due to the way you sampled participants, people in one condition of your experiment differ from people in the other condition (in terms of race, gender, political orientation, or whatever). >> In this case, the dependent variable would reflect the manipulation of the IV and individual differences between your participants.

Threats to Internal Validity - Selection-Maturation Interaction

- This occurs when selection / sampling is not random and natural maturation occurs during the course of an experiment. > Thus, the experimental groups would need to differ in terms of a subject variable, and natural maturation would have to happen during the experiment.

Describe z for a single score.

- Used to locate a score, in a distribution of scores, and compare scores from different distributions. > Tells you if a score is above or below a population mean. >Informs whether a score's deviation from the mean is relatively large or relatively small compared with the deviations of the rest of the data. You use the mean and the standard deviation of the distribution to help you interpret an individual score.

Types of Validity - Convergent Validity *Used to demonstrate that your measurement has Construct Validity.

- Using 2 different methods to measure the same construct, and showing that there is a strong relationship between the scores obtained from the 2 methods. > Examples(s): Aggression in kids; measure 2 ways: observe free play behaviors in a playground, record the incidence of behaviors (hitting, yelling, etc.); also, ask for teacher ratings of aggression in each child; if teacher ratings and free play observations correlate well, you have convergent validity.

Threats to Internal Validity - Instrumentation

- When a change occurs in the manipulations of an IV or in the measure of a DV during the course of an experiment. > If differences are observed in the DV, it could be due to the IV or to changes related to the IV or DV. - Example: There are a number of examples of changes through the course of an experiment. > Observers may become more sophisticated and careful in their observations. > Researchers may tweak and refine their procedures for manipulation. > Equipment might malfunction. > Technology advances or declines during measurement or manipulation. *Generally, you can lump instrumentation issues into 3 categories: > Human Error > Reliability or Validity Issues (that change during the experiment) > Technology Issues (that change during the course of the experiment).

✅Threats to Internal Validity - Repeated Testing

- When the DV is measured more than 1 time, it is possible that scores on the final measure are affected by the prior administration of the measure, scale, or test. If the scores on the DV change as a result of manipulation of the IV and because of the previous administrations of the DV, then this is a problem for internal validity. > Note that this is particularly problematic with pre/post-test and within-subjects designs, and when only 1 group is used (i.e., the IV is manipulated in between administrations of the DV). > Example: Suppose I am interested in the effects of exercise on problem solving. I ask people to solve a few math problems, then I ask them to exercise, and finally I administer the same few math problems. When people score better on the post-test, I conclude that is due to the exercise, when in reality, it could just as easily be due to practice.

Computing a Confidence Interval with t Distribution

- When you don't now the population standard deviation, you use: > SEM(s) = SD / square root of N > Use tCI instead of zCI - When computing the Margin of Error (MOE): > The tCI changes based on sample size (unlike with a z score); look up in Appendix B, two-tailed a = .05, df = N - 1

How do you compute the 95% Confidence Interval with t Distribution? > tCI *It is not 1.96 like when working with a z Distribution, where it is always 1.96. > It is always the same, because the z Distribution DOES NOT change its shape based on sample size like the t Distribution does.

- df = N-1 - Look up the df in the two-tailed probabilities t table, in the .05 column. - Use this number for tCI.

Independent Variable (IV)

A variable with 2 or more levels that are expected to have different impacts on another variable.

Although learning to use statistical software can be helpful, it is always best to compute data by hand. True False

False

Obtaining a representative sample that minimizes sampling error is minimally important True False

False

The best scientific conclusions convey certainty even when the results are uncertain. True False

False

Define Split-Half Correlation.

Involves splitting the items into 2 sets, such as the 1st & 2nd halves of the items or the even= and odd-numbered items. *A split-half correlation of +.80 or greater is generally considered good internal consistency.

Dependent Variable (DV)

Is the outcome variable that is used to compare the effects of the different IV levels.

Comparing measures of Central Tendency

Mode - Most common scores Median - Middle score (after placing numbers in ascending order) Mean - Point that balances deviation scores or the point that balanced the deviation / variability of all the scores *The mean, median, & mode will all be the same when you have a perfectly symmetrical curve (normal distribution).

Choosing the correct measure of Central Tendency:

Nominal - Mode Ordinal - Median Interval or Ration - Mean *If data is highly skewed use MEDIAN *If there are extreme outliers use MEDIA

Comparing measures of Variability

Range - Distance between minimum & maximum score > very INSENSITIVE Standard Deviation - The typical difference between all the scores and the mean > very SENSITIVE

Define reliability.

Refers to the consistency of a measure.

What is Test-Retest CORRELATION?

Requires using the measure on a group of people at 1 time, using it again on the same group of people at a later time, and then looking at the correlation between the 2 sets of scores. *In general, a test-retest correlation of +.80 or greater indicates good reliability.

What is Test-Retest Reliability?

Says that scores should be consistent across times.

Define Internal Consistency.

The consistency of people's responses across the items on a multiple-item measure.

Define Interrater Reliability.

The extent to which different observers are consistent in their judgments. > Different observers' ratings should be highly correlated with each other. - Interrater reliability is often assessed using Cronbach's alpha when the judgments are quantitative or an analogous statistic called Cohen's K when they are categorical.

Scale of Measurement

The precision with which a variable is measured. - Nominal (least precise) > simply categorizes >> Categorize things into groups that are qualitatively different from other groups. > COUNT >> Only allows you to count how many things are in each group. - Ordinal > categorizes in rank order >> Does not QUANTIFY how much more! > COUNT & RANK - Interval > quantifies how much of something people have > COUNT, RANK, & QUANTIFY - Ratio (most precise) > also quantifies how much something people have, but a score of 0 on a ratio scale indicates that the person has none of the thing being measured. >> There is a possibility of 0 or none. > COUNT, RANK, & QUANTIFY

Population Parameter

The value that would be obtained if the entire population were studied.

MEAN - Point that balances deviation scores or the point that balanced the deviation / variability of all the scores (visual)

This will always happen.

Because you want as little sampling error as possible, the standard error of the mean should be small. True False

True

Sample means with large amounts of sampling error are rare. True False

True

The primary reason for learning statistics for students in varying fields of study is because decisions made have the potential to improve lives and therefore should be informed by data. True False

True

Descriptive Statistics

When intent is to summarize only collected data.

Inferential Statistics

When using a sample statistic to infer the population parameter.

When do you use t instead of z, to find the probability of a given sample mean?

When you do not know the population standard deviation.

Frequency Tables

X = score > The type of response the participants could give. f = frequency > The # of people who gave that response % = percentage of sample with that score %tile = percentage of sample with that score or lower

A threat to internal validity could be due to the experimenter changing some of the procedure during the course of the experiment. In other words, the threat is referred to as _______. a. Instrumentation b. Experimenter bias c. History d. Maturation

a. Instrumentation

If you are interested to know whether your scale is consistent across time, which component would you want to analyze? a. Test-retest reliability b. Inter-rater reliability c. Internal consistency d. Validity

a. Test-retest reliability

Choosing a random sample that has the same mean as the population is _______. a. common b. risky c. unlikely d. optional

a. common

Sampling error is created when a _______. a. sample does not represent the population of interest very well b. study does not have a well-defined independent variable c. study does not have a well-defined dependent variable d. sample is too large

a. sample does not represent the population of interest very well

Researchers use samples to estimate population parameters because it is rarely feasible to obtain data from an entire population. Sample statistics estimate population parameters. The discrepancy between sample statistics and population parameters is called _______. a. sampling error b. Type II error c. statistical power d. Type I error

a. sampling error

In the context of the measures of reliability, correlating the total score of the first 20 questions on a test with the total score of the last 20 questions on a test is an example of _______. a. split-half reliability b. Cohen's K c. Cronbach's alpha d. test-retest reliability

a. split-half reliability

If you were to give your participants the same exact IQ test to complete multiple times in the study, and you see an improvement in their scores, it is possible that it is happening not due to your IV, but rather because of ______. a. Regression to the mean b. Repeated testing c. Maturation d. Instrumentation

b. Repeated testing

If a measure is reliable, then a researcher must find a _______, a. correlation coefficient of zero b. high positive correlation coefficient between scores on the measure. c. negative linear relationship between the two variables being studied d. high negative correlation coefficient between scores on the measure

b. high positive correlation coefficient between scores on the measure.

When sample characteristics differ population characteristics, _______ occurs. a. a skewed distribution b. sampling error c. a normal distribution d. data analysis error

b. sampling error

Wide confidence intervals are common with _______. a. large sample sizes b. small sample sizes c. sampling error d. sample sizes of 30 or larger

b. small sample sizes

You determine the lower boundary (LB) of the confidence interval by _______ the margin or error _______ the point estimate. a. dividing; by b. subtracting; from c. multiplying; by d. adding; to

b. subtracting; from

Find the mode for this sample of test scores. a. 80 b. 60 c. 90 d. 70

c. 90

You decided to measure people's sexual attraction by having them complete a self-report measure, as well as monitoring their heart rate when they view various photos of people. If there is a strong positive relationship between participants' self-report scores and their heart rates, it is likely that your measure has strong _______. a. Concurrent validity b. Content validity c. Convergent validity d. Divergent validity

c. Convergent validity

If you are interested to know whether there is consistency of people's responses across items on a multiple item measure, which component would you want to analyze? a. Inter-rater reliability b. Test-retest reliability c. Internal consistency d. Validity

c. Internal consistency

The mean of the distribution of sample means is equal to _______. a. standard deviation of the sample b. standard error of the mean c. mean of the population d. mean of the sample

c. mean of the population

To accurately interpret statistical results requires careful consideration of the _______ used to generate those results. a. rationale b. subjects c. methodology d. researcher

c. methodology

Suppose the researchers told you that they need to have a standard error that is smaller. They could increase _______ to make it smaller. a. variability of the population b. size of the treatment effect c. sample size d. population size

c. sample size

What potential problem is created when researchers use samples of participants rather than entire populations in their research studies? a. Type I error b. statistical power c. sampling error d. Type II error

c. sampling error

The _______ of the distribution should help in the decision of which measure of central tendency to use. a. accuracy b. direction c. shape d. size

c. shape

∑X represents _______. a. population mean b. sample mean c. sum of the scores d. deviation score

c. sum of the scores

Which of the following refers to the extent to which the measurement or manipulation of a variable accurately represents the theoretical variable studied? a. content validity b. face validity c. predictive validity d. construct validity

d. construct validity

To see all percentile values in a set of data, you need to look at _______. a. bar graphs b. line graphs c. boxplots d. frequency distribution tables

d. frequency distribution tables

The variable below is people's response to, "What is your religion?" What measure(s) of central tendency should you use? a. mean b. mean or media c. median or mode d. mode

d. mode

The first pillar of scientific reasoning is hypothesis testing with a continuous p value, is also known as _______. a. effect size b. population error c. sampling error d. null hypothesis testing

d. null hypothesis testing

p Value

p stands for probability - The probability of a given result (e.g., z score) with certain assumptions