Research Methods in Pyschology Exam 2

Physiological measures

Measuring internal processed that are not really visible

Inter-Item (internal consistency)

To what degree do all items (questions) on your scale assess the same construct? Assess using Cronbach's alpha coefficient

How to prevent observer bias and effects

Train observers well, develop clear rating instructions, use multiple observers to estimate reliability, and masked designs (a.k.a. blind designs)

Observer effects (reactivity)

When observers' presence inadvertently change the behavior of those who they are observing (e.x. the horse responding to its owner's nonverbal cues but not responding to other people asking the same questions)

Observer bias

When observers' expectations influence their interpretation of participants' behaviors (e.x. Browns fans and Steelers fans interpreting the same game differently)

Observer effects (a.k.a. expectancy effects)

When observers' presence inadvertently change the behavior of those who they are observing. This equals reactivity

Observational research

When researchers observe subjects and systematically record what they are doing, or what they did in the past using some type of documentation

Coercion to participate

When someone in authority applies pressure or attempts to influence individuals into participating. Individuals believe that not participating will have negative consequences. Offering exceptionally high or excessive rewards for participating.

What must an informed consent form include?

"Why is the researcher doing this study?" ,"what will the participant be doing?", "are there any risks involved?", consequences of refusal/withdrawal, confidentiality assurance, "does the participant have any questions?", "how does the participant contact the researcher afterwards?", and signature lines.

Cronbach's alpha coefficient

(#of items * covariance of all items)/(average variance of each item + (# of items-1) * average covariance of all items)

Validity

(Think accuracy)

How to create operational variables

1. Define the conceptual variable (e.x. stigma) 2. Create an operational definition (i.e., decide how you are going to measure the construct) 3. Evaluate how consistently and accurately the operationalization measures the construct we're interested in

Two reasons people will score differently when we measure behavior

1. They actually differ in their behavior 2. Measurement error

What you should ask when questioning construct validity

Are the researchers really measuring what they think they are measuring? How did the researchers measure/manipulate the variable(s) of interest? Does the measure capture all of the behavior/variable of interest or just part of it? How well did the researchers operationally define a concept or hypothetical construct? Did the researchers check to see if they really manipulated their variable the way they believe?

Ways to ensure confidentiality

Anonymous responding, coded data, and obscuring details of the particular case

Which of these is NOT possible? A. A measure is reliable and valid B. A measure is reliable and invalid C. A measure is unreliable and valid D. A measure is unreliable and invalid

C. A measure is unreliable and valid

Reactivity

Change in behavior when study participants know they are being observed

The goals of debriefing

Clarifies the nature of the study and why deception was used, allows participant to express his/her feelings about being deceived, remove negative consequences that the study induced, allows participants to reveal other reactions/comments about study, and allows researchers to convey their appreciation for individual's participation

The Belmont Report

Commission of many people (physicians, ethicists, scientists, philosophers, etc.) called together by Congress in 1976 to outline basic ethical principles researchers should follow when using human subjects

Two types of criterion-related validity

Concurrent validity and predictive validity

Test-Retest reliability

Consistency of responses on the same measure over time. People get consistent scores every time they take the test.

What if it is impossible to observe behavior in a "real world" setting?

Contrived observation and field studies

Objections to deception

Deontological objection, development of mistrust, and they may enter future studies already suspicious

Nominal scale

Describes categories. Numbers serve as labels only. No numerical meaning. No rank order of data. Cannot perform mathematical operations with nominal scale measurements. E.X.: man=0, woman=1, and no-gender=2. Yes=1 and no=2. Not depressed=0, mild depression=1, moderate depression=2, and severe depression=3

Discriminant validity

Does our measure only weakly relate (or have no relationship) to other measures that have little/nothing to do with our construct? Looking for weak correlations. Your self-report measure is less strongly associated with self-report measures of dissimilar constructs. E.X.: depression has a weak or zero correlation to shyness, intelligence, and boredom

Convergent validity

Does our measure positively correlate with other measures that are assessing similar constructs?/Does our measure negatively correlate with other measures that are assessing opposite constructs? Looking for moderate-strong correlations! Your self-report measure is strongly associated with self-report measures of similar constructs. E.X.: depression has a strong positive correlation with depression, pessimism, and social withdrawal. Depression has a strong negative correlation with happiness, positive adjustment, and life satisfaction

Criterion-Related validity (think behaviors)

Does the measure allow you to distinguish people on the basis of a particular criterion (behavioral outcome)? Can a self-report measure help predict behavioral outcomes that researchers/clinicians are interested in? It is very important to pick behaviors that are truly related to our construct. Your measure is correlated with a relevant behavioral outcome. E.X.: If I want to study people's stigmas towards a particular group, if my behavioral outcome was how often they sing karaoke then it is not related. Discriminatory behaviors towards that group would be a better behavioral outcome. Depression having a strong correlation to crying, social behaviors, change in eating habits, time spent in bed, and thoughts of suicide

Face validity

Does the measure appear to measure what its supposed to measure? Face validity does NOT equal actual validity but high face validity gives people higher confidence in the results and lack of face validity does not mean the measure is not valid. Sometimes tests lack face validity on purpose. It looks like what you want to measure.

Content validity

Does the measure capture all parts of the conceptual definition? The measure contains all the parts that your theory says it should contain. (E.X.: asking someone one question to see if they have a depressive order, but it could be a symptom of a different ailment).

Neural imaging/activity measures

EEG, EMG, PET scans, MRI, CT scans, and fMRI

Reaction time

Elapsed time between event and response

Inter-Behavior latency

Elapsed time between two behaviors

Latency measures

Elapsed time between two behaviors, or an event and a behavior

Measurement error

Factors that distort your score, so that it isn't what it should be

Principle of justice

Fair balance between the people that participate in research and the people that benefit from the findings. Researchers must show that the people participating in the study are REPRESENTATIVE of the people that will be helped by any positive results.

Rating scales

For measuring quality or intensity of a behavior. E.X.: locomotor scale and emotional intensity rating

How much aversive stimulation is too much?

Generally thought ok to perform research that involves only minimal risk. Controversial procedures rely on decisions of the IRB. Often have to perform a cost-benefit analysis.

Autonomic nervous system activity

Heart rate, respiratory rate, blood pressure, eye movement

How can you achieve unobstructive observation?

Hide, habituation, participant observation, group infiltration, or measure behavior's results (i.e. retrospective studies).

Blood, urine, and saliva assays

Hormone levels, immune response

Duration measures

How long behavior lasts

Construct validity

How we operationalize, or measure, our conceptual variables of interest

Deontological deception

Lying and deceit are immoral and reprehensible acts, even when they are used for good purposes

Can people expect to be observed? E.G., in classrooms, museums, and parks vs. in a public restroom

If they consented to being observed, then they can expect to be observed. IRBs helps ensure ethical observation is used.

Principles of respect for people ((persons))

Individuals should be treated as autonomous agents through obtaining informed consent, not misleading about risks/benefits, and cannot coerce people into perticipating. Those who are not autonomous are provided special protection like children, those with intellectual or developmental disabilities, and prisoners.

Informed consent

Informing the research participants of the nature of the study and obtaining their explicit agreement. Ensures that researchers do not violate the privacy of participants and that participants have enough information to decide whether they want to participate.

What enforces these ethical guidelines?

Institutional Review Board (IRB) and Institutional Animal Use and Care Committee (IACUC).

What constitutes and invasion of privacy?

Investigator decides, but it is generally agreed that public places do not invade privacy but private places (where people expect privacy) may be an ethical issue.

Potential problems with informed consent

It compromises the validity of the study, participants are unable to give consent, or it is impractical to get consent

Seven concerns of the IRB

Lack of informed consent, invasion of privacy, coersion to participate, potential physical or mental harm, deception, lack of debriefing, or violation of confidentiality

Two types of temporal measures

Latency measures and duration measures

Inter-Rater reliability

Measuring the consistency between two behavioral rates (e.x.: two nurses doing blood tests or delivering the same study). Two coders' ratings of a set of targets are consistent with each other

Field notes

More concise descriptions of behavior. Difficult to analyze quantitatively (as are narratives)

What do participants report after they were deceived during a research study?

Most people realize the need for deception in some research, and that it is not the same as lying in everyday life. Most participants report positive feelings about participation regardless of deception. Most participants actually rate study with higher scientific value if it includes deception.

Relationship between reliability and validity

Must have reliability in order to have validity, but you cannot have validity without reliability. Reliability is necessary, but not sufficient, for validity. If a measure is unreliable, then it has a lot of measurement error. Error cannot be valid. Therefore, to the extent to which a measure is unreliable, it is also invalid (it is necessary for a measure to be reliable if it is valid). However, a reliable measure could still be invalid (e.x.: using a scale to repeated weigh a pencil is reliable but it is not valid if you set your paper transcript on it and say that is how you are going to measure academic achivement)

Tools for behavioral recording

Narratives, field notes, checklists, temporal measures, and rating scales

Why use deception in behavioral research?

No other way to answer your research question

Categorical scale of measurement

Nominal scale

Reliable and valid observations

Observer bias and observer effects (reactivity)

Masked (Blind) designs

Observers are unaware of the purpose of the study and the conditions to which participants were assigned

Naturalistic observation

Observing an individual or group in their natural habitat ("real world" environment). Variety of behavioral recording methods. Can only describe behavior (no speculating or assuming). Unobtrusive observation. Importance of external validity. Where lab studies are difficult or less useful, researchers can observe subjects in their real-life environment

Behavioral recording

Observing behavior and recording "what" is happening

Ratio scale

Ordered scale with consistent intervals throughout. Contains a "true zero". Can perform complex mathematical operations and statistics with ratio scale measurements. Cannot perform mathematical operations with nominal scale measurements. E.X.: age and height

Interval scale

Ordered scale with consistent intervals throughout. However, no "true zero". E.X.: temperature (celcius and Fahrenheit)

Continuous scales of measurement

Ordinal scale, interval scale, and ratio scale

Debriefing

Participants are told about the nature of the study after it has been completed

Internal reliability

People give consistent responses on every item of a questionnaire

Two ways to assess inter-rater reliability

Percentage of agreement between raters and correlation between raters

Three main principles for ethical decision-making

Principles of respect for people ((persons)), principle of beneficence, and principle of justice

Principle of beneficience

Protect participants from harm and ensure their well-being (cost/benefit analysis). You should as "what are the risks and benefits of the research itself? What are the risks and benefits to the participants? What are the risks and benefits to others/community?". You cannot withhold beneficial treatments. The control group must eventually recieve the treatments, you must inform about the risks/benefits of waiting, and new discoveries in beneficial treatments must be offered even if study has already started.

The two obligations of researchers

To further scientific research (value of knowledge) and to protect rights of participants (risk to subject). Ethical issues arrise when these are in conflict.

Ordinal scale

Rank ordering of data. Example: Liker-Type scales. "Good", "better", "best". No problems, some/possible problems, and definite problems. Rating movies like Friday the 13th poses a close gap between ranks 1 and 2 but a big gap between ranks 2 and 3. Not equal distance between categories

Evaluative characteristics of behavioral measures

Reliability and validity

Minimal risk

Risk no greater than what is encountered on a daily basis or during routine physical and psychological tests

What is included in scientific misconduct?

Scientific dishonesty (like fabrication, falsification, plagirism, and questionable data deletion), questionable research practices (like authorship of articles, failure to report data, and misuse of grant funds), and unethical behavior related to science (like sexual harassment, abuse of power, and discrimination)

Common types of operationalization measures

Self-Report measures (e.x. filling out a questionnaire about how happy you are or making a friend fill it out about you), observational measures (e.x. record what someone is doing to record interviews), and physiological measures (e.x. biological data like heart rate or brain activity)

How do you increase your reliability?

Standardize administration of measure, clarifying instructions and questions, train observers, and minimize coding errors

Checklists

Structured description containing a list of specific behaviors. Behaviors to record are decided upon in advance

Locomotor scale

Subjects receive a ranking depending on their stepping abilities

Emotional intensity rating

Subjects receive a rating depending on the intensity of their emotional reaction

Tests of reliability

Test-Retest reliability, interrater reliability, and interim reliability

Formal ethical directives in behavioral research

The Belmont Report, APA ethical principles, and APA ethical standards. Reflected in the seven IRB concerns.

Time spent

Time spent doing one activity during a specified time interval

Task completion time

Time taken to complete task

Reliability

The consistency or dependability of a measuring technique (think consistency). Do you get consistent scores every time?

The golden rule of confidentiality in research

The data participants provide can only be used for research purposes and observed by research personnel

Justifications for deception

The study cannot be done without deception, you cannot deceive participants about aspects of the study that may influence their willingness to participate, cannot deceive participants about risks or possible discomfort, and must include debriefing session

Sources of management error

Transient states of the participant (e.x.: being hungry or tired), stable attributes of the participant (e.x.: someone who is always a nervous test-taker), situational factors (e.x.: taking the test in a loud, dark room), characteristics of the measurement (e.x.: blood clips not being made for dark skin tones), and experimenter error (e.x.: handwriting or when a child I observes stole a toy instead of sharing it like I thought they would)

Narratives

Unstructured notes containing a full description of everything participant says or does. Typically from video or audio tapes. Difficult to analyze quantitatively (as are field notes)

Descriptive research

Used to describe the characteristics or behaviors of a given population in a systematic and accurate way as opposed to explanatory (experimental) research

Nominal data

Used to label variables without any quantitative value.

What constitutes deception?

Using confederates, providing false feedback, presenting related studies as unrelated, and lying about stimulus materials.

Concurrent validity (think current)

Validated measure and behavioral outcome measure are administrated at the same time. E.X.: hypochondriasis has a strong correlation to trips to the doctor and number of illnesses

Predictive validity

Validated measure is administered weeks, months, or years before behavioral outcome measure is taken. E.X.: SAT scores have a strong correlation to college performance

Two ways to collect descriptive data

Watch and record (observational and physiological measures) and surveys (questionnaires and interviews)

Conducting empirical research

We operationalize conceptual variables by assigning them tangible metrics that we can systematically observe. All behavioral research involves the measurement of some behavioral, cognitive, emotional, or physiological response. Construct validity is about how well we operationalize, or measure, our conceptual variables of interest. A study is only as good as the measuring techniques (i.e., operationalizations) that researchers use. You want to choose measures that consistently and accurately capture the variability in participants' behavior, thought, emotion, or physiological response

Temporal measures

When and how long a behavior occurs

Justification to waive informed consent

When the research involves no more than minimal risk to participants. The waiver of informed consent will not adversely affect participants. The research could not be carried out if informed consent was required.

Classical test theory

X=T+e X= a person's observed score on the operational variable T= a person's true standing on the construct (aka conceptual variable) e= measurement error

Are there times when you don't have to have informed consent to do your study?

Yes!