PSYC3000 - Advanced Research Methods and Stats
Aside from what's already been covered, what else is in the National Statement?
1. Ethical considerations related to different research methods 2. Ethical considerations related to different research participants 3. Process of research governance and ethical review
What are the general requirements for consent?
1. It is a voluntary choice 2. Based on sufficient information and adequate understanding of the research and implications 3. No coercion or pressure 4. Reimbursement is okay so long as it is proportional to the requirements 5. People are able to say no without reason 6. People can withdraw at any time
What qualities does research with merit possess?
1. Justifiable by its potential benefit 2. Designed and developed using appropriate research methods 3. Based on a thorough literature review 4. Designed to ensure respect for the participants 5. Conducted by experienced researchers 6. Conducted using the appropriate facilities.
What should be avoided when writing questions?
1. Leading questions: Subtly prompting the respondent to answer in a certain way 2. Double-barreled questions: Ask about more than one thing, but only allow for one response 3. Double-negatives: They become positive and rather confusing 4. Single negatives: Can also be difficult to rate depending on response options (e.g. Guns should never be controlled; Agree or disagree?) 5. Question order: Earlier questions an influence later questions, and in turn how the participant responds (e.g. How often do your children play? Responses may differ if you previously ask about sport, music, daily activities, etc).
Describe the Principle of Justice.
A fair balance between participants and those who will benefit must exist. Participants must be representative of those who will benefit; they cannot just be convenient.
What are observational measures and what are their advantages and disadvantages?
Observational measures operationalise a variable by recording observable behaviors (e.g. accuracy, RT, etc). Advantages include being more controlled and being a direct measure of behaviour. Disadvantages include being limited to behavioural measures that we use to make inferences.
What constitutes a good observation?
Observations should be reliable and valid, in that there are clear rating scales and multiple observers (to reduce bias). The findings of the observations should also be generalisable to other samples/populations.
What is a hypothesis?
Our predictions about the answers to the research questions if our theory is accurate.
Is observation more reliable than self-report?
Observation can be more accurate, as people won't/can't always tell you what they do. However, observer bias is a potential threat to construct validity.
What is the National Statement on Ethical Conduct in Human Research?
"'Ethical Conduct' is more than simply doing the right thing. It involves acting in the right spirit, out of an abiding respect and concern for one's fellow creatures." The purpose of the National Statement is to promote ethically sound human research, and requires that participants be accorded the respect and protection that they deserve.
What must a operational definition include?
1. A detailed definition of the concept 2. Information about the scope of the concept being measured 3. Detailed information about how the concept will be measured; standardised measures, numerous categories, etc. This process may include looking at how past research has defined the concept.
Describe the basic-applied cycle.
1. Applied research: Done with a practical problem in mind, and hope to apply the findings in a real-world context 2. Basic research: Done to enhance knowledge, often extended upon in later applied studies
What are the variants of random sampling?
1. Cluster sampling: Take a random sample of subgroups 2. Multistage sampling: Take a random sample from within a random cluster 3. Stratified random sampling: Randomly sample to meet population characteristics 4. Oversampling: Variation of SRS, sample a larger portion of characteristics, then weight results by true proportion 5. Systematic sampling: Randomly choose a start point and count number
What are some sub-types of the within groups design?
1. Concurrent measures: All participants are exposed to all levels of the IV at almost the same time and then required to give attitudinal or behavioural preference. E.g. Taste test preferences 2. Repeated measures: Measured on DV after exposure to each level of IV. E.g. Pre-test then post-test
What are the four types of validities?
1. Construct 2. External 3. Statistical 4. Internal
What are some non-random sampling methods?
1. Convenience sampling: Ask people who are easily available 2. Quota sampling: Convenience sample people to meet certain criteria 3. Purposive sampling: Find people who match specific characteristics 4. Snowball sampling: Ask your participants to find you more participants
How can we establish causation?
1. Covariance: An association that establishes that A and B are related 2. Temporal precedence: Determine which of the variables comes first 3. Internal validity: Is there a third variable that is associated with A and B independently?
What are the three different types of correlations?
1. Cross-sectional: Look at variables measured at the same time point 2. Autocorrelations: Looks at the same variables over time 3. Cross-lag correlations: Look at the relationship between different earlier and later variables (shows temporal precedence and 3 possible patterns)
What three major ethical violations were committed during the Tuskegee Syphilis Study?
1. Men were harmed: They were not told about the treatment, and were subjected to painful tests 2. Men were not treated respectfully: They were lied to about the aim of the study, had information withheld from them, and had no opportunity for informed consent 3. Researchers targeted a disadvantaged social group: Anyone can contract syphilis.
What are the four levels of measurement?
1. Nominal: Categorical, assigning number/code to levels that have no numerical meaning 2. Ordinal: Rank order, but without idea of distance between ranks 3. Interval: Numerals represent equal intervals, but no real zero 4. Ratio: Numerals represent equal intervals and there is a true zero
What are the different types of question format?
1. Open-ended questions: Allow time for a response, time consuming and costly (code and categorise responses). E.g. Tell me about someone you admire. 2. Forced-choice: Pick the best alternative from two or more options, sum the number of times a certain response is given. E.g. Do you prefer red or green apples? 58% prefer red. 3. Likert Scale: Strongly Disagree <----> Strongly Agree. Easy to score, can include a neutral option 4. Semantic differential: Rate statements on different dimensions (happy vs sad, hot vs cold). Easy to score, and can also have the neutral conundrum
How does one get a representative sample?
1. Probability sampling: Draw a sample at random, so that every member of the population has an equal chance 2. Simple random sampling: Basic version of probability sampling, e.g. All names in a hat. Hard to perform, so we often use variants
How can people be observed ethically?
1. Public places: Expect behaviour in these places to be public 2. One-way mirrors/hidden cameras: Need to tell participants afterwards and give them the option to remove their data 3. Ultimately it is up to the HREC to decide
What are the three Rs of animal research?
1. Replacement: Find an alternative method if possible 2. Refinement: Minimize distress by modifying experimental procedures. 3. Reduction: Use as few animals as possible.
What are the three ethical principles?
1. Respect 2. Beneficence 3. Justice
What causes biased samples?
1. Sampling only convenient people 2. Sampling only the people you contact 3. Sampling only people who invite themselves (self-selection, this is common for internet polls)
What might undermine the reliability of observation?
1. Seeing what the observer wants to see, and disregarding anything else. 2. The act of observation can change behaviour: Consider masked or blind studies 3. The observed might react to being observed (reactivity): Hide, wait for the observer to get used to you, or measure the behaviour's results (unobtrusive data)
What are the different types of consent?
1. Specific: For this study only 2. Extended: For this and future related projects 3. Unspecified: For any future projects
What are the different types of associations?
1. Strength: Strong (r=.90), Moderate (r=.70), Weak (r<.70) and No (r=.00), depending on the presence of a clear linear pattern and how uniform the line is. 2. Positive/Negative: If X increases as Y does, then it is positive; If X increases as Y decreases, then it is positive. 3. Linearity: Association can be linear or curvilinear.
What is the difference between a Type I and Type II error?
1. Type I (false alarm): Saying there is a relationship when there isn't 2. Type II (miss): Saying there isn't a relationship when there is.
What types of inaccurate responses can participants give?
1. Using shortcuts/response sets: Adopting a consistent way of answering all of the questions. Can be difficult to deal with 2. Yea- or Nay-saying: Strongly agree or disagree with every question, threatens construct validity. Can be controlled for using reverse-worded items 3. Fence sitting: Sitting in the middle of the scale; this is common for controversial items and may indicate that some items are unclear. Can be remedied by removing neutral option, but this could cause problems as well 4. Socially desirable responding: People trying to make themselves look better on a survey reduces construct validity. Can be remedied by keeping responses anonymous, including filler items to distract from the study's focus, and including check questions. 5. Self-reporting more than the individual knows: Sometimes it's their only option (How much do you like X), or when asked about a reason for behaviour 6. Self-reporting memories of events: Often not very accurate, even for memorable events, usually still confident though.
What questions should be asked to assess the statistical validity of association claims?
1. What is the effect size? 2. Is the result significant? 3. Are there any subgroups? (Sometimes associations don't hold between subgroups) 4. Could there be outliers affecting the relationship? (Outliers can have big effects, especially in small samples) 5. Is the relationship curvlinear? (If so, a quadratic model will need to be employed to test significance)
What ethical questions surround the Milgram obedience study?
1. What was the risk to participants? The experiment was extremely stressful, with numerous reports of shaking, sweating, anxiety, etc, and there was a possibility of long lasting effects despite being debriefed (knowing that they were willing to harm someone because someone told them to. Researchers did follow up many years afterwards to check on them). 2. What were the benefits? We learnt a lot about human behaviour; the original hypothesis was that 1-2% would comply 3. Should they have stopped after the first study? Hard to say
Describe Bushman (2002) experiment.
600 students wrote a political essay, and 'Steve' gave them a very negative feedback to induce anger. Group 1 was then asked to sit quietly for two minutes, 2 exercised for 2 minutes using a punching bag, and 3 punched a punching bag for two minutes imagining Steve's face on it. All students then played a quiz with Steve, in which they were able to blast his ears with loud music. Based on our own experience, one would imagine that Group 3 calmed down the most, however Group 3 was the most riled up (followed by 2, and then 1).
What are causal claims?
A claim in which one variable is responsible for changing another variable. These claims are often held to a higher standard; such claims start with an association, and then it is shown which variable came first (i.e. caused changes in the second variable).
What is an association claim?
A claim that argues that one level of a variable is likely to be related to a particular level of another variable; e.g. People with poorer health tend to be overweight. A minimum of two variables are included, which are only ever measured (not manipulated). These claims can be used to make predictions, and statistics can be used to investigate the relationship.
What is a frequency claim?
A claim that describes the rate of something; e.g. 30% of Australians received speeding tickets last year. They focus on one variable, which is only ever measured (not manipulated).
What is test-retest reliability?
A consistent pattern of results each time a person is tested on a stable construct. There should be a strong positive association between initial and subsequent test scores.
What is a moderator?
A moderator is present when the relationship between two variables changes depending on the level of a third variables. They often reduce the external validity as it makes it harder to generalise the findings.
What does the National Statement on Ethical Conduct in Human Research cover?
A number of values and principles: 1. Respect for human beings 2. Research merit and integrity 3. Justice 4. Beneficence These values of ethical conduct help shape the relationship between researchers and participants as one of trust, mutual responsibility, and ethical equality.
What is a risk?
A potential harm, discomfort, or inconvenience, whether it be physical, psychological, social, economic, or legal. Minimizing risk involves the assessment of the research aims, their importance, and the methods by which they can be achieved.
What is observational research?
A process that involves a researcher watching and systematically recording what a person or animal does. This can be used for all claim types.
What is debriefing?
A process that is used to explain the importance of the study and to outline deception, so that participants can remove their data should they wish to do so. It is also used for non-deceptive studies as an educational tool.
What is data?
A set of observations that may strengthen our confidence in a theory or lead to a revision of our theory.
What is a theory?
A statement, or set of statements that describes general principles about how variables relate to one another.
What is multiple regression?
A statistical test that tells us if a key relationship still holds even when a suspected 3rd variable is statistically controlled for. Key terms include: 1. Dependent or Criterion variable: The variable that we want to understand/predict 2. Independent or Predictor variables: The variables used to predict the criterion variable
What is face validity?
A subjective measure of validity that involves the measure looking like it's measuring what it's supposed to. This is usually checked by consulting experts in the field.
What is content validity?
A subjective measure of validity that questions whether or not the measure properly captures every aspect of the construct being measured. The breadth of this will depend on the operational definition of the construct being measured.
What are the advantages and disadvantages of within subjects designs?
Advantages: 1. Ensures equivalence among participants exposed to each level of IV (each person is their own control) 2. More power to notice differences (reduced random error) 3. Need fewer participants (e.g. 12 people to do 2 levels vs 24 people to do one level each) Disadvantages: 1. Potential for demand characteristics: Participants might get cues as to what the experiment is about and change their behaviour accordingly
What are the advantages and disadvantages of a between-subjects design?
Also referred to as independent-groups, people are allocated to a single level of the IV. It's main advantage is that there are no order effects, whilst it's main disadvantage is that it is the least effective method for controlling participant related variables
What are claims?
An argument that someone is trying to make.
What makes a good theory?
Being supported by data, falsifiable, parsimonious, and never proving anything.
Significance is related to effect size; the __________ the effect size, the __________ likely it will be significant. Significance is also related to sample size; a __________ correlation may be significant in a large sample, but not a small sample because small samples are more easily affected by _________.
Bigger; more. Small; chance.
What is empirical research?
Collecting data and using it to develop, support, or challenge a theory.
What is interrater reliability?
Consistent results no matter who measures or observes. If two observers are rating the same people at the same time, we have good interrater reliability if r is strong and positive.
What should be assessed when investigating frequency claims?
Construct, external and statistical validity should be assessed: - How well did they measure their variables? - How generalisable are their findings? - How big is the margin of error in their estimate?
What should be assessed when investigating association claims?
Construct, external and statistical validity should be assessed: - How well have all the variables been measured? - How generalisable are the findings? - How strong and reliable is the association? - Could it be a Type I or Type II error?
What should be assessed when investigating causal claims?
Construct, external, statistical, and internal validity should be assessed: - How well have all of the variables been measured? - How generalisable are the findings? - How strong and reliable is the association? And could there be any Type I/II errors? - Was the study an experiment? - Does it achieve temporal precedence? - Can you rule out any alternative explanations?
What is meant by critical thinking?
Critical meaning does not mean being constantly skeptical. However, it does mean knowing what to prioritise when assessing a study, how to ask relevant questions when assessing a study, and knowing how to assess the answers to your questions when assessing a study.
What did Elliot et al (2007) find on the colour red?
Elliot et al gave participants a cognitive test with either a red, green, or white paper cover. Students with red covered booklets scored lower than the other two groups. In another study students with the red cover attempted more of the easy than difficult suggestions. This suggests that the colour red primes people with an avoidance mindset.
What is predictive validity and concurrent validity?
Empirical measures of validity that are sometimes grouped as criterion validity. They investigate whether or not a measure is associated with something it should be. Predictive validity predicts the future outcome, whilst concurrent validity predicts the outcome of another measure taken at the same time.
What is convergent and discriminant validity?
Empirical measures of validity that is concerned with whether or not the measure shows meaningful patterns of relationships with other measures. In convergent validity, the measure of interest should correlate more strongly with other measures of the same construct. In discriminant (divergent) validity, the measure of interest should show weaker correlations with measures of different constructs.
Compared to research, is experience a good source of information?
Experience is a powerful source of information that can assist us in deciding what to and what not to engage in. However, it possess too many confounds and no comparison group. While our personal experience may differ from research, we cannot simply disregard research; behavioural research is probalistic, and conclusions don't explain every case, but rather a high proportion of them.
What are the four sources of information?
Experience, intuition, authority, and research.
Can cognitive training improve fluid intelligence?
Fluid intelligence is our ability to reason and solve new problems, whist working memory is our ability to hold and manipulate information in our mind. Previous links between WM and GF have been found in the past, and after an experiment involving 19 training sessions with an m-back task, people in the training group showed a larger improvement in Gf than those in the control group.
What are the three types of claims?
Frequency, Association, and Causal.
What can be considered as a benefit in research?
Gains of knowledge, insight and understanding; improved social welfare; increased individual well-being, gains in skill and/or expertise, and direct benefits to participants.
What is statistical validity?
How accurate and reasonable the claims are. Good statistical validity relies on minimizing Type I (false alarm) and Type II (miss) errors. The strength (r) and significance (p-value) should also be examined.
What is reliability?
How consistent a measurement is.
What is external validity?
How generalisable the findings are to other contexts and populations. If the construct is operationalised using a very narrow scope, it may not be generalisable. Random sampling helps external validity.
What is construct validity?
How well the variables were measured; particular attention should be payed to how the constructs were operationalised. The variable must be measured reliably to have construct validity.
What qualities does research with integrity possess?
It has been carried out by researchers with a commitment to: 1. Searching for knowledge and understanding 2. Following recognized principles of research conduct 3. Conducting research honestly 4. Disseminating and communicating all findings.
What is the SNARC effect?
It is interested in our mental number line: do we associate numbers with the left and large numbers with the right, and is this innate or learned? Three groups (Canadians words and numbers L>R, Palestinians words and numbers R>L, and Israelis numbers L>R and words R>L) took part in a digit parity task (is digit odd or even with counterbalanced blocks of L and R hand responses). Results were as follows: Canadian neg, Israeli none, and Palestinian positive.
What are some of the best sources of information?
Journal articles are the best, being the most up-to-date and providing details on research methods. Chapters in edited books aren't as good as journal articles, as the possess less info than original papers and are usually a summary of a researcher's work. Books are good and provide a general overview, but can be out of date.
What is justice?
Justice involves taking into account the scope and objectives of the proposed research, the selection, the exclusions and inclusion of categories of research participants is fair and is accurately described in the reporting of the research. The process of recruiting people is fair, and there is no unfair burden of participation. There is no unfair exploitation of participants, and there is fair access to the benefits of the research. The outcomes should be made accessible to participants in a clear and timely manner.
What are we obligated to treat our participants with in our research?
Kindness, respect, and fairness.
What are longitudinal designs?
Measure the same variables in the same people at different times. It is often used in psychology to study changes in a trait with ability or age, and can be used to show temporal precedence.
Describe the pseudoword effect.
Memory studies are typically performed in two phases - a study phase and a test phase - and are measured in terms of hit rate and false alarm rate. In a normal word-memory test, a mirror effect occurs; high frequency words possess a lower hit rate and higher false alarm rate than low frequency words. However, Greene (2004) found that using psuedowords removes this mirror effect (non words have a higher hit and false alarm rate than words).
Describe the peer-review cycle.
Most commonly occurs when we submit a paper for publication in a journal; the article is sent to experts to review, and then returned with comments and suggestions. Researchers then address these and send it back, and thus the cycle continues.
Is authority a good source of information?
Not really, what they say should be taken with a grain of salt and thoroughly investigated before accepting any information.
What is internal reliability?
Participant gives a consistent pattern of answers to questions about the same construct. We usually ask the same thing in a number of ways to reduce random errors in responses. We want the answers to these items to correlate strongly with each other so we can calculate a mean score.
Describe the Principle of Beneficence.
Participants must be protected against harm and their well-being must be ensured. The benefits must outweigh the risks, and bigger benefits typically allow for greater risk.
Describe the Principle of Respect for Persons.
Participants should be treated as autonomous agents able to make their own decisions. They have a right to privacy, and are entitled to give informed consent; they cannot be mislead or coerced. Some cases such as autonomous surveys and naturalistic observations do not required written informed consent.
What are physiological measures and what are their advantages and disadvantages?
Physiological measures operationalise a variable by recording biological data (e.g. Brain activity, HR, hormone levels, etc). Advantages include removing bias from both the participant and the experimenter, as well as gathering subconscious or hard-to-articulate data. Disadvantages include being expensive, data can be difficult to interpret, and not always accurate.
What must we be cautious of with within subjects designs and how can we deal with this?
Potential for carryover/practice/order effects can threaten the internal validity of the experiment, as exposure to one condition may change reactions to another conditions, thus introducing a potential confound. Counterbalancing conditions can fix this.
What is research misconduct and what are the different types?
Research misconduct is another ethical dilemma that impedes the progress of science by leading people on a wild goose chase. Types include: 1. Data fabrication: Creating own data to fit hypothesis 2. Data falsification:Influencing collected data 3. Plagarism
Random __________enhances external validity, random assignment enhances __________ internal validity.
Sampling; assignment
Describe the theory-data cycle.
Scientists collect data to test, change, or update their theories.
What is self-report and what are its advantages and disadvantages?
Self-report operationalises a variable by recording people's answers to questions about themselves (e.g. surveys, interviews, etc). Advantages include getting the participant's own view and the method being quick and inexpensive. Disadvantages include being subject to response bias, possible inaccuracy, and the need to be careful with question wording.
What are the common types of measurement?
Self-report, observational (behavioural), and physiological measures.
Should deception ever be used in a study?
Sometimes it is necessary, as seen in studies investigating the effect of subliminal messages; if participants were aware that they were listening to something else, the data would be useless. Deception breaks the rules of respect, but the risks and benefits of its use need to be evaluated before its implementation. For example, people may be spending lots of money on subliminal educational CDs; are they wasting their money?
What does statistical significance mean?
Statistical significance calculations help us evaluate the probability that our result came from a population in which there really isn't a difference. The probability estimate (p-value) tells us the probability of our result being a false alarm, thus if p is very small then we can say it is very unlikely that our result is a false alarm. Instead, we say it is statistically significant. If p is large, then we can't rule out the possibility of a false alarm and that there isn't really an association/difference in the population.
What is beneficence?
The likely benefits must justify the risks of harm or discomfort. Benefits may be to the participants, wider community, or both. Researchers are responsible for designing research to minimize harm/discomfort, clarifying potential risks/benefits to participants, and the participants welfare in the research context. When risks are no longer justified by the benefits, research should be suspended or modified.
Describe the journal-journalism cycle.
The media is the most common place you will need to interrogate information. It can be beneficial to have the media publicize your research, but only if it is reported accurately.
What is respect?
The recognition of someone's intrinsic value. This requires having due regard for their welfare, beliefs, perceptions, customs, and cultural heritage. This also involves having respect for participants' privacy, confidentiality, and cultural sensitivities, as well as giving them due scope to enable them to make their own decisions (if they are unable to do so, respect involves empowering them where possible, and protecting them as necessary).
How does the National Statement on Ethical Conduct in Human Research define 'human research'?
The statement defines human research as research conducted with or about people, or their data, or tissue. This can include surveys, interviews, research groups; psychological, physiological, medical testing or treatment; being observed by researchers; researchers having access to personal documents; collection and use of body organs, tissues, fluids or exhaled breath; and access to information as part of an existing published or unpublished source or database.
What are the four scientific cycles?
Theory-data, basic-applied, peer-review, and journal-journalism.
Since psychological concepts are difficult to see/touch/feel/etc, what must we do in order to measure them?
We need to operationalise them. This can be done outlining exactly how such variables are defined and measured. However, this can be difficult and can cause discrepancies in past research.
Is intuition a good source of information?
We often think so, but we are too biased for it to be effective. We often think the 'easy way' (some information comes to mind more easily, easier to pay attention to memorable events, accept conclusions that "make sense", and present/present bias) and to think what we want (resistant view changes, cherry-picking info, asking biased questions, intuition makes us over confident).
What is internal validity?
Whether or not there are any other possible explanations for the claim being made. These can include extraneous variables that may not have been controlled for, not using standardised instructions, the presence of counter-balanced order in tests, and the inclusion of demand characteristics.
