Research Methods Exam 1

Confound (or confounding variable)

A variable that varies systematically with the variables in a study and is a potential alternative explanation for causality. - messes up relationship - can't pinpoint it

Validity:

Accuracy of findings or measures

References

All the sources cited within the article will be listed in a References section or in footnotes throughout the article

Debriefing

Clearing up any misconceptions that the participant might have and addressing any negative effects of the study. - If the study involves any risk or deception, the researcher should include a debriefing in order to reduce or mitigate any longer-term effects on the participants. In most cases, debriefing occurs right after the participant completes the study - discussing the study immediately afterwards can help assess and reduce the distress, and the researchers can identify an appropriate follow-up plan for those who may need additional help.

Reliability

Consistency of findings or measures.

Commentaries:

Critique or comments about a published research article.

One common way to compute the internal consistency of a scale is

Cronbach's alpha (α) or Split-half reliability

Authors

If there are multiple authors, the author list is usually organized by the degree each contributed, with the first author as the person who was most responsible for the work.

Content validity:

Inclusion of all aspects of a construct by items on a scale or measure. - content domain - measure through SMEs

Theses and Dissertations

Most often they are original research studies and thus primary sources, but some are reviews or meta-analyses

discriminant validity

Negative or no relationship between two scales measuring different constructs. (close to 0)

Random selection with replacement

a selected member of the population is returned to the pool of possible participants and thus may be selected into the sample more than once.

Split-half reliability

a simple correlation of the sum of the scores of half the items to the sum of the other half of the items. - a correlation of .70 or higher is the accepted standard

The Scientific Approach

a specific type of critical thinking that involves: 1) approaching a topic with a genuine desire to understand it 2) identifying and minimizing biases that interfere with this understanding 3) avoiding overly simplistic explanations 4) following a systematic method to study the topic - scientific approach can improve our decision making, and it can help us develop a deeper understanding of a topic - science must include a transparent method that can be evaluated and replicated by others - the scientific method is not strictly linear, but rather is a cycle

sample

a subset of the population that is meant to represent the full population

power

ability to find stat sig patterns/relationships - dependent upon sample size (must be large enough)

Construct:

abstract trait that can't be directly observed (aggression, intelligence, attachment, hope)

Types of Scholarly Works

academic journals, conference papers/posters, unpublishe manuscripts, scholarly books, thesis/dissertations, undergrad research, abstracts

open-ended response format

allows respondents to provide their own answers, it is nonnumerical and qualitative - · Used in interviews or pilot studies

confidence interval

an estimation of the margin of error for your scores, or the range of values within which your scores will fall (5%)

anonymous vs confidential

anonymous: occurs when it is impossible for anyone, including the researcher, to link a participant to his or her data confidential: no one gets the info; researcher may be able to link the participant with his or her responses

Unpublished Manuscripts

articles that have been accepted for publication in an academic journal but are not yet published (in press), are currently under review for publication, have not been submitted for publication, or were rejected from an academic journal. - should be cautious of using other unpublished manuscripts that you find online, paying special attention to the quality of the work

secondary source

at least one step removed from the original source of information - ex: a review or discussion of previous research that does not include a report on an original research study

nonmaleficience

avoiding harm to others - researchers must weigh the potential benefits of the study with the potential risk to humans/animal participants

Title

brief description of the study and will usually include the key variables examined in the study.

scholarly work

can be a primary or secondary source - The goal of the work is to advance knowledge and scientific study in the field. - The author(s) have expertise in the field. - The work is written for an audience with knowledge in the field, as opposed to the general public. - The work builds on other sources that meet the above criteria for scholarly works, and these sources are clearly cited.

Measurement reliability

consistency of measurement

probability sampling is the best choice when your main goal is to

describe a population and you are able to identify all the members or clusters in a population, obtain the appropriate sample size, and minimize the nonresponse rate

for deception

essential in creating and studying a rare occurrence (e.g., emergencies) and eliciting genuine responses from participants. Additionally, some claim that deception has only a negligible effect on participants' well-being and the credibility of the field

an ethics code

guides ethical decision making and describes the ethical standards - researchers must consider ethics at every stage of the research process - ex: Privacy and Confidentiality, Human Relations, Competence

KR20

internal consistency for dichotomous data

critical thinking

involves carefully analyzing info based on current knowledge (not personal opinions/beliefs) - identify/avoid bias - parallel path with thinking like a researcher

confidence level

is a measure of how sure you are that your scores will fall within that confidence interval (95% or 99%)

Quota sampling

is nonprobability sampling that results in the sample representing key subsets of your population, or subpopulations based on characteristics such as age, gender, and ethnicity - accomplished without random selection

ethical principles

moral values/ideals - don't explain how to behave but serve as guidelines in ethical decision making -ex: Principle A: Beneficence and Nonmaleficence, Principle B: Fidelity and Responsibility

anonymity

no one knows the identity of the participant

If your primary goal is not to describe a population but rather examine relationships, as in correlational and experimental designs,

nonprobability sampling is a perfectly fine and common method of sampling

Ratio scales

o A scale of measurement where values measure quantity and have order, equal intervals, and a true zero. o Ex: distance, time, weight, reaction time

Ordinal scales

o A scale of measurement with numbers that have order so that each number is greater or less than other numbers but the interval between the numbers is not equal; also called rankings § Runners finishing 1st, 2nd, 3rd · Nonequal intervals

A measure cannot be valid unless it is

reliable

In testing reliability, the closer the correlation is to 1.0, the more

reliable the scores on the scale are

The Scientific Approach and Decision Making

scientific approach can help us make better decisions, both individually and as a society

descriptive research

simply describes a sample or a population

Prevalence

the commonness, or frequency, of a behavior, attitude, characteristic, or condition within a specified time period - Descriptive research can be used to provide a quick snapshot of the prevalence of a phenomenon.

external validity

the extent to which the results can be generalized to other sample settings, procedures

internal validity

the extent to which you can demonstrate a causal relationship between the IV and DV

confidentiality

the identities are confidential but may be known to the researcher

Convenience sampling

the most basic type of nonprobability sample in which those who were available and willing to provide data make up the sample -

Closed-ended response formats

typically quantitative measures that provide options the respondents select from and that can vary from dichotomous options, to multiple choice, to likert - most questionnaires

· A measure can be reliable but not .

valid

Test-retest reliability

· A measure of the stability of scores on a scale over time · Test-retest reliability is the only type of reliability that can be assessed for a single item that is used to measure a variable - same ppl take the test - pearson correlation

Population

· The group that a researcher is interested in examining defined by specific characteristics such as residency, occupation, gender, or age.

Measurement validity

· accuracy of measurement o Represents variable we are studying o Instrument/factor is measuring what it's supposed to measure

Descriptive studies also allow for

· an in-depth examination of a topic. This type of research can be especially useful for examining a relatively new phenomenon · might be used to gain a better understanding of behaviors and attitudes.

Examine a Phenomenon in a Different Population

· another reason for conducting a descriptive study might be to consider if patterns and prevalence of behaviors and attitudes that are found in one population are similar in a different population

The larger the sample size relative to the population size and heterogeneity, the better...

· external validity the study has because it improves your chances that your results will generalize to other samples.

Survey Research

· involves asking people to report on their own attitudes and behaviors. Such self-reports can provide insight into how individuals see themselves and allow the researcher to obtain information about people's thoughts and feelings that cannot be directly observed. · Self-reports may not be accurate - Social desirability bias:

Observational research

· involves observing and recording the behavior of humans or animals. · Observations may stand alone, or they may supplement other research methods. · The key advantage of observations in human research is that observations focus on what people actually do, not what they say they do or what they intend to do. · downside to observations: they are time-consuming, prone to observer bias

participant observation

· involves the active involvement of the researcher in the situation under observation. o Participant observation may involve a confederate who interacts with participants in a brief task or situation, or it might involve a deeper infiltration into a social group, either covertly or overtly

Nonprobability sampling (also called nonrandom sampling)

· is any method of sampling that does not rely on random selection

Nonparticipant observation

· occurs when the researcher is not directly involved in the situation

Sampling bias

· occurs when the sample does not represent the population, such as when some members of the population are more likely to participate than others and therefore be overrepresented. - Sampling bias is of particular concern in descriptive research where the primary goal is to describe the population and maximize the study's external validity

Random assignment

· refers to how you assign members of your sample to groups within your study and is used only in experimental designs.

Random selection

· requires careful planning to ensure that the sample was chosen only on the basis of membership in a specific population or subset of the population and not other individual characteristics. It also means that each member of the population has an equal chance of being selected. · Random selection is a sampling procedure used to ensure a representative population

unobtrusive observations

· the observer examines traces of animal or human behavior, such as tracks or garbage

sampling

· the procedure used to obtain the sample

Confederate

Someone who pretends to be a participant or uninvolved person, but in actuality is working with the researcher and has been told to behave in a particular way.

Overview of the Research Process (A.K.A. the Scientific Method)

Step 1: Identify Your Topic Step 2: Find, Read, and Evaluate Past Research Step 3: Further Refine Your Topic and Develop a Hypothesis or Research Question Step 4: Choose a Research Design Step 5: Plan and Carry Out Your Study Step 6: Analyze Your Data (based on the hypothesis) Step 7: Communicate Results

Popular works

those that serve to educate or entertain a general audience that includes those without specialized training or expertise in the field - may be written by experts in the field or by journalists or others without specialized knowledge or training in an area

meta-analysis

uses the statistical results and sample sizes of past studies to synthesize results. Like a literature review, it does not report the method or results of a new study

constructs

variables that cant be directly observed/assessed

Scholarly Books

written by experts in the field and are typically published by professional organizations or universities - The time lag from implementation to publication of studies within scholarly books is often lengthy, and you should be aware that more recent work on a topic can probably be found in journal articles.

Subpopulation

· A portion or subgroup of the population.

Replication

· Conducting the same study with new participants (literal replication) or conducting a study examining the same patterns or relationships but with different methods (conceptual replication).

Reliability of a study

· How consistent the results are across similar studies. · In research we are seeking to identify general patterns of behavior

Validity in Descriptive Studies

· In order to accurately describe something, the measures used must be valid - · internal validity of the study is not a concern in descriptive research. Instead, external validity of the study is of primary concern in descriptive research.

Qualitative measure

· Nonnumerical assessment. o Ex: observations

Quantitative measure

· Numerical measure. o Rate on scale Still requires interpretation of what the value means according to behavior

Internal validity:

· The degree to which we can say that we found an accurate relationship among variables, in that changes in one variable (the DV) are caused by changes in another variable (the IV). Relevant only to studies examining causation - We want to be confident that our findings are directly related to the independent variable -

alternate forms reliability

· The relationship between scores on two different forms of a scale · the respondent takes a measure twice - more than one form of the measure and the forms are considered to be equal in their ability to measure a construct · one way to avoid practice effects that can occur when one uses the same test or scale to establish reliability - same ppl take the test - pearson correlation

Secondary Data (Archival Data)

· These are data that were collected for research purposes by some other researcher or organization. - Sources of secondary data include governmental agencies, nonprofit organizations, colleges and universities, data repositories, and individual researchers

Face Validity

· Whether a particular measure seems to be appropriate as a way to assess a construct.

Probability sampling (also called random sampling)

· any method of sampling that uses random selection in which all members of a particular population or subpopulation have an equal chance of being selected. - reduces sampling bias and increases the chance that the sample will be representative of the population

Contrived observations

· are those that are set up for research purposes and might include observing participants' reactions to an event or physical stimulus or asking participants to complete a task or engage in an activity o can occur in a laboratory setting where the researcher has the most control over the setting, but they may also occur in a natural environment such as a home, school, or public place.

Cronbach's alpha (α)

· computes the correlation between responses to all of the items in a scale o alpha of .70 or higher (α≥ .70) is desired Cronbach's alpha analysis can tell us the intercorrelations among items and also how alpha would be affected (increased or decreased) if we delete an item

Records and Documents (Archival Data)

· countless records and documents that were not created for research purposes. o This includes confidential information such as school or medical records that can be accessed only if consent is given by an individual or institution o websites, newspaper articles, public records, and historical documents. o Online social networking, dating, or shopping sites can provide data about users' attitudes, social networks, and behaviors

Researchers can also use descriptive research to determine

· if the patterns they found in their study fit with existing research and theories

Structured interviews

· include a standard set of questions that the interviewer asks all participants. The interviewer does not vary the order of questions or the manner in which they are asked. The interviewer also has strict guidelines for how to answer any questions that the participant might have o The rationale for a structured interview is that it ensures that all participants had similar interview experiences and the exact same questions. It also reduces the potential for interviewer bias

Archival research

· involves the analysis of existing data or records

Maximum variation sampling

· is a sampling strategy in which the researcher seeks out the full range of extremes in the population. - The goal is to achieve a representative sample through this purposeful sampling instead of relying on probability. - Maximum variation sampling is most commonly used with small samples such as those achieved through interviews

Semi-structured interviews

· more flexible than structured interviews. The interviewer has a base set of questions or topics that he or she wants to cover, but can prompt the participant for more information, add new questions based on the participants' responses, and clarify questions as necessary

Naturalistic observations

· occur in the participants' (or animal subjects') natural environment and take place without any interference by the observer or researcher A downside to the naturalistic approach is that the observer has no control over the situation, and a long time may pass before the behavior of interest occurs. Or, the behavior might happen so frequently that it is difficult to capture

Internal Consistency

there is consistency in the way that the participant or observer responded to the multiple items on the scale

Strategies to Identify and Find Past Research

1. Searching Library Databases by Topic 2. Identify the Appropriate Databases to Search 3. Conduct the Database Search

Discussion

(also named Conclusions in some journals) will usually begin with an explanation of the results without the technical language. - It will also put the results into context—usually first stating if the results support or do not support the hypotheses and then explaining how the results fit or do not fit with past research. - The Discussion section will also suggest what the larger implications and applications of the study might be, point out limitations of the study, and offer suggestions for future research that may address limitations and expand on the results of the study.

APA Format for References

- Author(s) names (last name followed by comma, initial[s] followed by period[s]; comma between individual author's names) - Year of publication, in parentheses - Article title = Journal title and volume - Do not include issue number unless the journal begins numbering each issue with page 1. - Page numbers of article - doi number, if available Italicize the journal title and the volume number, but not the page numbers (pg 108)

Against deception

- deception may harm participants by embarrassing them, making them feel uncomfortable, or leading them to mistrust others - deception may harm the field by increasing suspicion of research and decreasing the integrity of the individual researcher and the entire research community - deception may invalidate research results even in studies that do not use deception (respond differently)

Incentives for Participation

- offer an incentive for participation in order to recruit participants. 1. Researchers should carefully consider who their potential participants are and not offer incentives that they would have a difficult time refusing. 2. The incentive should not be contingent on the participant completing the study. - should be fair

External validity

- the ability to generalize the results of a study to other settings, other samples, and other methods - The more diverse the sample and the more realistic the setting and procedures, the greater the likelihood that the results will generalize beyond the particular people and setting of your study

Steps to the Scientific Method (7) T.R.H.D.D.A.C (Tiffany Ran Home During DOdgeball And Cried)

1) Identify a Topic 2) Find, Research, and evaluate past research 3) Refine topic and develop a Hypothesis 4) Design the study 5) DO the study 6) Analyze the data 7) Communicate results

An experiment requires that:

1. The experimenter systematically manipulates the independent variable (IV). 2. The experimenter randomly assigns participants to receive different levels of the IV. 3. The experimenter measures the effect of the IV manipulation on the dependent variable (DV).

APA ethics code (2010a) specifies that deception is allowable under the following conditions:

1. The use of deception is necessary and justifiable given the potential benefits of the study. 2. The study is not expected to cause any physical pain or significant emotional distress. 3. The researchers debrief participants as soon as possible regarding the deception.

Organization of Primary Research Articles

1. Title 2. Authors 3. Abstract 4. Introduction 5. Method 6. Procedure and Design 7. Results 8. Discussion 9. References

· Interrater reliability:

A measure of agreement between different raters' scores. - Interrater* reliability is computed by correlating the different raters' scores

Physiological measure

A measure that assesses physical reactions or bodily functioning. - Temperature, brain activity, skin conductance, heart rate, muscle activity, pupil diameter, scans (MRI, fMRI) - ex: PT stress test

Unobtrusive measure

A measure that is made of behaviors or situations without disturbing the naturally occurring behavior or situation in order to reduce changes that might occur if there was awareness of measurement - condition of observation

Observational measure

A measure that is rated by observers and sometimes made without the awareness of the person performing the behavior (measure something while you observe it) - can be obtrusive or unobstrusive

Confidentiality

A participant's responses are kept private although the researcher may be able to link the participant with his or her responses - anonymity isn't necessary

Pilot study:

A preliminary study with a small sample to test measures and procedures - a good way to test the measures and work out any kinks prior to conducting the full study.

Convergent validity

Positive relationship between two scales measuring the same or similar constructs - we expect the new scale to be positively correlated to the existing measures or to accepted measures (+1, -1)

Peer review

Process in which scholarly works are evaluated by other experts in the field.

Conference Papers or Posters

Professional conferences provide a forum for researchers to present their scholarly work (both primary and secondary) in the form of a paper or poster presentation - these types of scholarly work often represent cutting-edge research.

Blind observer

Observer who is not informed of the research hypotheses in order to reduce observer bias

Criterion validity

Positive correlation between scale scores and a behavioral measure.

Concurrent validity:

Positive correlation between scale scores and a current behavior that is related to the construct assessed by the scale

Predictive validity:

Positive relationship between scale scores and a future behavior that is related to the construct assessed by the scale

Undergraduate Research

Reading these works can give you some great ideas and inspiration, but be judicious in using them as sources for a research study.

Primary research article (or empirical journal article)

Report of the method and results of an original research study that is published in an academic journal.

Procedure and Design

The Procedure section describes the steps the authors followed in the study. A general rule of thumb is that the Procedure section should contain enough information that the reader (you) could replicate the study. - The procedures will help you identify the exact research design (or designs) utilized in the study

Operational definition

The explicit explanation of a variable in terms of how it is measured or manipulated. - · We use operational definitions to explicitly define abstract constructs for a specific study

Observer bias

The observers pay closer attention to behaviors that support their expectations or interpret behaviors in ways that support their expectations or lose their focus on the target behavior - Having a blind observer can greatly reduce observer bias

Introduction

The purpose of the introduction is to provide the rationale for the study. Reading the introduction will give you insight into the authors' thinking about the topic and the reason they conducted the study. It will also give you a sense of some of the past research in this area.

Scale score

The score that is computed from items assessing a particular construct, most commonly a sum or average of the numbers representing responses to individual items in the document

Response format

The type of response, either participant generated or choice from among listed options, required by items on a questionnaire

· Construct validity:

Whether a measure mirrors the characteristics of a hypothetical construct; can be assessed in multiple ways.

Simple random sampling

a type of probability sampling in which every single member of the population has an equal chance of being selected for the sample

Cluster sampling

a type of probability sampling in which groups, or clusters, are randomly selected instead of individuals. A cluster might be defined as a neighborhood a school, or a class within a school

Correlational and experimental research designs

examine relationships among variables, with experimental research testing a causal relationship

Experimental research

examines the relationship between two or more variables and, if properly conducted, can demonstrate causation

Correlational research

examines the relationship between two or more variables but does not test causality. A correlational study tests the degree to which behaviors, events, and feelings co-occur with other behaviors, events, and feelings.

Descriptive research

examines the who, what, when, where, and how, but does not examine relationships among the who, what, when, where, and how. - Descriptive research can be exploratory in nature. It is often used to examine phenomena in more depth or to examine an area of research that is either new or needs updating.

Method

explains the method used to test the hypotheses or to help answer the research questions. The Method section will include information about the participants (or animal subjects), the measures or materials used in the study, and the procedures of the study. Reading the Method section is the best way to identify the research design of the study

the greater the internal validity, the harder it is to achieve

external validity

To estimate the minimum sample size required for a descriptive study using probability sampling, you must know

how large the population is as well as identify both the confidence interval and confidence level

Deception

if you intend to mislead or downright lie to your participants - During the informed consent process, you do not need to disclose all the details of the study, such as what you expect to find or that some participants will be exposed to different conditions

A variable

is something that varies in that it has at least two possible values. Gender is a variable because the categories can be male, female, or transgender.

Secondary Sources in Academic Journals

literature reviews and meta-analyses. - maybe commentaries

Nominal scales

o Categories o Numbers have no numerical value o Only have identity o Gender, ethnicity, marital status § 1 = single, 2 = married, 3 = divorced

· Questionnaires

o Participants respond to a question or questions regarding a particular topic, variable, trait, attitude, and so on. Each question, or item, on a questionnaire consists of a stem and a response o One questions or multiple questions can represent a construct o Open ended or close ended responses

Interval scales

o ratings that have both order and equal intervals between values on the scale. o No true zero o Ex: Likert scale § commonly used type of interval scale response in which items are rated on a range of numbers (usually between 5 and 7 response options) that are assumed to have equal intervals.

Shape of a Primary Research Article

often described as an hourglass in that a primary research article is organized so that it starts broad, moves to the more narrow or specific, and then gets broad again.

Random selection without replacement

once a member of the population is selected, that member is removed from the pool and cannot be selected into the sample again

Interviews

one-on-one conversations directed by a researcher that can take place in person, over the phone, or via e-mail

Abstracts

one-paragraph summaries of scholarly works. - They are not complete works, but rather part of a conference presentation or research article

Abstract

one-paragraph summary of the entire article. - "Abstract," "Overview," or "Summary"

Primary Sources in Academic Journals

original research study/ empirical journal article

Social desirability bias:

participants may respond based on how they want to be perceived rather than how they actually think or behave

snowball sampling

participants recruit others into the sample - typically used to seek out members of a specific population who are difficult to find or who might be distrustful of a researcher, such as sex workers, undocumented workers, the homeless, etc

informed consent

potential participants are informed of the topic, procedure, risks, and benefits of participation prior to consenting to participate - once you start manipulating situations, interacting with participants, making audio or visual recordings of participants, or asking them to complete questionnaires, informed consent is almost always necessary. - Participants should also know that they can decline or withdraw from the study at any time without negative repercussions - researchers must determine how informed the consent must be in order for the study to both be ethical and yield meaningful results

Stratified random sampling

probability sampling that results in the sample representing key subpopulations based on characteristics such as age, gender, and ethnicity. With stratified random sampling, the sample has the same proportion of these groups as are in the population

Beneficence

promoting the well being of society or individuals

Descriptive research

seeks to answer these types of "who, what, where, when, and how" questions - A descriptive research study is the most basic type of study, and serves as an important first step prior to predicting or explaining events, attitudes, or behaviors. ** research that helps describe your study

skills vs critical thinking

skills are something you have/gain while critical thinking is something that you do - researchers think by doing

quasi-experimental design

some, but not all, of the requirements for an experiment are met.

ethical standards

specific rules/obligations that promote the ethical principles - ex: informed consent, debriefing, incentives, etc.

literature review

summarizes the findings of many primary research articles but does not report the method or results of an original study

Covert observation

the observers do not reveal that they are observing the participants - designed to capture the participants' natural and spontaneous reactions to situations. They can be especially important when observing behaviors that are prone to the social desirability bias

Overt observation

the observers do reveal that they are observing the participants

primary source

the one closest to the original source of information - ex: a research study

trend

the pattern of change in prevalence over time - measure prevalence by looking at trends - Descriptive research also helps us understand trends

Results

the results of analyses used to test the hypotheses or help answer the research questions. - You might also examine the means, percentages, or other numbers associated with the statistically significant result so that you have some understanding of how the authors tested the hypotheses.

The Scientific Approach and Knowledge

the scientific approach can be used to build our knowledge base, improve or refute theories, and develop new ideas. - may serve as a foundation for future research that has real-life applications.