Research methods test
Reverse Coding
Means that the value assigned for a response is treated oppositely from the other items.
Single-item scale
Only one question is used to measure the concept
Cronbach's alpha
Reliability test to determine the internal consistency or average correlation of items in a survey instrument
"ratio scale" or "interval scale"
Should report mean and standard deviation
Proportional Stratified Sample
The number of sampling units drawn from each stratum is in proportion to the population size of that stratum.
Latent variable
Variables that are not directly observable but are measurable through indirect indicators, such as verbal expression or overt behavior.
Alternative hypothesis (Ha)
We state what we think is wrong about the null hypothesis. The sign is opposite to the sign in the null hypothesis ( ≠ , >, < )
Structured questionnaire question
imposes a limit on the number of allowable responses
Evidence of causality
- Temporal sequence - Concomitant variation - Nonspurious association
Self-Selection Bias
A bias that occurs because people who feel strongly about a subject are more likely to respond to survey questions than people who feel indifferent about it.
Scales
A device providing a range of values that correspond to different characteristics or amounts of a characteristic exhibited in observing a concept.
The Sampling Frame (working population)
A group of people or entities that are drawn from the entire population.
Sensitivity
A measurement instrument's ability to accurately measure variability in stimuli or responses. Generally increased by adding more response points or adding scale items.
Snowball Sampling
A sampling procedure in which initial respondents are selected by probability methods and additional respondents are obtained from information provided by the initial respondents.
Probability Sampling
A sampling technique in which every member of the population has a known, nonzero probability of selection.
Nonprobability Sampling
A sampling technique in which units of the sample are selected on the basis of personal judgment or convenience. The probability of any particular member of the population being chosen is unknown.
Summated Scale
A scale created by simply summing (adding together) the response to each item making up the composite measure.
Multiple-item scale
A set of multiple items which reduces potentially idiosyncratic influences of any single item. Aggregating across items increases the precision of measures.
Sampling Unit/Planned sample
A single element or group of elements subject to selection in the sample. Computer programs can provide a random selection
Systematic Sampling
A starting point is selected by a random process and then every n th number on the list is selected.
Kaiser-Meyer-Olkin (KMO)
A statistic used for testing Exploratory Factor Analysis (EFA) validity by indicating the proportion of variance in your variables that might be caused by underlying factors. High values (close to 1.0) generally indicate that a factor analysis may be useful with your data.
Exploratory Factor Analysis (EFA)
A statistical method used to uncover the underlying structure of a relatively large set of variables. Reduces variables into a smaller set (aka factors). Establishes underlying dimensions between measured variables and latent constructs, thereby allowing the formation and refinement of theory. Provides construct validity evidence of self-reporting scales.
Correlation analysis
A statistical technique to estimate the relationships among variables
Regression Analysis
A statistical technique used to investigate the relationship between the output variable and one or more input variables.
Sample
A subset, or some part, of a larger population.
Hypothesis
A testable prediction, often implied by a theory. Formal statement of an unproven proposition that is empirically testable.
Ladder of Abstraction
Abstract Level (General, Cannot be measured easily) - exists only as an idea or a quality apart from an object. Empirical Level (Specific, Can be measured) - knowledge that is verifiable by experience or observation.
Cluster Sampling
An economically efficient sampling technique in which the primary sampling unit is not the individual element in the population but a large cluster of elements. Clusters are selected randomly. Then every single entity within each cluster is selected as the sample
Systematic Error (Nonsampling Errors)
An error as a result of nonsampling factors, primarily the nature of a study's design and the correctness of execution.
Administrative Error
An error caused by the improper administration or execution of the research task. Data-processing error Sample selection error Interviewer error Interviewer cheating
Unbalanced rating scale
An even-numbered scale that has no neutral point.
Judgment (Purposive) Sampling
An experienced individual selects the sample based on personal judgment about some appropriate characteristic of the sample member.
Census
An investigation of all the individual elements that make up a population.
Balanced rating scale
An odd numbered scale with an equal number of positive and negative categories; there is a neutral point in the center of the scale
Population (universe)
Any complete group of entities that share some common set of characteristics.
Nominal scales
Assigns a value to an object for identification or classification purposes. It is a qualitative scale: we cannot tell which value is higher than others (Gender, Race)
Simple Random Sampling
Assures each element in the population of an equal chance of being included in the sample.
APA 6
Author, A. A., Author, B. B., & Author, C. C. (Year). Title of article. Title of Journal, Vol, Issue, pages.
Social Desirability Bias
Bias in responses caused by respondents' desire to gain prestige or appear in a different social role. Two aspects: Self-deception (enhance their self-esteem) Other-deception (make them look good to others).
Interval scales
Capture information about differences in quantities of a concept like Temperature. They do not have a true zero point.
Secondary Data
Data gathered and recorded by someone else prior to and for a purpose other than the current project.
Primary Data
Data observed or collected directly from first-hand experience Researchers are collecting information for the specific purposes of their study
Proposition
Deals with pure concepts for which no laboratory test is currently available. Statements explaining the logical linkage among certain concepts by asserting a universal connection between concepts.
Anonymity
Does not give his/her identify away
Between-Subjects Designs
Each subject receives only one treatment combination. Usually advantageous although they are usually more costly. Validity is usually higher.
Quota Sampling
Ensures that various subgroups of a population will be represented on pertinent characteristics to the exact extent that the investigator desires.
Empirical
Examining a research hypothesis against reality using data
Ratio Scales
Highest form of measurement. Zero represent the complete absence of value.
Within-Subjects Designs
Involves repeated measures because with each treatment of the same subject is measured.
Multistage Area Sampling
Involves using a combination of two or more probability sampling techniques.
Convenience Sampling
Obtaining those people or units that are most conveniently available.
Confidentiality
Personal information (e.g., name) is NOT shared.
Causal relationship
Positive (+) relationship: If A increases, B will increase Negative (-) relationship: If A increases, B will decrease
One-tailed hypothesis (or directional hypothesis)
Prediction will go in only one direction (e.g., either positive or negative); We assume that it is impossible for the opposite to occur.
Two-tailed hypothesis (or non-directional hypothesis)
Prediction will go in two direction. (e.g., both positive and negative can happen).
Solutions to avoid social desirability bias
Provide clear information on the purpose and rationale of the research. Ensure confidentiality. Allows anonymity. Ask the respondents to answer each question as honestly as they can. Ensure that there are no right or wrong answers.
Scientific Method
Question - background research - hypothesis - experiment - comclusion
Ordinal scales
Ranking scales allowing things to be arranged based on how much of some concept they possess.
Level of significance
Refers to a criterion of judgment upon which a decision is made regarding the value stated in a null hypothesis. The criterion or level of significance is typically set at 5% (0.05)
Null hypothesis (H0)
Reflects that there will be no observed effect for our experiment. The only reason we are testing it is because we think it is wrong. always includes the equal sign ( = , ≤, ≥ )
Three Criteria for Good Measurement
Reliability, Sensitivity, and Validity
Internal Consistency
Represents a measure's homogeneity or the extent to which each indicator of a concept converges on some common meaning. Measured by correlating scores on subsets of items making up a scale or using a Coefficient alpha (α)
Causal Research
Research conducted to identify cause and effect relationships (inferences).
Cross-sectional study
Researchers compare different population groups at a single point in time.
Unconscious Misrepresentation
Respondent is consciously trying to be truthful but the question format, the question content, or some other stimulus that affects their response to a question.
Stratified Sampling
Simple random subsamples that are more or less equal on some characteristic are drawn from within each stratum of the population.
Levene's Test for Equality of Variances
Test to support the t-test assumption that "the variability of each group is approximately equal".
Validity
The accuracy of a measure or the extent to which a score truthfully represents a concept.
External Validity
The accuracy with which experimental results can be generalized beyond the experimental subjects.
Behavioral Component
The behavioral expectations (expected future actions) of an individual toward an attitudinal object.
Reliability
The degree to which measures are free from random error and therefore yield consistent results. An indicator of a measure's internal consistency.
Random Sampling Error
The difference between the sample result and the result of a census conducted using identical procedures. A statistical fluctuation that occurs because of chance variations in the elements selected for a sample.
Reverse causation
The direction of causality between two factors may be opposite from what we expect...makes it difficult for researchers to infer the direction of causality
Moderating effect
The effect of the independent variable (cause) on the dependent variable (effect) is contingent on the third factor which can either enhance or suppress the effect.
Internal Validity
The extent that an experimental variable is truly responsible for any variance in the dependent variable.
Main Effect
The influence of a single independent variable on a dependent variable.
Deductive reasoning
The logical process of deriving a conclusion, a.k.a. "top-down" approach because the researcher starts at the top with a very broad spectrum of information and they work their way down to a specific conclusion. Normally used in quantitative research. It is usually used for theory testing.
Inductive reasoning
The logical process of establishing a general proposition on the basis of observation of particular facts. This is sometimes called a "bottom up" approach. Normally used in scientific and qualitative research, It is usually used for theory building.
Result that satisfies the Levene's Test for Equality of Variances
The p-value of the Levene's Test should be > 0.05
Result that satisfies the Test of Homogeneity of Variances
The p-value should be > 0.05
P-value
The probability of obtaining a sample outcome, given that the value stated in the null hypothesis is true.
Response rate
The result of dividing the number of people who were interviewed by the total number of people in the sample who were eligible to participate and should have been interviewed.
Disproportional Stratified Sample
The sample size for each stratum is allocated according to analytical considerations.
Nonresponse Error
The statistical differences between a survey that includes only those who responded and a perfect survey that would also include those who failed to respond.
Qualitative Research
To gain an understanding of underlying reasons and motivations and provide insights into the setting of a problem, generating ideas and/or hypotheses for later quantitative research
Quantitative Research
To quantify data and generalize results from a sample to the population of interest. To measure the incidence of various views and opinions in a chosen sample Sometimes followed by qualitative research which is used to explore some findings further
Mediating effect
X does not influence Y directly. But it can influence Y indirectly through a third variable called a "Mediator"
The test statistic (t-statistics, F-statistics)
a mathematical formula that allows researchers to determine the likelihood of obtaining sample outcomes if the null hypothesis were true.
Hypothesis testing / significance testing
a systematic way to test claims or ideas about a group or population, using data measured in a sample.
Nonspurious association
an absence of alternative plausible explanations.
The one way analysis of variance (ANOVA)
an inferential statistical test that allows you to test if any of several means are different from each other. Used when you want to compare means among 3 or more groups. (Use "Turkey" if Equal variances Assumed / Dunnett's C if Equal variances Not Assumed)
Empirical testing
based on experimentation or observation (evidence). Such research is conducted to test a hypothesis
Unstructured question
does not restrict respondents' answers
Quasi-experimental design
has all the same elements as pure experiment; however, subjects are not randomly assigned to conditions. Subjects are selected based on the values of the independent variable (e.g., age, gender), rather than having the experimenter assign values of the independent variable to subjects
Experiment
is "an orderly procedure carried out with the goal of verifying, falsifying, or establishing the validity of a hypothesis".
correlation coefficient
is a number that indicates the strength and direction of a relationship between variables
Full mediation
occurs when the direct relationship between the independent variable and the dependent variable disappear when the mediator is included.
Partial mediation
occurs when the direct relationship between the independent variable and the dependent variable is still strong even when the mediator is included.
The p-test result where we reject the H0
p<0.05
The p-test result where we fail to reject the H0
p>0.05
"nominal scale" or "ordinal scale"
reports frequency and percentage
Temporal sequence
the appropriate causal order of events, ie. cause comes before event.
Attitudes
the degree to which a person has a favorable or unfavorable evaluation or appraisal of something.
Concomitant variation
two phenomena vary together.
Independent Samples t-test
used to see if two means are different from each other when the two samples, that the means are based on, were taken from different individuals who have not been matched
