Research methods test

Reverse Coding

Means that the value assigned for a response is treated oppositely from the other items.

Single-item scale

Only one question is used to measure the concept

Cronbach's alpha

Reliability test to determine the internal consistency or average correlation of items in a survey instrument

"ratio scale" or "interval scale"

Should report mean and standard deviation

Proportional Stratified Sample

The number of sampling units drawn from each stratum is in proportion to the population size of that stratum.

Latent variable

Variables that are not directly observable but are measurable through indirect indicators, such as verbal expression or overt behavior.

Alternative hypothesis (Ha)

We state what we think is wrong about the null hypothesis. The sign is opposite to the sign in the null hypothesis ( ≠ , >, < )

Structured questionnaire question

imposes a limit on the number of allowable responses

Evidence of causality

- Temporal sequence - Concomitant variation - Nonspurious association

Self-Selection Bias

A bias that occurs because people who feel strongly about a subject are more likely to respond to survey questions than people who feel indifferent about it.

Scales

A device providing a range of values that correspond to different characteristics or amounts of a characteristic exhibited in observing a concept.

The Sampling Frame (working population)

A group of people or entities that are drawn from the entire population.

Sensitivity

A measurement instrument's ability to accurately measure variability in stimuli or responses. Generally increased by adding more response points or adding scale items.

Snowball Sampling

A sampling procedure in which initial respondents are selected by probability methods and additional respondents are obtained from information provided by the initial respondents.

Probability Sampling

A sampling technique in which every member of the population has a known, nonzero probability of selection.

Nonprobability Sampling

A sampling technique in which units of the sample are selected on the basis of personal judgment or convenience. The probability of any particular member of the population being chosen is unknown.

Summated Scale

A scale created by simply summing (adding together) the response to each item making up the composite measure.

Multiple-item scale

A set of multiple items which reduces potentially idiosyncratic influences of any single item. Aggregating across items increases the precision of measures.

Sampling Unit/Planned sample

A single element or group of elements subject to selection in the sample. Computer programs can provide a random selection

Systematic Sampling

A starting point is selected by a random process and then every n th number on the list is selected.

Kaiser-Meyer-Olkin (KMO)

A statistic used for testing Exploratory Factor Analysis (EFA) validity by indicating the proportion of variance in your variables that might be caused by underlying factors. High values (close to 1.0) generally indicate that a factor analysis may be useful with your data.

Exploratory Factor Analysis (EFA)

A statistical method used to uncover the underlying structure of a relatively large set of variables. Reduces variables into a smaller set (aka factors). Establishes underlying dimensions between measured variables and latent constructs, thereby allowing the formation and refinement of theory. Provides construct validity evidence of self-reporting scales.

Correlation analysis

A statistical technique to estimate the relationships among variables

Regression Analysis

A statistical technique used to investigate the relationship between the output variable and one or more input variables.

Sample

A subset, or some part, of a larger population.

Hypothesis

A testable prediction, often implied by a theory. Formal statement of an unproven proposition that is empirically testable.

Ladder of Abstraction

Abstract Level (General, Cannot be measured easily) - exists only as an idea or a quality apart from an object. Empirical Level (Specific, Can be measured) - knowledge that is verifiable by experience or observation.

Cluster Sampling

An economically efficient sampling technique in which the primary sampling unit is not the individual element in the population but a large cluster of elements. Clusters are selected randomly. Then every single entity within each cluster is selected as the sample

Systematic Error (Nonsampling Errors)

An error as a result of nonsampling factors, primarily the nature of a study's design and the correctness of execution.

Administrative Error

An error caused by the improper administration or execution of the research task. Data-processing error Sample selection error Interviewer error Interviewer cheating

Unbalanced rating scale

An even-numbered scale that has no neutral point.

Judgment (Purposive) Sampling

An experienced individual selects the sample based on personal judgment about some appropriate characteristic of the sample member.

Census

An investigation of all the individual elements that make up a population.

Balanced rating scale

An odd numbered scale with an equal number of positive and negative categories; there is a neutral point in the center of the scale

Population (universe)

Any complete group of entities that share some common set of characteristics.

Nominal scales

Assigns a value to an object for identification or classification purposes. It is a qualitative scale: we cannot tell which value is higher than others (Gender, Race)

Simple Random Sampling

Assures each element in the population of an equal chance of being included in the sample.

APA 6

Author, A. A., Author, B. B., & Author, C. C. (Year). Title of article. Title of Journal, Vol, Issue, pages.

Social Desirability Bias

Bias in responses caused by respondents' desire to gain prestige or appear in a different social role. Two aspects: Self-deception (enhance their self-esteem) Other-deception (make them look good to others).

Interval scales

Capture information about differences in quantities of a concept like Temperature. They do not have a true zero point.

Secondary Data

Data gathered and recorded by someone else prior to and for a purpose other than the current project.

Primary Data

Data observed or collected directly from first-hand experience Researchers are collecting information for the specific purposes of their study

Proposition

Deals with pure concepts for which no laboratory test is currently available. Statements explaining the logical linkage among certain concepts by asserting a universal connection between concepts.

Anonymity

Does not give his/her identify away

Between-Subjects Designs

Each subject receives only one treatment combination. Usually advantageous although they are usually more costly. Validity is usually higher.

Quota Sampling

Ensures that various subgroups of a population will be represented on pertinent characteristics to the exact extent that the investigator desires.

Empirical

Examining a research hypothesis against reality using data

Ratio Scales

Highest form of measurement. Zero represent the complete absence of value.

Within-Subjects Designs

Involves repeated measures because with each treatment of the same subject is measured.

Multistage Area Sampling

Involves using a combination of two or more probability sampling techniques.

Convenience Sampling

Obtaining those people or units that are most conveniently available.

Confidentiality

Personal information (e.g., name) is NOT shared.

Causal relationship

Positive (+) relationship: If A increases, B will increase Negative (-) relationship: If A increases, B will decrease

One-tailed hypothesis (or directional hypothesis)

Prediction will go in only one direction (e.g., either positive or negative); We assume that it is impossible for the opposite to occur.

Two-tailed hypothesis (or non-directional hypothesis)

Prediction will go in two direction. (e.g., both positive and negative can happen).

Solutions to avoid social desirability bias

Provide clear information on the purpose and rationale of the research. Ensure confidentiality. Allows anonymity. Ask the respondents to answer each question as honestly as they can. Ensure that there are no right or wrong answers.

Scientific Method

Question - background research - hypothesis - experiment - comclusion

Ordinal scales

Ranking scales allowing things to be arranged based on how much of some concept they possess.

Level of significance

Refers to a criterion of judgment upon which a decision is made regarding the value stated in a null hypothesis. The criterion or level of significance is typically set at 5% (0.05)

Null hypothesis (H0)

Reflects that there will be no observed effect for our experiment. The only reason we are testing it is because we think it is wrong. always includes the equal sign ( = , ≤, ≥ )

Three Criteria for Good Measurement

Reliability, Sensitivity, and Validity

Internal Consistency

Represents a measure's homogeneity or the extent to which each indicator of a concept converges on some common meaning. Measured by correlating scores on subsets of items making up a scale or using a Coefficient alpha (α)

Causal Research

Research conducted to identify cause and effect relationships (inferences).

Cross-sectional study

Researchers compare different population groups at a single point in time.

Unconscious Misrepresentation

Respondent is consciously trying to be truthful but the question format, the question content, or some other stimulus that affects their response to a question.

Stratified Sampling

Simple random subsamples that are more or less equal on some characteristic are drawn from within each stratum of the population.

Levene's Test for Equality of Variances

Test to support the t-test assumption that "the variability of each group is approximately equal".

Validity

The accuracy of a measure or the extent to which a score truthfully represents a concept.

External Validity

The accuracy with which experimental results can be generalized beyond the experimental subjects.

Behavioral Component

The behavioral expectations (expected future actions) of an individual toward an attitudinal object.

Reliability

The degree to which measures are free from random error and therefore yield consistent results. An indicator of a measure's internal consistency.

Random Sampling Error

The difference between the sample result and the result of a census conducted using identical procedures. A statistical fluctuation that occurs because of chance variations in the elements selected for a sample.

Reverse causation

The direction of causality between two factors may be opposite from what we expect...makes it difficult for researchers to infer the direction of causality

Moderating effect

The effect of the independent variable (cause) on the dependent variable (effect) is contingent on the third factor which can either enhance or suppress the effect.

Internal Validity

The extent that an experimental variable is truly responsible for any variance in the dependent variable.

Main Effect

The influence of a single independent variable on a dependent variable.

Deductive reasoning

The logical process of deriving a conclusion, a.k.a. "top-down" approach because the researcher starts at the top with a very broad spectrum of information and they work their way down to a specific conclusion. Normally used in quantitative research. It is usually used for theory testing.

Inductive reasoning

The logical process of establishing a general proposition on the basis of observation of particular facts. This is sometimes called a "bottom up" approach. Normally used in scientific and qualitative research, It is usually used for theory building.

Result that satisfies the Levene's Test for Equality of Variances

The p-value of the Levene's Test should be > 0.05

Result that satisfies the Test of Homogeneity of Variances

The p-value should be > 0.05

P-value

The probability of obtaining a sample outcome, given that the value stated in the null hypothesis is true.

Response rate

The result of dividing the number of people who were interviewed by the total number of people in the sample who were eligible to participate and should have been interviewed.

Disproportional Stratified Sample

The sample size for each stratum is allocated according to analytical considerations.

Nonresponse Error

The statistical differences between a survey that includes only those who responded and a perfect survey that would also include those who failed to respond.

Qualitative Research

To gain an understanding of underlying reasons and motivations and provide insights into the setting of a problem, generating ideas and/or hypotheses for later quantitative research

Quantitative Research

To quantify data and generalize results from a sample to the population of interest. To measure the incidence of various views and opinions in a chosen sample Sometimes followed by qualitative research which is used to explore some findings further

Mediating effect

X does not influence Y directly. But it can influence Y indirectly through a third variable called a "Mediator"

The test statistic (t-statistics, F-statistics)

a mathematical formula that allows researchers to determine the likelihood of obtaining sample outcomes if the null hypothesis were true.

Hypothesis testing / significance testing

a systematic way to test claims or ideas about a group or population, using data measured in a sample.

Nonspurious association

an absence of alternative plausible explanations.

The one way analysis of variance (ANOVA)

an inferential statistical test that allows you to test if any of several means are different from each other. Used when you want to compare means among 3 or more groups. (Use "Turkey" if Equal variances Assumed / Dunnett's C if Equal variances Not Assumed)

Empirical testing

based on experimentation or observation (evidence). Such research is conducted to test a hypothesis

Unstructured question

does not restrict respondents' answers

Quasi-experimental design

has all the same elements as pure experiment; however, subjects are not randomly assigned to conditions. Subjects are selected based on the values of the independent variable (e.g., age, gender), rather than having the experimenter assign values of the independent variable to subjects

Experiment

is "an orderly procedure carried out with the goal of verifying, falsifying, or establishing the validity of a hypothesis".

correlation coefficient

is a number that indicates the strength and direction of a relationship between variables

Full mediation

occurs when the direct relationship between the independent variable and the dependent variable disappear when the mediator is included.

Partial mediation

occurs when the direct relationship between the independent variable and the dependent variable is still strong even when the mediator is included.

The p-test result where we reject the H0

p<0.05

The p-test result where we fail to reject the H0

p>0.05

"nominal scale" or "ordinal scale"

reports frequency and percentage

Temporal sequence

the appropriate causal order of events, ie. cause comes before event.

Attitudes

the degree to which a person has a favorable or unfavorable evaluation or appraisal of something.

Concomitant variation

two phenomena vary together.

Independent Samples t-test

used to see if two means are different from each other when the two samples, that the means are based on, were taken from different individuals who have not been matched