Intro to Social Research - Exam 2

Measurement Theory

(aka) true score theory nbased on the idea that an empirical measure of a concept reflects three components: n(1) the (t)rue or ideal construct or an absolutely perfect measure of it; n(2) (s)ystematic errors or biases (non-random); n(3) (r)andom errors, unavoidable error (chance errors). nMathematical Expression: X=T+S+R, where X is the empirical indicator or observation.

Tensions between relability and validity

*Reliable but not valid (grades and intellectual ability) *Valid but not reliable (# of sexual partners) *Reliability is necessary for validity (ACT and college performance)

False criteria

1) Complete causation 2) Exceptional Cases 3) Majority of cases

Causality and its criteria

1. A theoretical or substantive justification for the relationship of interest 2. Correlation: the variables must be correlated or associated such that changes in one variable are associated with changes in the other 3. Time or Temporal Order (the cause takes place before the effect) 4. Non-spurious (the relationship between variables is not spurious)

units of analysis

1. Individuals 2. Groups 3. Organizations 4. Social Interactions/Behaviors 5. Social artifacts

Sufficient causes

A condition that, if present, guarantees the effect in question.

Non-spuriousness

A criterion for establishing a causal relation between two variables; when a relationship between two variables is not caused by variation in a third variable.

internal consistency

A measure of reliability; the degree to which a test yields similar scores across its different parts, such as on odd versus even items.

split-half method

A method for assessing internal consistency by checking the results of one-half of a set of scaled items against the results from the other half.

Likert scaling

A type of composite measure developed by Rensis Likert in an attempt to improve the levels of measurement in social research through the use of standardized response categories in survey questionnaires to determine the relative intensity of different items. *1 Strongly disagree *2 Disagree *3 Neither disagree nor agree *4 Agree *5 Strongly agree

In order to achieve broad coverage of various dimensions of a concept, researchers need to make:

A.multiple observations

Descriptive Research

Descriptive studies answer questions of what, where, when, and how

Explanatory Research

Explanatory studies answer questions of why

_____ is the degree to which a measure covers the range of meanings included within a concept.

Face validity

A necessary condition represents a condition that, if it is present, will pretty much guarantee the effect in question.

False

A nominal measure can have only two categories.

False

A set of questionnaire items that Guttman scaled on one data set will usually Guttman scale on another data set.

False

A split-half reliability test taps the idea of the general stability of the instrument over time.

False

Always include at least one item in an index on which all respondents give the same answer.

False

An index is constructed by assigning scores to patterns of attributes.

False

Dr. Jordan chronicles people's attitudes toward an upcoming property tax levy. Jordan isdoing explanatory research.

False

If a measure is reliable, it must also be valid.

False

If three people (Smith, Jones, and Edwards) are rank-ordered with respect to the possession of an attribute on an ordinal variable, then we can conclude that the distance between Smith and Jones on that variable is equal to the distance between Jones and Edwards.

False

If two items in an index are perfectly correlated, both of them should be eliminated from the index.

False

If two variables are correlated with each other, there must be a causal relationship between them.

False

It is impossible to have several indicators of only one concept.

False

Item analysis allows you to examine the extent to which the composite index is related to other items in the questionnaire that are not part of the index.

False

Longitudinal studies may be either trend or cohort studies, but not panel studies.

False

Numbers assigned to ranks on an ordinal scale can legitimately be added, multiplied, subtracted, and divided.

False

Operationalization is the process of clarifying what is meant by the concepts being used in a study.

False

Precise measurement is more important than accurate measurement.

False

Precision and accuracy are synonyms.

False

Research designs are descriptive, explanatory, or exploratory-never more than one.

False

Researchers ignore previous work on a topic so that their research can be original.

False

The independent (effect) variable must occur later in time than the dependent (cause) variable.

False

Guttman scaling

Guttman scaling: a type of composite measure used to summarize several discrete observations and to represent some more general variable.

Social researchers tend to choose _____ as their units of analysis.

Individuals

Levels of Measurement

Nominal Ordinal Interval Ratio

Cross Sectional Research

Observations of a sample, or cross section, of a population or phenomenon that are made at one point in time

Longitudinal research

Observations of the same phenomenon over an extended period. 1) Trend Study 2) Cohort study 3) Panel Study

Ecological Fallacy

Occurs when a researcher gathers data at a higher or an aggregated unit of analysis but wants to make a statement about a lower or dis-aggregated unit

Exploratory Research

Research conducted to gather more information about a problem or to make a tentative hypothesis more specific

Reliability

Test-retest method Split-half-method Internal consistencey Using established measures Research-worker relability

Typologies

The classification of observations in terms of their attributes on two or more variables (e.g., political ideology, religious affiliation, cohabitations and so on so forth).

What do social researchers mean when they say there is a causal relationship between education and racial tolerance?

There is a statistical correlation between the two variables. A person's educational level occurred before their current level of tolerance. There is no third variable that can explain away the observed correlation.

Conceptualization

To conceptualize is to make fuzzy and imprecise notions (the mental processes) more specific and precise; an agreement about what terms mean.

A major shortcoming of exploratory studies is that they seldom provide satisfactory answers to research questions.

True

A particular variable can usually be measured in several distinct ways using different sources of information and various observation techniques.

True

A research proposal describes what you intend to accomplish and how.

True

Being at least 18 years of age is a necessary cause for voting in the United States.

True

Changing definitions almost inevitably results in different descriptive conclusions.

True

Conceptualization and operationalization are processes that continue throughout the research process.

True

Descriptive research answers the question "What's so?" and explanatory research answers the question "Why?"

True

Explanatory research depends on the notion of cause and effect.

True

For an index or scale to be considered unidimensional, its component items should be indicators of only one dimension.

True

Panel attrition is comparable to experimental mortality.

True

Predictive validity is another term for criterion-related validity.

True

The ecological fallacy refers to drawing conclusions about individuals based solely on the observation of groups.

True

The nomothetic model of explanation is probabilistic in its approach to causation.

True

The purpose of multivariate analysis in index construction is to discover the simultaneous interaction of the items to determine whether they are all appropriate for inclusion in the same index.

True

The result of combining several indicators of a variable is a composite measurement of the variable.

True

Thurstone scales use judges in their construction.

True

Typologies are nominal composite measures.

True

Validity refers to the link between the operational and conceptual definitions.

True

When one's research purposes aren't clear, it is advisable to choose the highest level of measurement possible.

True

Necessary causes

a condition that must be present for the effect to follow

Validity

a measure that accurately reflects the concept it is intended to measure. 1) Face validity 2) Criterion-related validity 3) Construct validity 4) Content validity

Bogardus Social Distance Scale (1925)

a measurement technique for determining the willingness of people to participate in social relations - of varying degrees of closeness - with other kinds of people. It is an especially efficient technique in that several discrete answers may be summarized without losing any of the original details of the data.

Test-retest method

a method of calculating reliability by repeating the same measure at two or more points in time

Semantic differential

a questionnaire format in which the respondent is asked to rate something in terms of two, opposite adjectives, using qualifiers such as "very," "somewhat," "neither," "somewhat," and "very," to bridge the distance between the two opposites.

Scale

a type of composite measure composed of several items that have a logical or empirical structure among them. Like index construction, scaling creates an ordinal measure of a variable expressed as a numerical score scales are common in situations where a researcher wants to measure the intensity, hardness, or potency of a variable.

Index

a type of composite measure that summarizes and rank-orders several specific observations (indicators) and represents some more general dimension. It is a composite measure in which each question (or item) measures a small piece of knowledge, and all the questions scored correct or incorrect are totaled to produce a single measure various components or subparts of a construct are each measured, then combined into one measure

Thurstone scales

a type of composite measure, constructed in accord with the weights assigned by "judges" to various indicators of some variables. Disadvantages: 1) Time consuming 2) Expensive

construct validity

agreement between a theoretical concept (construct) and a specific measuring device or procedure

Item analysis

an assessment of whether each of the items included in a composite measure makes an independent contribution or merely duplicates the contribution of other items in the measure.

Indicators

an observation that we choose to consider as a reflection of a concept we wish to study

Dimensions

as complete as possible

Nominal

assigned to a term with consensus

Which of the following are common characteristics shared by both indexes and scales?

both are ordinal measures both rank-order units in terms of specific variables

Why we need to consrtuct index

can scale variables

_____ refer to mental images

conceptions

In social research, the process of coming to an agreement about what terms mean is:

conceptualization

A _____ is an empirical relationship between two variables such that changes in one are associated with changes in the other.

correlation

Which of these are among the purposes of research?

exploration, description, and explanation

Give examples of nominal measures?

gender religious affiliation political party affiliation birthplace

Operational

how a concept is measured

Research-worker reliability

inter-rater reliability

A level of measurement describing a variable whose attributes are rank-ordered and have equal distances between adjacent attributes are _____ measures.

interval

Scientific inquiry comes down to

making observations and interpreting what you have observed

Multivariate relationships among items

many items at a time

A measure of religiosity that is extremely consistent, but is actually targeting the respondent's political affiliation could be considered:

neither valid nor reliable

Reductionism

occurs when a researcher observes a lower or dis-aggregated unit of analysis but makes statements about the operations of higher or aggregated units.

Face validity

on the face of it, the indicator really measures the construct

Which of the following represent steps in the construction of an index?

selecting possible items examining empirical relationships scoring the index

Which of these represents a questionnaire format in which the respondent is asked to rate something in terms of two opposites?

semantic differential

content validity

the degree to which a measure covers the range of meanings included within a concept

criterion-related validity

the degree to which a measure relates to some external criterion

Interval

the first two plus the amount of distance between categories can be specified and equal (e.g., test scores, IQ scores).

Ratio

the only difference between an interval and a ratio measure is that the later has a "true zero" (e.g., true zero exists for such variables as income and marriage rate).

Operationalization

the process of assigning a precise method for measuring a term being examined for use in a particular study

External validation

the process of testing the validity of a measure, such as an index or score, by examining its relationship to other presumed indicators of the same variable

Range of variation

the wider the better

Bivariate relationships between items

two items at a time

Using established measures

use measures that have proved their reliability in previous research The Revised Conflict Tactics Scales Center for Epidemiologic Studies Depression Scale (Revised: CESD-R)

Most social scientists would not accept the conceptualization of prejudice as foot size because such a measurement lacks:

validity

Nominal

variables whose attributes have only the characteristics of exhaustiveness and mutual exclusiveness are nominal measures (e.g., gender, religious preference, political party affiliation, college major, race, and sexual preference).

Ordinal

variables whose attributes may be logically rank-ordered are ordinal measures (e.g., social class, educational attainment, the level of importance or satisfaction, and how often you dated someone).