Intro to Social Research - Exam 2
Measurement Theory
(aka) true score theory nbased on the idea that an empirical measure of a concept reflects three components: n(1) the (t)rue or ideal construct or an absolutely perfect measure of it; n(2) (s)ystematic errors or biases (non-random); n(3) (r)andom errors, unavoidable error (chance errors). nMathematical Expression: X=T+S+R, where X is the empirical indicator or observation.
Tensions between relability and validity
*Reliable but not valid (grades and intellectual ability) *Valid but not reliable (# of sexual partners) *Reliability is necessary for validity (ACT and college performance)
False criteria
1) Complete causation 2) Exceptional Cases 3) Majority of cases
Causality and its criteria
1. A theoretical or substantive justification for the relationship of interest 2. Correlation: the variables must be correlated or associated such that changes in one variable are associated with changes in the other 3. Time or Temporal Order (the cause takes place before the effect) 4. Non-spurious (the relationship between variables is not spurious)
units of analysis
1. Individuals 2. Groups 3. Organizations 4. Social Interactions/Behaviors 5. Social artifacts
Sufficient causes
A condition that, if present, guarantees the effect in question.
Non-spuriousness
A criterion for establishing a causal relation between two variables; when a relationship between two variables is not caused by variation in a third variable.
internal consistency
A measure of reliability; the degree to which a test yields similar scores across its different parts, such as on odd versus even items.
split-half method
A method for assessing internal consistency by checking the results of one-half of a set of scaled items against the results from the other half.
Likert scaling
A type of composite measure developed by Rensis Likert in an attempt to improve the levels of measurement in social research through the use of standardized response categories in survey questionnaires to determine the relative intensity of different items. *1 Strongly disagree *2 Disagree *3 Neither disagree nor agree *4 Agree *5 Strongly agree
In order to achieve broad coverage of various dimensions of a concept, researchers need to make:
A.multiple observations
Descriptive Research
Descriptive studies answer questions of what, where, when, and how
Explanatory Research
Explanatory studies answer questions of why
_____ is the degree to which a measure covers the range of meanings included within a concept.
Face validity
A necessary condition represents a condition that, if it is present, will pretty much guarantee the effect in question.
False
A nominal measure can have only two categories.
False
A set of questionnaire items that Guttman scaled on one data set will usually Guttman scale on another data set.
False
A split-half reliability test taps the idea of the general stability of the instrument over time.
False
Always include at least one item in an index on which all respondents give the same answer.
False
An index is constructed by assigning scores to patterns of attributes.
False
Dr. Jordan chronicles people's attitudes toward an upcoming property tax levy. Jordan isdoing explanatory research.
False
If a measure is reliable, it must also be valid.
False
If three people (Smith, Jones, and Edwards) are rank-ordered with respect to the possession of an attribute on an ordinal variable, then we can conclude that the distance between Smith and Jones on that variable is equal to the distance between Jones and Edwards.
False
If two items in an index are perfectly correlated, both of them should be eliminated from the index.
False
If two variables are correlated with each other, there must be a causal relationship between them.
False
It is impossible to have several indicators of only one concept.
False
Item analysis allows you to examine the extent to which the composite index is related to other items in the questionnaire that are not part of the index.
False
Longitudinal studies may be either trend or cohort studies, but not panel studies.
False
Numbers assigned to ranks on an ordinal scale can legitimately be added, multiplied, subtracted, and divided.
False
Operationalization is the process of clarifying what is meant by the concepts being used in a study.
False
Precise measurement is more important than accurate measurement.
False
Precision and accuracy are synonyms.
False
Research designs are descriptive, explanatory, or exploratory-never more than one.
False
Researchers ignore previous work on a topic so that their research can be original.
False
The independent (effect) variable must occur later in time than the dependent (cause) variable.
False
Guttman scaling
Guttman scaling: a type of composite measure used to summarize several discrete observations and to represent some more general variable.
Social researchers tend to choose _____ as their units of analysis.
Individuals
Levels of Measurement
Nominal Ordinal Interval Ratio
Cross Sectional Research
Observations of a sample, or cross section, of a population or phenomenon that are made at one point in time
Longitudinal research
Observations of the same phenomenon over an extended period. 1) Trend Study 2) Cohort study 3) Panel Study
Ecological Fallacy
Occurs when a researcher gathers data at a higher or an aggregated unit of analysis but wants to make a statement about a lower or dis-aggregated unit
Exploratory Research
Research conducted to gather more information about a problem or to make a tentative hypothesis more specific
Reliability
Test-retest method Split-half-method Internal consistencey Using established measures Research-worker relability
Typologies
The classification of observations in terms of their attributes on two or more variables (e.g., political ideology, religious affiliation, cohabitations and so on so forth).
What do social researchers mean when they say there is a causal relationship between education and racial tolerance?
There is a statistical correlation between the two variables. A person's educational level occurred before their current level of tolerance. There is no third variable that can explain away the observed correlation.
Conceptualization
To conceptualize is to make fuzzy and imprecise notions (the mental processes) more specific and precise; an agreement about what terms mean.
A major shortcoming of exploratory studies is that they seldom provide satisfactory answers to research questions.
True
A particular variable can usually be measured in several distinct ways using different sources of information and various observation techniques.
True
A research proposal describes what you intend to accomplish and how.
True
Being at least 18 years of age is a necessary cause for voting in the United States.
True
Changing definitions almost inevitably results in different descriptive conclusions.
True
Conceptualization and operationalization are processes that continue throughout the research process.
True
Descriptive research answers the question "What's so?" and explanatory research answers the question "Why?"
True
Explanatory research depends on the notion of cause and effect.
True
For an index or scale to be considered unidimensional, its component items should be indicators of only one dimension.
True
Panel attrition is comparable to experimental mortality.
True
Predictive validity is another term for criterion-related validity.
True
The ecological fallacy refers to drawing conclusions about individuals based solely on the observation of groups.
True
The nomothetic model of explanation is probabilistic in its approach to causation.
True
The purpose of multivariate analysis in index construction is to discover the simultaneous interaction of the items to determine whether they are all appropriate for inclusion in the same index.
True
The result of combining several indicators of a variable is a composite measurement of the variable.
True
Thurstone scales use judges in their construction.
True
Typologies are nominal composite measures.
True
Validity refers to the link between the operational and conceptual definitions.
True
When one's research purposes aren't clear, it is advisable to choose the highest level of measurement possible.
True
Necessary causes
a condition that must be present for the effect to follow
Validity
a measure that accurately reflects the concept it is intended to measure. 1) Face validity 2) Criterion-related validity 3) Construct validity 4) Content validity
Bogardus Social Distance Scale (1925)
a measurement technique for determining the willingness of people to participate in social relations - of varying degrees of closeness - with other kinds of people. It is an especially efficient technique in that several discrete answers may be summarized without losing any of the original details of the data.
Test-retest method
a method of calculating reliability by repeating the same measure at two or more points in time
Semantic differential
a questionnaire format in which the respondent is asked to rate something in terms of two, opposite adjectives, using qualifiers such as "very," "somewhat," "neither," "somewhat," and "very," to bridge the distance between the two opposites.
Scale
a type of composite measure composed of several items that have a logical or empirical structure among them. Like index construction, scaling creates an ordinal measure of a variable expressed as a numerical score scales are common in situations where a researcher wants to measure the intensity, hardness, or potency of a variable.
Index
a type of composite measure that summarizes and rank-orders several specific observations (indicators) and represents some more general dimension. It is a composite measure in which each question (or item) measures a small piece of knowledge, and all the questions scored correct or incorrect are totaled to produce a single measure various components or subparts of a construct are each measured, then combined into one measure
Thurstone scales
a type of composite measure, constructed in accord with the weights assigned by "judges" to various indicators of some variables. Disadvantages: 1) Time consuming 2) Expensive
construct validity
agreement between a theoretical concept (construct) and a specific measuring device or procedure
Item analysis
an assessment of whether each of the items included in a composite measure makes an independent contribution or merely duplicates the contribution of other items in the measure.
Indicators
an observation that we choose to consider as a reflection of a concept we wish to study
Dimensions
as complete as possible
Nominal
assigned to a term with consensus
Which of the following are common characteristics shared by both indexes and scales?
both are ordinal measures both rank-order units in terms of specific variables
Why we need to consrtuct index
can scale variables
_____ refer to mental images
conceptions
In social research, the process of coming to an agreement about what terms mean is:
conceptualization
A _____ is an empirical relationship between two variables such that changes in one are associated with changes in the other.
correlation
Which of these are among the purposes of research?
exploration, description, and explanation
Give examples of nominal measures?
gender religious affiliation political party affiliation birthplace
Operational
how a concept is measured
Research-worker reliability
inter-rater reliability
A level of measurement describing a variable whose attributes are rank-ordered and have equal distances between adjacent attributes are _____ measures.
interval
Scientific inquiry comes down to
making observations and interpreting what you have observed
Multivariate relationships among items
many items at a time
A measure of religiosity that is extremely consistent, but is actually targeting the respondent's political affiliation could be considered:
neither valid nor reliable
Reductionism
occurs when a researcher observes a lower or dis-aggregated unit of analysis but makes statements about the operations of higher or aggregated units.
Face validity
on the face of it, the indicator really measures the construct
Which of the following represent steps in the construction of an index?
selecting possible items examining empirical relationships scoring the index
Which of these represents a questionnaire format in which the respondent is asked to rate something in terms of two opposites?
semantic differential
content validity
the degree to which a measure covers the range of meanings included within a concept
criterion-related validity
the degree to which a measure relates to some external criterion
Interval
the first two plus the amount of distance between categories can be specified and equal (e.g., test scores, IQ scores).
Ratio
the only difference between an interval and a ratio measure is that the later has a "true zero" (e.g., true zero exists for such variables as income and marriage rate).
Operationalization
the process of assigning a precise method for measuring a term being examined for use in a particular study
External validation
the process of testing the validity of a measure, such as an index or score, by examining its relationship to other presumed indicators of the same variable
Range of variation
the wider the better
Bivariate relationships between items
two items at a time
Using established measures
use measures that have proved their reliability in previous research The Revised Conflict Tactics Scales Center for Epidemiologic Studies Depression Scale (Revised: CESD-R)
Most social scientists would not accept the conceptualization of prejudice as foot size because such a measurement lacks:
validity
Nominal
variables whose attributes have only the characteristics of exhaustiveness and mutual exclusiveness are nominal measures (e.g., gender, religious preference, political party affiliation, college major, race, and sexual preference).
Ordinal
variables whose attributes may be logically rank-ordered are ordinal measures (e.g., social class, educational attainment, the level of importance or satisfaction, and how often you dated someone).
