Quiz 5: Conceptualization, Operationalization, and Measurement

Which of the following statements is FALSE?

Definitions are more problematic for explanatory than descriptive research Definitions can be equally problematic for explanatory and descriptive research. Without a clear and well-agreed definition, we can not properly carry out studies.

Quantitative, nomothetic, and structured techniques tend to be more valid, while qualitative and idiographic methods tend to be more reliable.

False

A nominal level of measurement can be represented as a ratio level of measurement, but a ratio level of measurement cannot be represented as a nominal level of measurement .

False A nominal level of measurement has less information than a ratio level of measurement. Therefore, you can use a nominal level of measurement to represent a ratio level measurement. BUT not vice versa. Here is an example. You can either define educational attainment as the number of years in school, say K- college. This would be a ratio level of measurement, since there is a true "0" in this measure. You can also define educational attainment as a series of categories such as high school graduates, those who finished elementary school but did not complete high school, and those who did not finish elementary school. You could assume that there is no rank order. Educational attainment would become a norminal measure.

Reliability ensures validity.

False The two concepts are different. One does not ensure the other

In social science research, the unit of analysis means the same as the unit of observation.

False The unit of observation is the unit upon which one collects data, while the unit of analysis is the major entity that is being analyzed in the study. Therefore, the answer is false.

In the article "Study suggests link between pesticides, autism", researchers treated pesticide usage as a ratio variable.

False The variable is coded as whether a pregnant woman lived within 547 yards of fields sprayed with organochlorine pesticides during their first trimester of pregnancy. Therefore, it is an interval measure.

It is impossible to have multiple aspects to one single concept.

False There are often multiple aspects to one concept. Therefore, it is possible to have several indicators. For example, when you do not feel well and go to see a doctor, medical professionals are likely to check multiple indicators, such as your body temperature and blood pressure. The concept here is sickness.

Reliability and validity are basically the same in social science research.

False Validity is the proximity of measurement results to the true value. In contrast, reliability is the repeatability or reproducibility of the measurement. Therefore, these two terms are not the same.

An operational definition specifies how a concept will be measured—that is, the operations we'll perform.

True

Dimensions refer to the various aspects of a concept.

True

In real life, it is more difficult to see if a piece of research is valid than if it is reliable.

True

Measurement is the process of observing concepts, as indicated by their operational definitions, and assigning some type of score or meaning to people's responses.

True

The split-half technique for assessing reliability is closely linked to concept of the interchangeability of indicators.

True

The test-retest method assumes that the phenomena under study does not change.

True

To social science researchers, most of concepts are open to varied meanings. It is therefore important to define them properly.

True

Validity refers to the linkage between the operational and conceptual definitions.

True

Predictive validity is often used as another term for criterion-related validity.

True As discussed before, criterion-related validity is based on some external critarion

Construct validity is not the same as predictive validity.

True Construct validity is the degree to which a measure relates to other variables, as expected within a system of theoretical relationships. It is based on the logical relationship among variables. In contrast, predictive validity, sometimes also called criterion-related validity, is based on some external criterion. For instance, the validity of SAT is shown in its ability to predict students' success in college. The validity of driver's license test depends on the driver's subsequent driving record. In these two cases, college success and driving ability are the criteria.

In creating the nominal level of measurements, the two criteria for classification are exhaustiveness and mutual exclusiveness.

True Nominal measusres differentiate between items or subjects based on on their names, categories, or other qualitative classificiations they belong to. Therefore, they have to be exhaustive and mutually exclusive.

The only feature differentiating from the interval measurement is that the ratio measurement has a true zero.

True The ratio measurement provides complete information about variables and the exact value between units. Moreover, it has an absolute zero-which allows researchers to apply a wide range of both descriptive and inferential statistics.

In the eyes of social science researchers, the ratio measurement is almost always more preferable than the nominal measurement.

True This is because it provides more information to the researchers.

Jan has compiled a list of 20 indicators of prejudice and its dimensions. She finds that women are more prejudiced than men on some of the indicators but not on others. Sarah should

check to see if the two sets of indicators represent different dimensions of prejudice. The process is called conceptualization through which researchers give definite meaning to a concept by specifying one or multiple indicators in mind. People may disagree about certain aspect of the concept in the process. If there are systematic differences in the understanding, in this case, bewteen men and women, then researchers should consider whether there is more than one dimension to the concept. A complete concepturalization includes both specifying dimensions and identifying the varous indicators for each.

In social science research, the process of coming to an agreement about what terms mean is:

conceptualization

Reliability assessment is a matter of checking for __________, while validity assessment is a matter of checking for __________.

consistency; accuracy

Julius wanted to validate his measure of driver awareness. He did so by checking with the Utah Department of Motor Vehicles a few years after giving his measure to young drivers to see how many tickets they had received. Which method of validity does this example reflect?

criterion Criterion validity is a measure of how well one variable or set of variables predicts an outcome based on information from other variables. The time dimension is important in this concept.

Prof. Ribar developed four aspects for her concept of happiness: engagement, trust, relationships, and accomplishment. These aspects are known as:

dimensions a distinctive and specifiable aspect of concept

A researcher must decide in the process of operationalization:

how and what to measure, the range of variation to consider, the use of single or multiple indicators, what level of measurement to us

A researcher must decide in the process of operationalization:

how and what to measure, the range of variation to consider, the use of single or multiple indicators, what level of measurement to use

Ken used IQ in his study of K-12 education outcomes. What is the level of measurement of IQ?

interval Essentially, interval data are ordinal, but they have an extra property - the ability to add and subtract measurements. In interval data, the gaps between the numbers are comparable, unlike with ordinal data. Any interval has the same meaning regardless of its location on the scale. "X is five inches longer than Y" has meaning regardless of the values of X and Y. However, ratios are meaningless on an interval scale because an interval scale has no true zero. Temperature scales (Celsius and Fahrenheit) are an example of this. Zero degrees Fahrenheit does not mean the total absence of temperature. Furthermore, if it is 80 degrees outside today and it was only 40 degrees outside yesterday we cannot say that today is twice as hot as yesterday. In short, if the data can be ordered and the arithmetic difference is meaningful, then the data are at least interval data. In this case, IQ is numeric data expressed in intervals using a fixed measurement scale. For instance, someone with an IQ of 140 is NOT twice as intelligent as someone with an IQ of 70. Another case is your SAT or ACT score.

Classifying someone as a "resident" of Utah or "non-resident" treats residency as:

it depends on the research design could be nominal or ordinal

Kate measured class level of students at her university, which has graduate programs, with these attributes: first-year, sophomore, junior, and senior. Which quality of measures does her variable lack?

must be exhaustive

When the variable religious affiliation is classified as Protestant, Catholic, and Jewish, this variable has the important quality of being:

mutually exclusive These options are not exhaustive or comprehensive, since many religions are not included. They are mutually exclusive, because they are not interchangeable. They are nominal measures and not ordinal, because there is not an order among these options.

Suppose a survey item asks someone if something is very important, somewhat important, not very important, or unimportant. This is an example of what level of measurement?

ordinal

Matthew wanted to be particularly careful to represent all levels of feelings about how ethical members of Congress are. So, instead of just asking how much respondents agreed with a statement that "Members of Congress are ethical," he instead asked for responses ranging from "Members are completely unethical" to "Members are always very ethical." Matthew was responding to which operationalization choice?

range of variation The answer is NOT level of measurement. The four levels of measurement are nominal, ordinal, interval, and ratio. Since this is an opinion survey, the answer has to be ordinal. This is predetermined by the nature of opinion surveys. You want to ensure that there is a proper range of variation. In this case, some people might not have paid any attention to the question whether Members of Congress are ethical or not. If this is the case, the range of variation could be even larger by including "No knowledge."

We study how much time you spend on studying each day, measured in hours and minutes. The measure is

ratio A ratio scale possesses a meaningful (unique and non-arbitrary) zero value.

Professor Yu gave an exam on Monday. On Tuesday and Wednesday he gave the same class the same exam. Professor Yu was clearly interested in assessing the exam's:

reliability

The ___________ of concepts in scientific inquiry depends on nominal and operational definitions?

specification

Jen used her twelve-item scale on happiness to see how separate sets of six indicators correlated with each other. Which type of reliability is reflected in this example?

split-half The split-half method breaks a scale in half and uses each half to independently verify the other.

Reliability involves:

whether a particular technique applied repeatedly to the same object would yield the same results each time High reliability means that the same data would have been collected each time in repeated observations of the same phenomenon.

The main reason you need to know the level of measurement for your variables is so that:

you will know which statistics to use.