13.3 The Survey Research Design

Clearly, the purpose of these questions is to obtain the demographic characteristics of customers;

that is, to put together a description of the people who are likely to buy this product so that the company can do a better job of targeting its advertising.

Conducting survey research presents researchers with four issues that must be addressed for the results to be accurate and meaningful.

-First, survey questions must be developed. -Second, the questions must be assembled and organized to produce a well-constructed survey. -Third, a selection process must be developed to determine exactly who will participate in the survey and who will not; survey participants must be representative of the general group to be studied. -Finally, researchers must determine how the survey will be administered. Will participants receive printed surveys through the mail; will the survey questions be read to people over the telephone; or will participants complete the questions online in an Internet survey, or in person?

An interview provides an opportunity for follow-up questions, and it is possible to explore complex issues more fully than could be done with a few isolated paper-and-pencil questions

. Finally, interviews allow you to gather information from individuals who are unable to read and answer printed questions such as young children, people who cannot read, and people with low IQs.

There is no absolute rule for determining the number of categories for a rating-scale question; however, researchers commonly use from 5 to 10 numerical values. The reasoning behind this range of values is based on two observations:

1. Participants tend to avoid the two extreme categories at the opposite ends of the scale, especially if they are identified with labels that indicate extreme attitudes or opinions. Thus, the actual scale is effectively reduced by two categories. 2. Participants have trouble discriminating among more than 9 or 10 different levels. If the scale offers more than 10 options, the participants usually blend categories and effectively create their own 10-point scale.

2. Sensitive questions or items that may cause embarrassment or discomfort should be placed in the middle of the survey.

By the time participants encounter these items, they are more likely to have warmed up to the topic and to have become committed to completing the survey.

Restricted Questions

A restricted question presents the participant with a limited number of response alternatives, thereby restricting the response possibilities. Like a multiple-choice question, a restricted question typically asks the participant to select the best or most appropriate answer in a series of choices. For example: 1. If the election were held today, which candidate would receive your vote ? A. Mr.Jones B. Ms.Smith C. Mr.Johnson Because these questions produce a limited and predetermined set of responses, they are easy to analyze and summarize. Typically, the data are tabulated and reported as percentages or proportions of participants selecting each alternative.

5. Finally, vocabulary and language style should be easy for participants to understand.

A survey with language appropriate for college students probably would not be appropriate for elementary school students.

3. Questions dealing with the same general topic should be grouped together.

Also, questions in the same format should be grouped together; for example, all rating-scale questions should be grouped together. Grouping questions simplifies the survey so participants do not have to jump from one topic to another or switch from one type of question to another.

With a survey, a researcher does not have to wait until a behavior or response occurs; for example, it is not necessary to wait until after an election to discover people's attitudes about candidates or issues; we can ask at any time.

Although surveys can be used to obtain scores for a variety of different research designs, a survey often is conducted simply to obtain a description of a particular group of individuals. A study using the results from a survey simply for descriptive purposes is classified as a survey research design.

The primary advantage of an open-ended question is that it allows an individual the greatest flexibility in choosing how to answer.

An open-ended question imposes few restrictions on the participant and, therefore, is likely to reveal each individual's true thoughts or opinions.

Open-Ended Questions

An open-ended question simply introduces a topic and allows participants to respond in their own words. For example: 1. What do you think about the current availability of food on this campus? 2. In your view, what are the most important factors in choosing a college or university?

Mail Surveys

Another common method of administration is to mail the survey to a large sample of individuals. For individual participants, a mailed survey is very convenient and nonthreatening. Individuals can complete the survey at their own convenience, and can be relatively confident that responses are anonymous and confidential. On the other hand, the fact that the survey is anonymous means that a researcher can never be sure exactly who in the household completed and returned the survey.

Telephone Surveys

Another method of administering a survey is to contact individuals by telephone. However, administering a survey by telephone can be incredibly time-consuming. The obvious problem with a telephone survey is that there is a direct, one-to-one relationship between the time spent by the researcher and the time spent by the participants; to complete 60 minutes of survey questions and responses, a researcher must spend 60 minutes on the telephone. Therefore, most telephone surveys are restricted to situations in which a large number of researchers or assistants can share the telephone assignments.

Finally, it is possible to increase response rates by giving participants advance warning of the survey, then providing a follow-up reminder after the survey has been received. Typically, participants are notified about 1 week before the survey is mailed, that they have been selected to participate. The advance warning helps make the individuals feel special (they are a select group) and helps ensure that they will be watching for the survey in the mail.

Approximately 1 week after the surveys have been received, a follow-up is sent to remind each person to complete and return the survey (if they have not done so already) and to thank each person for participating. Essentially, the advance notice and reminder provide a polite way to add an extra please and thank-you to the recruitment process and can significantly increase the response rate.

Surveys and questionnaires are used extensively in the behavioral sciences as relatively efficient ways to gather large amounts of information.

By presenting people with a few carefully constructed questions, it is possible to obtain self-reported answers about attitudes, opinions, personal characteristics, and behaviors. The simple notion behind a survey is that it is not necessary to observe directly where people shop or what foods they prefer, or how many hours they sleep each night; instead, we simply ask

A second technique for improving response rates is to include a gift or token of appreciation with each survey.

Common examples include a pen ("Please use this pen to fill out the survey, then keep the pen as our gift to you.") or money. Some surveys arrive with a dollar taped to the top and a note suggesting that the recipient use the money to buy a cup of coffee ("Sit back and enjoy your coffee while you complete the survey.").

LO7

Define open-ended, restricted, and rating-scale questions; identify examples of these three types of questions; and describe the strengths and weaknesses of each.

LO8

Describe the four methods for administering a survey (mail, phone, Internet, and in person) and explain the strengths and weaknesses of each, including the problems of nonresponse bias and interviewer bias.

LO6

Describe the general characteristics of the survey research design.

It also is possible to obtain quantitative information from restricted questions by using an ordered set of response alternatives. For example:

During a typical week, how often do you eat at a fast food restaurant? A. Not at all B. Once C. Twice D. Three times E. Four times or more With this type of question, it is often possible to compute some kind of average response for a group of participants.

For example, a question may severely restrict response options (Which of the following three flavors of ice cream do you prefer?), or a question may give each participant complete freedom in choosing a response (What is your favorite ice cream flavor?). The wording of a question also can introduce bias into participants' answers (Are you one of those bland, unimaginative people who prefer vanilla ice cream?).

Finally, different types of questions permit different types of statistical analysis and interpretation. If answers are limited to non-numerical categories on a nominal scale, for example, you cannot compute a group average. In this section, we consider three general types of self-report questions. Each type has its own individual strengths and weaknesses and is designed to obtain specific information.

The survey research design introduces a few additional concerns regarding sample selection.

First, many surveys address a specific issue that is relevant to only a small subset of the general population. For such a survey, care must be taken to select survey participants to whom the questions are relevant. Ex : For a survey about childcare issues, for example, participants should be parents with small children. A sensible strategy might be to hand out surveys to parents as they pick up their children at childcare centers around the city. Or you might obtain mailing lists or e-mail addresses from the different childcare centers. Similarly, participants for a shopping survey might be selected from the people in a shopping mall, and participants for an education survey could come from the parents of children in the local school district.

Although it is impossible to eliminate nonresponse bias completely, several actions can increase the overall response rate for a mail survey and thereby reduce the bias.

First, response rates can be significantly improved if a good cover letter accompanies the survey. A cover letter should introduce the survey and ask for participation and should include the following elements:

Administering a survey by telephone does have some advantages.

First, the survey can be conducted from home or office. If several people place the calls and the survey is relatively brief, it is possible to contact a fairly large number of participants in only a few days. If you are considering a telephone survey, here are a few important notes for improving your chances for success.

The goal of the survey research design is to obtain an accurate picture of the individuals being studied. The survey provides a "snapshot" of the group at a particular time. Sometimes, survey research focuses on a specific characteristic such as eating behavior or political attitudes; other survey research may seek a more complex picture of a variety of behaviors and opinions

For EX : a researcher could use a survey to investigate alcohol use at a local high school. Depending on the questions asked, the results could provide a description of how many students drink alcohol, how much they drink, and when and where. Other questions could yield a description of student attitudes toward alcohol use among their peers.

1. An explanation of why the topic is important.

For a survey on television program preferences, for example, you should point out the major role that television plays in the entertainment and education of most people.

A major concern with the interview is that interviewer bias can distort the results.

For example, a participant may perceive a smile or nod from the researcher as a sign of approval or encouragement to continue on the current topic. Thus, the participant's response may be influenced by subtle actions on the part of the interviewer. Although it is impossible to completely eliminate this problem, it can be limited if the interviewer maintains a consistent attitude throughout the entire interview. A common strategy is to adopt a universal, mildly positive response to anything the participant says.

Second, although some surveys focus on a specific topic or group of people, some surveys seek to describe a broad cross-section of the general population. In this case, the sample of survey participants must not be too restricted.

For example, administering surveys to the students in a psychology class would not result in an accurate description of the political attitudes of people in the community. A researcher should take some time to identify the group to be described, and then make an effort to select individuals who accurately represent the group. This often means that the individuals who participate in the survey are not necessarily the ones who are easiest or most convenient to obtain.

Although the question may lead the participant in a particular direction or suggest a specific point of view, individuals are free to express their own thoughts. However, this can also be a major disadvantage.

For example, different participants may approach the question from entirely different perspectives, leaving you with answers that are impossible to compare or summarize. To the question about food on the college campus, for example, one individual may respond with a list of food suggestions, another may suggest new locations for selling food, and a third participant may state simply that the current situation is "okay." All three answers may be useful, but they are clearly not compatible with each other, and they may be very different from the issue you had in mind when the original question was written.

In addition to the costs of a low response rate, there may be a bias differentiating those who do and those who do not return surveys. One obvious possibility is that people who are most interested in the survey topic (those with the most intense feelings) are most likely to complete and return the survey. As we noted with Internet surveys, this trend creates nonresponse bias in the sample, which simply means that the individuals who return surveys are usually not representative of the entire group who receives them.

Imagine, for example, a survey about blocking Internet sites on the computers at a public library. Although the surveys are mailed to all library patrons, they are most likely to be completed by people who are passionate about free speech and those who are paranoid about pornography. Neither group accurately represents the people who typically use the library. As a result, nonresponse bias can limit your ability to generalize survey results and poses a threat to the external validity of your study.

There is also no absolute rule for labeling the categories. Typically, the opposite extremes are identified with verbal labels called anchors that establish the endpoints of the scale.

In addition, the central category is often labeled, especially if it represents a neutral response. Beyond the endpoints and the middle, however, labeling categories is optional.

A secondary advantage of rating-scale questions is that participants usually find them easy to understand and easy to answer. Because the scale permits different degrees of response, participants are not forced into an absolute yes or no, all-or-none choice

Instead, they can qualify their answers by indicating degrees of agreement or approval. It is also easy for participants to breeze through a long series of questions after the rating scale has been introduced at the beginning of the survey. Thus, it is possible to collect a lot of data on a variety of different topics in a single, relatively efficient survey.

. Furthermore, it can be difficult to control the sample of respondents. For example, there is no simple system for organizing e-mail addresses. Many households have several computers with several different users, all of whom have different e-mail addresses. In addition, many people have more than one e-mail address. This makes it difficult to identify and select a sample of individuals or households who will be asked to participate in the survey.

Internet surveys are controlled best when they are administered to a closed group of e-mail users such as a university or other organization with a common address. For example, e-mails directing people to a survey website can be sent to all the students at a university by using the university list serve. If a link to a survey is simply posted on an existing website, you have no idea who might visit the site and decide to participate in the survey. Although people who visit a website are likely to be interested in the content, and therefore are a relevant sample, you have no ability to control or even determine the composition of the sample.

Internet Surveys

Internet surveys provide an economical and efficient medium for reaching a large number of potential respondents. A related advantage of Internet surveys is that a researcher has greater access to participants with a particular characteristic Another advantage of an Internet survey is the flexibility in presenting questions and response alternatives. administering a survey on the Internet has numerous disadvantages related to issues of the sample. . Internet surveys, similarly to mail surveys, are subject to nonresponse bias, which means that the people who complete surveys are a self-selected sample that may not be representative of the population.

2. Practice reading the survey aloud.

Listening to a question can be different from reading a question. On the telephone, participants cannot see the punctuation and other visual cues that help communicate the content of a written question. A good strategy is to pretest your survey questions by reading them to a set of friends. Be sure that your listeners understand the questions as you intended.

Rating-Scale Questions A rating-scale question requires a participant to respond by selecting a numerical value on a predetermined scale.

Movie critics often use this type of scale to evaluate films with a number from 1 to 10. The numerical scale that accompanies each question typically presents a range of response alternatives from very positive to very negative. A common example uses a 5-point scale on which individuals rate their level of agreement or disagreement with a simple statement: 1-Strongly disagree 2-Disagree 3-Neither agree or disagree 4-Agree 5-Strongly agree

A second disadvantage of open-ended questions is that the answers are often difficult to summarize or analyze with conventional statistical methods. As with the food question, different participants may provide responses that are difficult to group together or to average in any meaningful way.

Often, the researcher must impose some subjective interpretation on the answer, such as classifying a rambling response as generally positive or generally negative. Finally, the responses to open-ended questions may be limited by a participant's ability or willingness to express his or her thoughts. Inarticulate or tired people may give very brief answers that do not completely express the true breadth of their thinking.

3. Beware of interviewer bias. Whenever a researcher has direct contact with participants, even over the telephone, there is a risk that the researcher will influence their natural responses.

On the telephone, the primary problem is exerting influence by tone of voice or by rephrasing questions. The standard solutions are to practice reading the survey questions in a consistent, neutral tone, and never to alter a survey question. If a participant does not understand a question and asks for clarification, your only option is to reread the question. If you paraphrase a question or try to explain what it means, then you have changed the question and maybe even changed the participant's answer. Consider the following two versions of the same question. The first version uses neutral wording and focuses on the library hours. The second version is phrased in a leading way; that is, it appears to be an invitation for the participant to join a happy little group (especially if the question is read in a very friendly tone of voice). Do you think there should be an increase in the hours that the library is open on weekends? Don't you think we should increase the hours that the library is open on weekends?

Constructing a Survey

Once the survey questions are determined, the next step is to organize the questions into a coherent survey that participants can easily understand and complete. The details of constructing a survey are beyond the scope of this text, but there are a few general guidelines for creating a well-organized survey.

Administering a Survey

Once you have developed the survey questions, constructed the survey, and identified the participants, the next step is to distribute the survey to the individuals you would like to investigate. There are a number of options for administering a survey, each of which has advantages and disadvantages. In this section, we examine some of the most common methods for administering surveys: on the Internet, by mail, by telephone, and in person.

4. A contact person (name, address, and telephone number) whom participants can call or write to if they have any questions or comments.

Participants rarely contact this person, but a real name and address help personalize the survey.

4. Begin by identifying yourself and your survey.

People are constantly bombarded by "junk" telephone calls and are inclined to hang up whenever a stranger calls. You can help avoid this problem if you immediately identify yourself and your topic, and make it clear that you are conducting a survey and not trying to sell anything. Your first few sentences on the telephone are similar to the cover letter for a mail survey and should contain the same elements.

5. The signature of a person who is recognized and respected by individuals in the sample.

People are more likely to respond if they are asked to by someone they know and like.

Strengths and Weaknesses of Survey Research

Picture attached summarizes the strengths and weaknesses of each method for administering a survey. In general, one of the real strengths of survey research is its flexibility. Surveys can be used to obtain information about a wide variety of different variables including attitudes, opinions, preferences, and behaviors. In fact, some of these variables are very difficult to describe in any other way. In addition, surveys typically provide a relatively easy and efficient means of gathering a large amount of information.

In-Person Surveys and Interviews

Probably the most efficient method for administering a survey is to assemble a group of participants and have all of them complete the survey at the same time. You can ask people to sign up for predetermined meeting times, or simply ask for volunteers to gather at a specific time and place. Another possibility is to approach preexisting groups such as those in school classrooms or workplace lunchrooms. By having participants volunteer before the survey is presented, you guarantee a 100% response rate. The efficiency comes from the fact that you give instructions once to the entire group and then collect a whole set of completed surveys in the time it takes one participant to finish.

4. If participants are going to read the survey, the format for each page should be relatively simple and uncluttered.

Questions that are crammed together and seem to fill every square inch of the page create an overwhelming appearance that can intimidate participants.

Selecting Relevant and Representative Individuals

Researchers typically want to generalize their results from the study's sample to the target population. In addition, the external validity of a research study is limited, in part, by the representativeness of the sample to the population (chp 6)

CAUTION !

Simply because a study uses a survey does not mean that it is a survey research design! The defining element of the survey research design is that the results of the survey are used simply to describe the variables being studied.

Mailing surveys is usually a relatively simple and easy process, although printing a large number of surveys, addressing them, and paying postage can be expensive and time-consuming.

The expense is compounded by the fact that response rates tend to be very low for mailed surveys. A response rate of 10%-20% is fairly typical. This means that you need to distribute at least five times the number of surveys you hope to have returned.

3. An emphasis on the importance of each individual response.

The intent is to encourage all people to respond, whether or not they feel strongly about the issues in the survey. The cover letter should point out the importance of results that represent the entire population (not just a small group with special interests) and that it is, therefore, especially important that each person respond.

Types of questions

There are different ways to ask participants for self-report information. Sometimes, you may be satisfied with a simple yes or no answer (Have you ever...), but in other circumstances, you may want a quantitative answer (how much, how often). Different types of questions encourage different types of responses. Also, different types of questions permit different degrees of freedom in the participants' answers.

Occasionally, individual researchers seek professional help preparing surveys and identifying participants. In most major metropolitan areas, there are several research companies that design, administer, and analyze surveys

These companies usually have access to specialized mailing lists that can focus on a specific, well-defined population. Typically, a researcher supplies specific demographic characteristics, and the computer generates a list of individuals who meet the criteria, for example, single mothers between the ages of 20 and 29 who have an annual income greater than $35,000. Focusing a survey in this way can increase the chances of obtaining a reasonable number of useful responses.

1. Demographic questions (such as age, gender, level of education) should be placed at the end of the survey.

These items are considered boring, and you do not want participants to quit because they are bored by the first few questions. In addition, identifying age, race, or gender first may influence how the participant answers survey questions that relate to these variables.

2. An explanation of the usefulness of the results. Usually, the results of a survey are used in future planning or to help determine a future course of action.

This should be explained in the cover letter so that participants know that the information they are providing may actually influence them in the future.

One criticism of rating-scale questions is that whenever questions in a series all have the same choices for responding, participants tend to use the same response to answer all (or most) of the questions.

This tendency is called a response set. With a Likert-type scale, for example, some participants use the neutral (#3) answer for everything. One rationalization is that they do not feel strongly about any of the items, so they really are neutral.

The rating scale is usually presented as a horizontal line divided into categories so that participants can simply circle a number or mark an X at the location corresponding to their response (Figure 13.1).

This type of rating-scale question is often called a Likert scale (or a Likert-type scale) Notice that the scale is presented with equal spacing between the different response choices. The idea is to simulate an interval scale of measurement, and the responses from rating scales are usually treated as interval measurements. Thus, the distance between agree and strongly agree is treated as a 1-point distance that is equivalent to any other 1-point difference on the scale.

Using the five items in Figure 13.1 as an example, each participant receives a total score obtained by adding the responses from the five items. A participant who answered 1 (strongly disagree) to all five items would have a total score of 5. Someone who answered 5 (strongly agree) to all five items would have a total score of 25.

Thus, we can position each individual on a scale that represents attitudes toward mathematics. This way, we can compare different individuals and compute means to describe different groups of participants. In general, it is very easy to use standard statistical procedures to summarize and interpret the results from a rating-scale question.

(A more likely explanation is that they simply want to finish quickly.) Another possibility is that a participant may use the agree category for all responses except to those few items where there is serious disagreement. To minimize this problem, it is recommended that the items include a mixture of positive and negative statements, including some alternate phrasing of the same item. For example :

Today's teenagers are rude and disrespectful. Later in the series, an alternate item might be: Today's teenagers are polite and courteous. The intent is to force respondents to move back and forth between opposite sides of the scale so that they cannot fall into a single response set for answering the questions. The primary advantage of rating-scale questions is that they produce numerical values that can be treated as measurements from an interval scale.

We have already noted some of the disadvantages of survey research, such as low response rates and nonresponse bias. Responses to survey questions can also be difficult to analyze or summarize. This problem is especially important with open-ended questions, to which participants are allowed to respond in their own words. A final concern about survey research is that the information obtained is always a self-report.

Ultimately, the quality of a survey study depends on the accuracy and truthfulness of the participants. It is certainly possible that at least some participants will distort or conceal information, or simply have no knowledge about the topic when they answer certain questions. Therefore, if your survey results show that 43% of the high school students use alcohol at least once a month, keep in mind that the results actually show that 43% of the students report using alcohol at least once a month.

It also is possible to administer a survey in person to a single participant. In this case, the survey becomes a one-on-one interview. Although this appears to be a very inefficient method of collecting information, an interview can be quite valuable.

Usually, interviews are reserved for a very small group of specially selected individuals, often called key informants. Typically, these are people who have unique perspectives on the issues or unique access to information (such as a college president, a chief of police, or a mayor). Interviews are also useful in situations in which you are willing to accept the limitations of a small group of participants in exchange for the in-depth information that can come from a detailed interview.

A common application of survey research is by companies to obtain more accurate descriptions of their customers.

When you buy any electronic device, for example, a warranty registration card usually accompanies it. In addition to your name and address and the serial number of the product, other demographic questions are usually asked: What is your age? What is your occupation? What is your income? How did you hear about our product?

Finally, an element of open-endedness can be allowed in a restricted question by including a blank category where participants are free to fill in their own responses. For example:

Which of the following is your favorite local grocery store? A. Wegmans B. Trader Joe's C. Whole Foods D. Other (specify)

1. Keep the questions short and use a small number of response alternatives.

With a telephone survey, the participants do not have a written copy for reference, so you must depend on the listener's memory. If a participant gets confused or lost in the middle of a long, complicated question, you may not get a sensible response.

survey research design

is a research study that uses a survey to obtain a description of a particular group of individuals.