Research methods final
designing qualitative interviews
Must decide what order to tell the story • A diachronic delivery of material starts at the beginning and progresses chronologically. • A synchronic framework does not depend on time
when designing self report items you must
avoid social desirability adhere to ethical standards of confidentiality and anonymity
Quota sampling 161
begins with a matrix or table describing the characteristics of the target population we wish to represent
Criminal justice research sometimes combines field research with surveys or
data from official records, newspaper accounts, and telephone discussions
surveys may be
descriptive, explanatory, exploratory, or applied research
confidence levels and intervals provide the basis for 150
determining the approximate sample size for a study
problems of matrix questions
develops a pattern of answering respondents get bored or fatigued must keep it short to avoid these problems
Field research is a data collection method that involves the ____of phenomena in their natural settings.
direct observation
random selection solves 2 problems 144
draws on probability theory which allows us to estimate population parameters stops unconscious bias
An important portion of criminal justice research involves estimating population parameters on the basis of sample observations. The summary description of a given variable in the sample is called a sample statistic. Sample statistics are used to make ____________ parameters.
estimates of population
Which factor is a serious consideration for researchers who adopt the role of complete participant when doing field research?
ethics validity generalizability
Different situations require different roles for researchers. Field researchers rely on their understanding of the situation, their judgment, and their experience. In making a decision, researchers must be guided by both _____ considerations. Both of these often conflict.
methodological and ethical
negation in questions leads to
misinterpretation
close ended questions must include 2 things
must be exhaustive and mutually exclusive
sample statistic 144
summary description of a given variable in a population
representativeness in a study is limited to
the characteristics that are relevant to the substantive interest of the study
[True/False] The essential characteristic of interviewers is that their presence in the data collection process must not have any effect on the responses given to questionnaire items.
true
[True/False] The standard error is the measure of sampling error.
true
2 advantages of probability sampling 143
typically more representative than other types of samples permits us to estimate the accuracy/representativeness of a sample
sample element 143
unit about which information is collected and that provides the basis of analysis
Which type of interview is the most natural, in-depth, and exploratory?
unstructured interview
Which of the following is not a guideline for good field note taking?
use only key words and phrases in your notes
targeted victim surveys
used for evaluating policy innovations do surveys before and after policy changes important to evaluate any policy that may increase crime reporting impossible to identify specific location of victimization from data
multi stage cluster sampling 155
used when either impossible or impractical to compile an exhaustive list of elements that compose a target population sampling groups of clusters/elements followed by selecting elements within the clusters are efficient if you can limit the number of population elements
3 advantages of matrix questions
uses space efficiently respondents find it easier to answer makes it easier to compare responses to different questions
population parameter 144
value for a given variable in a population
types of analysis
• Data from agency records: agencies collect a vast amount of crime and CJ data • Secondary analysis - analyzing data previously collected • Content analysis - researchers examine a class of social artifacts (typically written documents)
field research
Field research encompasses two different methods of obtaining data: • Direct observation • Asking questions • May yield qualitative and quantitative data • Often no precisely defined hypotheses to be tested • Used to make sense out of an ongoing process
topics appropriate for agency records
Most commonly used in descriptive or exploratory research • Agency records can also be used for explanatory studies • Applied studies • Content analysis often center on links between communication, perceptions of crime problems, individual behavior, CJ policy
acceptable response rates
50 is adequate 60 is good 70 is very good
recording data
After recording information, researchers must transcribe the dialogue verbatim • After returning from interviews, you must write up field notes no later than the morning after • Memoing: involves writing about your research process and is important to recognize subjectivity Operational, coding and analytic are three types of memos • Operational memos are steps that you took in the research process • Coding memos allow you to document how you coded data • Analytic memos provide ways to explore relationships in the data
identify codes and themes
Coding information from interviews assigns units of meaning to data • Involves the organization of raw data into conceptual categories • Often involves lower-level concepts and higher-level concepts • Themes: higher-level concepts which include a group of lower level concepts that are usually produced from open coding
content analysis
Content analysis: systematic study of messages; can be applied to virtually any form of communication • Decide on operational definitions of key variables • Decide what to watch, read, listen to & time frame • Analyze collected data • Well-suited to answer "Who says what, to whom, why, how, and with what effect
aspects of sampling and coding in content analysis
First establish your universe, then your units of analysis and sampling frame, then sample • Communications need to be coded according to some conceptual framework • Choice between depth & specificity of understanding: • Manifest content - visible, surface content - similar to using closed-ended survey questions • Latent content - underlying meaning Reminders: • Remember operational definition of variables, and their mutually exclusive & exhaustive attributes • Pretest coding scheme • Assess coding reliability via intercoder reliability method and test-retest method
topics appropriate for field research
Gives comprehensive perspective - enhances validity • Go directly to phenomenon, observe it as completely as possible • Especially appropriate for topics best understood in their natural setting • E.g., Disney World's environmental mechanisms of informal social control
focus group interview
Interview schedule: the structure of the interview that may have predetermined questions or topical areas to be discussed • Structured interview: consists of predetermined questions and answer sets • Semi-structured interview: have standardized questions but allows the interviewer to explore themes that emerge during the interview • Unstructured interview: are the most open style interviewing; provides the most breadth, depth and natural interaction with participants
kinds of qualitative interviews
Interview schedule: the structure of the interview that may have predetermined questions or topical areas to be discussed • Structured interview: consists of predetermined questions and answer sets • Semi-structured interview: have standardized questions but allows the interviewer to explore themes that emerge during the interview • Unstructured interview: are the most open style interviewing; provides the most breadth, depth and natural interaction with participants
conducting focus group interviews
Must decide whether to have a natural or artificial group, what the physical arrangement of the group should be, and the appropriate length of the interview • Need to be aware of groupthink and dominant group members • If you are gathering data on a sensitive topic, you must realize that participants can be upset by having to share such information and that you cannot ensure confidentiality
cannot be used to evaluate crime prevention programs locally
NCVS
2 national crime survey
NCVS British crime survey Pg 158-159
key features of qualitative interviewing
Richness, meaning, and shared cultural views • Thick descriptions that reveal meaning of actions in addition to describing actions • Critical realist approach and qualitative interviewing • Critical realist perspective: each perspective contributes to some reality of behavior • Platform for creating questions • Before creating questions, you need to figure out who you will interview and the depth of your questions
correct way of choosing the right role
There is a trade-off between the role observes adopt and their ability to learn from what they see. • Researchers must think about the trade-off • Reactivity: when subjects are affected by the role of the researcher • Different situations require different roles
quality and rigor in analyzing qualitative interviews
To enhance the quality of qualitative analysis, researchers should have an established audit trail • Audit trail: extensive log of data • An important check is to look for negative cases that contradict the emerging themes • Also perform member checks where other researchers read the descriptions and verify the accuracy of the work
linking field observation and other data
Useful to combine field research with surveys or data from official records • Safety audit: type of environmental survey that focuses on security and safety of places
If the researcher feels they will need to ask questions of the subjects, what is the best technique in field research to use?
a combination of observer and participant
sampling frame 146
a list of elements in a population
what is stratified sampling and what populations are they used for 153
a method for obtaining a greater degree of representativeness decreasing the probable sampling error done on homogeneous subsets of a population
systematic samplings one danger 153
a periodic arrangement of elements in the list can make systematic sampling unwise
critical realist perspective
a philosophical view that reality exist but knowledge is constructed through multiple meanings
equal probability selection method (EPSEM) 143
all members of a population have an equal chance of being selected in the sample
2 advantages of probing for questions
allows time to devise the best, most neutral probes ensures interviewers will use some probes as needed
warning mailings and cover letters
can be used to verify tenants and increase response rate tells people to expect a survey and describe purpose
The reliability of quantitative field studies can be enhanced by ____.
careful attention to details of observation
[True/False] A good way to avoid sampling bias in the assessment of public opinion about a specific criminal justice topic is to employ the use of a radio call-in show.
false
[True/False] Fortunately, when doing field research sampling is very straight forward and you don't have the concerns that would be present in survey techniques.
false
Survey research is, of necessity, based on an unrealistic stimulus-response theory of cognition and behavior. It is based on the assumption that a questionnaire item will mean the same thing to every respondent, and every given response must mean the same thing when given by different respondents. Although this is an impossible goal, survey questions are drafted to approximate the ideal as closely as possible. The interviewer assists to accomplish this goal by being a __________________ through which questions and answers are transmitted.
neutral medium
respondents should be able to
read item quickly and understand intent and select or provide an answer without difficulty
General population surveys and surveys of offenders tend to present which types of difficulties?
recall error
Field research is especially effective for studying the subtle nuances of behavior and for examining processes over time. For these reasons, the chief strength of this method is the depth of understanding it permits. Validity is usually a strength of field observation, but ____________________ are sometimes weaknesses.
reliability and generalizability
samples do 2 things
represent a population generalize from a sample to an unobserved population the sample represents
in person interview perks
higher response rates with less ID answers can help respondent with questions get reactions to the study
multi stage cluster sampling with stratification has a primary goal which is 156
homogeneity
snowball sampling 163
identifies a single or small # of subjects and then asking the subjects to identify others who might be willing to participate in a study often used to study users and dealers may be necessary when target population is difficult to locate or identify
[True/False] Qualitative field research is the preferred method for studying many types of active offenders, especially those with no recorded arrests or convictions for particular crimes.
true
follow up mailing is best to be sent to
non respondents with a follow up letter
[True/False] Sampling is often used when it is not possible to collect information from all persons you want to study and when it is not necessary to collect information from all persons.
true
Which of the following is not an advantage of the use of matrix questions?
they do not take much thought on the part of the respondents to answer
self reports
to explore and explain why people commit criminal, delinquent, or deviant acts and asking questions is the best method focuses on frequency of offending but others focus on prevalence (how many crimes v. how many offenders)
2 ways to ensure control of in person interviews
train interviewers and supervise work
[True/False] Data and records produced by formal organizations may be the most common source of data in criminal justice research.
true
[True/False] Missing data is a type of connector where you are alert for what participants are failing to mention.
true
when creating a questionnaire avoid
cluttering the questionnaire
data analysis and making claims
Data is managed through tables, charts and other visual displays • Data reduction involves putting aside information that seems irrelevant • Thinking units: simple framework for making sense of all the stories that emerge • Possible thinking units: meanings, practices, episodes, encounters, roles, relationships, groups, organizations, settlements, social worlds and lifestyles • Sensitizing concepts: general references and guides about what you are looking for
gaining access to participants
Establish your role to determine if you are an insider or outsider • Insider/outsider status: your level of insiderness depending on how close you are to a group • To gain access to a formal organization, you will need identify yourself as a researcher and make a formal request and receive formal approval Best to use a four step process: sponsor, letter, phone call, and meeting • To gain access to informal subcultures, researchers can use a sponsor or hang out where subjects hang out • Compensation might be necessary to encourage participation
various roles of the observer
Ethnography: focuses on a detailed and accurate description rather than explanation • Complete participant: participates fully; true identity and purpose are not known to subjects • Participant-as-observer: make known your position as researcher and participate with the group • Observer-as-participant: make known your position as a researcher; do not actually participate • Complete observer: observes without becoming a participant
grounded theory
Grounded theory: stems from an analysis of patterns, themes, and common categories discovered in data • Combines a naturalist approach with a positivist concern for a systematic set of procedures • Theoretical sampling might be used to confirm themes that are observed • Theoretical saturation occurs when you reach the point when additional interviewing will not yield new results
Confidence levels allow the researcher to have some idea of how closely their samples reflect the parameter. Which statement about confidence levels is true?
If a simple random sample is used, probability theory suggests that researchers can almost be positive (99+ percent) that estimates of that sample fall within three standard errors of the true value.
Which of the following is true concerning sampling error?
If many independent random samples are drawn from a population, then the sample statistics from those samples are distributed in a predictable way around the population parameter.
validity and reliability in field observations
Observations are strong in validity Problems occur when subjects are aware of the researcher's role Reviewing field notes and interview transcripts can improve validity Can have potential problems with reliability Interpretation of observations might vary by observer Reliability can be improved through careful attention to the details of observation Generalizability is often a problem
structured observations
Observer makes marks on a closed-ended questionnaire • Training for this is extensive and time consuming • Have higher reliability • Environment surveys: observers record information about conditions of a specified environment
probes
Prompt participants to elaborate on responses by filling in more detail and depth. • It is important to have built in prompts in case you have quiet respondents • You can use an attention probe (e.g., lean in), a continuation probe (e.g., nod), clarification probe (e.g., ask the respondent to clarify), or follow-up questions
types of agency records
Published Statistics - government organizations routinely collect and publish compilations of data (e.g., NCVS, Census Bureau, BJS; often available in libraries and online • Nonpublic Agency Records - agencies produce data not routinely released (e.g., police departments, courthouses, correctional facilities) • New Data Collected by Agency Staff- collected for specific research purposes • Less costly • Allows the researcher more control
conducting qualitative interviews
Qualitative interviews can be in-person, on the phone, online, or a survey • Face-to-face are most common • Reflexivity: refers to your subjectivity and the meaning you give to information • During interviews you will need to develop a rapport with respondents • Can be done through informal conversations or finding something you and the respondent have in common
topics appropriate to qualitative interviewing
Qualitative interviews can be the sole way of gathering data in criminal justice studies • Allows the research to understand the subjects' perspectives • Can gather first hand accounts of their impressions and their lived experience. • Can also be used to understand how people feel about their roles and identities
rapport
Rapport: how you connect with subjects • Influenced by insider/outsider status • Can do a number of things to maintain rapport once an interview begins • Converse • Listen • Pay attention to meaning • Probe • Persist • Play innocent • Pay attention to interview length • Word questions clearly • Sequence your moves • Be candid • Show respect • End on a positive note
selecting cases for observation
Snowball sampling Must consider potential selection bias Also might be possible to use probability sampling methods Purposive sampling Variation in observations to consider when sampling ◦ Population ◦ Space ◦ Time, micro ◦ Time, macro ◦ Weather
reliability and validity problems with agency records
Social production of data: most criminal justice record keeping is a social process ◦ Most data reflects reactions to behaviors Agency data are not designed for research ◦ Data reflect internal agency needs that might not be the same as research needs Tracking people, not patterns Error increases as volume of data increases
secondary analysis
Sources • Interuniversity Consortium for Political and Social Research (ICPSR): central repository of data collected by social science researchers • National Archive of Criminal Justice Data (NACJD): source of criminal justice data established by BJS Advantages - cheaper, faster, benefit from work of skilled researchers Disadvantages - data may not be appropriate to your research
strengths and weaknesses of field observation
Strengths: Chief strength is depth of understanding Flexibility Study active offenders in the field Study justice officials in natural settings Weaknesses: Usually does not yield precise descriptive statements about a large population Precise probability samples cannot normally be drawn
recording observations
Various methods for recording observations Span from pen and paper to video recordings The methods selected for recording observations are directly related to issues of measurement Do not trust your memory Take notes in stages Note everything that seems important and things that do not seem important at the time
likert scale
strongly agree through strongly disagree
In addition to be used for exploratory and explanatory studies, agency records are frequently used in ___________ as well. Evaluations of new policies that seek to reduce recidivism might draw on arrest or conviction data for measures of recidivism.
applied research
matrix questions
ask several questions with the same set of answers
improperly laid out questionnaires
can cause respondents to miss questions, confuse them about nature of the data desired, and even lead them to throw the questionnaire away
questionnaires must be
clear and unambiguous
cover letters help response rate because
communicates why survey is being conducted along with how they were chosen also identifies the institutional affiliation of the survey
advantages of computer assisted in person interviews
complex contingency questions are automated language translation facilitate respondents the cannot read
Coding in content analysis involves the logic of _____________________________________. In content analysis, as in other research methods, researchers must refine their conceptual framework and develop specific methods for observing in relation to that framework.
conceptualization and operationalization
types of bias 141
conscious and unconscious bias
[True/False] Data from agency records or archives may have originally been gathered in any number of ways, from sample surveys to direct observation. Because of this, such data may, in principle, be inappropriate for just about any criminal justice research topic.
false
[True/False] Stratified sampling results in a greater degree of representativeness, but at the same time, increases the probable sampling error.
false
[True/False] The advantages and disadvantages of using secondary data are not the same as agency records.
false
Audit trails consists of extensive logs of data, such as ____.
field notes and memos
To gain access to a formal criminal justice organization for the purpose of conducting qualitative interviewing, which step should be followed?
find a sponsor, send a letter, follow up with a phone call and then set up a meeting
probing for questions
frequently required to elicit responses to open ended questions "how was that" "anything else" probe must not affect the nature of subsequent response
Snowball Sampling is another type of nonprobability sampling that closely resembles the available-subjects approach. Snowball sampling is commonly used in field observation studies or specialized interviewing. It begins by identifying a single subject or small number of subjects and then asking the subject(s) to _______________________.
identify others like him or her
timing of sending a survey
important to avoid holiday months when mail is heavy and likely to be thrown away and be slow responses
qualitative interview
interaction between an interviewer and respondent where the interviewer has a general plan of inquiry, including the topics covered, but not necessarily a specific set of question that have to be asked in a particular order
Agency data are not collected with the research process, research questions, or the researcher in mind. In many cases, criminal justice officials collect data because the law requires them to do so. Recordkeeping procedures reflect __________________________. As a consequence, researchers sometimes find it difficult to adapt agency records for a specific research purpose.
internal needs and directives from higher authority
standard error and sample size correlate 149
inversely as sample size rises the standard error decreases
The methods selected for recording observations are directly related to issues of measurement, especially how _____. For example, if we are interested in policies to increase nighttime pedestrian traffic in some city, we might want to know why people do or do not go out at night and how many people stroll around different neighborhoods.
key concepts are operationalized
cluster sampling involves 2 repeating steps 155
listing and sampling
The behavior of people, together with the characteristics of people and places, can vary by population group,___________________________________.
location time and weather
Reflexivity refers generally to one's subjectivity in the research process. Thus, the process of reflexivity requires that one be critically conscious of ____.
ones feelings
ordering questions in a questionnaire
order in which questions are asked can affect answers given content of one question can affect later answers demographic data should be at the end of the questionnaire
A sampling frame is the list or quasi-list of elements from which a probability sample is selected. It is a sampling frame is a list or quasi-list of target population. It is a "quasi-list" because, even though an actual list might not exist, samples can be drawn as if there were a list. Properly drawn samples provide information appropriate for describing the _____ that compose the sampling frame.
population of elements
types of qualitative interview
producing thick description that helps reveal the richness of human experience
types of nonprobability sampling 160
purposive sampling quota sampling reliance on available subjects snowball sampling
Which of the following is accurate concerning data acquired during field research?
qualitative data that is not easily reduced to numbers
Mail in surveys
questionnaires accompanied by a letter of explanation and self addressed stamped envelope for returning questionnaire
contingency questions
questions in a series that are only answered if it pertains to a follow up of the first question
problems of computer based surveys
representativeness most is sent to junk mail not everyone has a computer people worry about spam and wont open
purposive sampling 161
research is aimed at our own judgment and purpose of the study use this sampling to represent patterns of complex variation pretesting a questionnaire is an example
systematic sampling 153
researcher chooses all elements in the list for inclusion in the sample
open ended question
respondent gives their own answers answers up for interpretation which leads to bias
perks of computer based surveys
responses are automatically recorded and save $ can include multiple types of questions
large standard error indicates 149
sample estimates are widely disbursed while small standard errors means estimates are clustered around population parameters
The standard error is a valuable piece of information because it indicates how closely the sample estimates will be distributed around the population parameter. A larger standard error indicates ____________________, while a smaller standard error means that estimates are more clustered around a population parameter.
sample estimates are widely dispersed
disproportionate stratified sampling 154
samples not representative of population larger samples can be drawn to adequately represent the larger sampling error in population
Investigators who conduct research funded by federal agencies such as the National Institute of Justice are usually obliged to release their data for public use. Thus, if a researcher uses and analyzes data previously collected by others or government agencies usually for purposes that differ from the current researcher's, then that is called ___________________.
secondary data analysis
sampling refers to
selecting parts of a population
close ended question
selection answer from a list useful due to uniform responses
3 ways of administering questionnaires
self administering, face to face, and telephone interviews
when attempting to avoid bias it is best to not 143
self selection
Focus groups are _____ used to develop or elaborate understanding of complex topics. Focus groups are most useful when precise generalization to a larger population is not necessary and the focus groups participants and the larger population are relatively homogeneous.
semi structured group interviews
the nature of reality is viewed in 2 distinct ways
some people believe the real world exist beyond our perceptions and constructions whereas others, known as radical constructivist, think that reality is a construction based on what we perceive reality to be
The ___________________ is the person who will assist you in gaining access to a subculture.
sponsor
reliance on available subjects 162
stopping people on the street to ask questions seldom produces good data
Many governmental agencies disseminate highly aggregated summaries of statistics. This is called summary data. This _____ is not much useful if a researcher wants to know about individual correctional facilities or facility inmates. For that a researcher will have to seek alternate sources of data.
summary data
Which of the following would be considered an appropriate method for increasing the response rate in mailed surveys?
targeting a narrowly defined population
The reliability and validity of agency records, as well as the general suitability of those data for a research project, is primarily dependent upon understanding _____________________. Understanding how the data were originally collected helps researchers identify potential new uses of data and be better able to detect potential problems with reliability and validity.
the details of how agency records are produced
what is standard error 149
the measure of a sampling error that indicates how closely sample estimates will be distributed around population parameters
It is usually best to begin a self-administered questionnaire with ____.
the most interesting set of questions
Which of the following are necessary in the construction of closed-ended questions?
the response categories must be exhaustive
[True/False] Self-Report surveys are used to ask people about crimes they may have committed seeks to explore or explain why people commit criminal, delinquent, or deviant acts.
true
[True/False] Surveys may be used for descriptive, explanatory, and applied research.
true