Research Methods 1

An Interactive of Qualitative Research Design

Purpose and Theory --> Research Questions --> Methods and sampling strategy

RCT=...

Randomized Controlled Trial the "gold standard"...

General ethical responsibilities

- General ethical responsibilities o Requirement of report of sexual or physical abuse of children overrides any confidentiality agreement made o While you have particular ethical responsibilities as a researcher this does not mean that you have a privileged voice on what constitutes ethical behavior on others.

experimental fixed designs need....

1) random assignment 2) manipulation 3) Control

Advantages in carrying out experiements in natural settings

1. Generalizability 2. Validity- demand characteristic are reduced 3. Participant availability

Explanatory

explanation of a situation, patterns, flexible / fixed

Features of Non-experimental strategy • Samples are individuals from __________ • The samples are allocated to different ____ • Measuring a _____ of variables • _______ other variables • May or may not involve __________

• Samples are individuals from known populations • The samples are allocated to different experimental conditions • Measuring a small number of variables • Controlling other variables • May or may not involve hypothesis testing

Features of a case Study

• Selection of a single case (or a small number of related cases) regarding a situation, an interest, or a concern • Studying the case in its context • Range of data collecting techniques—observation, interview, documentary analysis

Characteristics of real world inquiry (9)

• Solving problems rather than just gaining knowledge • Getting/looking for large effects rather than just relationships between variables • Field rather than laboratory • Outside organization (industry, business, school) rather than laboratory research institution • Strict time constraints rather than as long as the problem needs • Topic initiated by sponsor rather than researcher • Often generalist researchers (familiarity with large range of methods) rather than highly specialist researchers • Multiple methods rather than single method • Need for well developed social skills

Emancipatory Paradigm:

• focus on diverse groups (women, disabled, minorities) • focus on why inequities exist • relationship to political and social action

Realism: Realism

• model of scientific explanation which avoids both positivism and relativism • notes perspectives of participants, promotes social justice • uses the model: the outcome of an action follows from mechanisms acting in particular contexts • external reality exists • embeddedness: human actions can only be understood in terms of their place within different layers of social reality

Relativistic Approach

• scientific accounts/ theories not superior to other approaches • reality is represented through the eyes of the participants • qualitative emphasis • importance of meaning of experience and behavior stressed • research is generating working hypotheses rather than empirical facts

Post-positivism:

• theories, hypotheses, background knowledge and values of the researcher influence what is observed • committed to objectivity • a reality does exist but can only be KNOWN IMPERFECTLY due to researchers' limitations

Critical Realism:

• type of realism that criticizes social practices that it studies

Approaches to Problem Solving (5)

•Traditional approach •Building bridges between researcher and user •Researcher-client equality •Client-professional exploration •Client-dominated quest

Starting where you are in designing a study

- strong interest necessary -background knowledge assists in planning--> experimental knowledge - bias still a risk, experience memo recommended—articulates expectations, beliefs, and understandings you have from previous experience

Laboratory Experimentation may lack these types of realism (2)....

1) Experimental Realism- an experiment is realistic if the situation which it presents to the subject as realistic 2) Mundane Realism- subjects encounter labs that are likely to happen in the real world

Mixed Method Designs

A research approach that uses two or more methods and may include both quantitative and qualitative data.

construct validity

does it measure what you think it measures? (aka face validity)

Single case design focuses on....

individuals rather than groups

Quasi-experiments lack...

random allocation to different conditions

weakness of fixed designs

they cant capture the subtleties and complexities of individual human behavior

Acknowledging the constraints

- choice of research must be realistic in terms of time and resources

citations in research

- citation indices—enable you to travel forward in time from a particular reference via later authors who have cited the initial work. In real world research likely to be very few really central references that you can get hold of and citation indices help to see how others have taken them forward

Deciding of the Research Question

- general focus, activity and involvement, convergence, intuition, theory, real world value - significant research is a process and a way of thinking - Know the area - Widen the base of your experience—research in other fields and from other disciplines - Consider using techniques for enhancing creativity - Avoid pitfalls of allowing a pre decision on method or technique to decide questions to be asked, posing a question that cant be answered, asking questions that have already been answered satisfactorily - Cut it down to size - Think in terms of the purposes of your research—clarifying purpose

Developing a focus for your design

- identifying what it is that you want to gather information about - previous research experience can influence decision but should beware of "straitjacket effect" / allowing these to limit what you look at - own idea, discussion with others, solving a problem, concern or change, curiosity based on media, life..

Making a Group Decision for design

- if working with a group valuable for each to think about write down proposals and then meet as a group to discuss

description of flexible design

- interpretive, ethnographic or qualitative

Having the decision made for you in design

- main task in this situation = clarificatory: translating problem presented into something researchable and do-able within limits of time resources, finance.

problems with flexible design

- rarely "on the cards," researcher often looking for something quite specific while still being open to unexpected discoveries. - Normally quantitative research associated with testing of theories and qualitative associated with the generation of theories but this can be mixed and matched...

Ethical Considerations for design

- refer to rules of conduct, typically to conformity to a code or set of principles - ethics—referring to general principles of what one ought to do - morals—concerned with whether or not a specific act is consistent with accepted notions of right or wrong. o Can follow ethical guidelines and be accused of immoral behavior - Examples: involving people without knowledge, coercion, withholding information about true nature of research, deceiving participant, inducing to commit acts diminishing self esteem, violating rights of self detemrinations, exposing to physical / mental stress, invading privacy, withholding benefits, unfair / disrespectful treatment - Working with vulnerable groups o Children, mental handicap...issue is whether they can knowingly and freely give informed consent

Politics and social research

- research is political based on the fact that we acknowledge that values and value judgments are involved o research commitment to producing knowledge (knowledge preferred to ignorance) o requires resources, allocating them to research represents a value and political choice o founded upon presuppositions reflecting values of the researcher which may derive from gender and ethnicity o has effect on peoples lives through involvement / in a context affected by research findings. Researchers values influence the research o see box 3.9 for more p. 74...(pretty much common sense) - those with power influence which research projects are actually executed. Ex funding provided to certain projects

Conceptual frameworks and realist mechanisms

- the theory about what is going on, what is happening and why, particularly when expressed in diagrammatic form. - Aka concept maps, integrative diagrams, systems or diagrams and conceptual modeling - Talking in terms of hypotheses best fits fixed design research where we should be in a position to make predictions before the data is gathered

Researching the background in making a design

- thorough and up to date understanding of literature, theoretical or methodological concerns - citation indices—enable you to travel forward in time from a particular reference via later authors who have cited the initial work. In real world research likely to be very few really central references that you can get hold of and citation indices help to see how others have taken them forward - databases—key word searches... - Documenting your search, collecting biographical information, networking (through conferences and meetings of professional associations who have done research in this area can give good leads).

2 strategies on how to deal with internal validity

-history -testing -instrumentation -regression (if atypical results the first time, then might not get these atypical scores again) -mortality- dropping out -maturation-growth -selection- initial differences between groups prior to involvement in the enquiry -selection my maturation interaction-predisposition of groups growing apart -Ambiguity about causal direction (does A cause B or does B cause A?) -Difusion of treatments-when one group finds out info or treatment that was only meant for second group -compensatory equalization of treatment- if one group recieved "special" treatment, there will be pressures to give it to the other group -compensatory rivalry- when an individual sees themselves under threat bc of other group comparison and therefore improves performance

Threats to internal validity

1. If you know what the threat is, you can take the specific steps needed to dal with them 2. Randomization

Causes of unreliability

1. PARTICIPANT ERROR- the participant might perform differently in different circumstances (ex: not sleeping the night before) or PARTICIPANT BIAS- where ps might want to please of help researcher , messing up results 2. OBSERVER BIAS-causes problems in interpretation

Components for Research Design and sample questions:

1. Purpose: what is the study you are trying to achieve? Why is it being done? 2. Theory: What theory will guide your study? How will you understand the findings? 3. Research Questions: What questions are geared to providing answers? What do we need to know to achieve the purposes of the study? 4. Methods: What specific techniques will you need to collect data? How will data be analyzed? How do you show data is trustworthy? 5. Sampling Strategy: From whom will you seek data? Where and When? In flexible designs, there should be a repeated revisiting of all aspects of these components→ The framework of design emerges throughout the study

Positivisim 1.objective knowledge (facts) can be gained from ____________ 2.facts without subjectivity=________ 3.science is based on _____ derived from strict procedures 4. scientific positions are founded on _____ 5. purpose is to develop ________ 6. cause is established through ______. 7. explaining an event is relating it to a ______. 8. can ____1_____ and methods of natural science to _____2_____.

1. direct experience/ observation 2.value-free 3. quantitative data (fundamentally different from common sense) 4.facts: hypotheses are tested against these facts 5.universal causal laws 6.empirical regularities or constant conjunctions: causation via empirical patterns 7.general law 8. transfer assumptions/social sciences

Critiques of Positivism ('standard view'): 1. claim that ______ can provide basis for ______ 2. claim that science only deals with ______ 3. impossible to distinguish between _____ and _______ 4. theoretical concepts do not have ________ with _____ 5. scientific laws NOT based on ______ between events in world 6. _______ cannot be separated

1. direct experience/scientific knowledge 2. observable phenomena 3. language of observation and theory 4. a 1:1 correspondence with reality 5. constant conjunction 6. facts and values

Critiques of Positivism within Social Research: 1._____ exist not 'out there' but within ______ 2.___ cannot be defined objectively 3.overemphasis on _____ 4.standardization is______ -->_________

1.social phenomena/minds of people and interpretations 2.reality 3.quantitative 4.unrealistic- personal involvement of researcher is necessary

Fixed designs

A research method where pre-specification about what you are doing and how you are doing it is completed prior to the main part of the research study begins. This requires an understanding of theories involved in the research so you know what to look for in the study. Experiments and surveys are two examples of fixed designs. Considered to be "scientific" in comparison to flexible because they rely on quantitative data and statistical generalization.

Flexible designs

A research strategy where the research design develops during the data collection and analysis process. Almost always includes the collection of qualitative data, but can involve quantitative.

Design for experiment (steps)

A) Deciding on the Focus B) Making a Group Decision C) Having the decision made for you D) Starting where you are E) Researching the background F) Acknowledging the constraint

Action research

Action research: Research oriented to bringing about CHANGE. Researchers are actively involved with the situation or phenomena being studied -In some cases it is important that researchers only communicate their understanding of what is going on to the participants but don't get more involved than that.

Grounded theory studies

Central Aim- to generate theory from data collecting during the study (particularly in new, applied areas where there is a lack of theory and concepts)

Client-dominated quest-->approach to problem solving

Client requests help from a specialist or colleague with social science background. Specialist examines problem and then suggests line of action

Constructivism

Constructivism: • naturalistic and interpretive • qualitative • use interviews and observation to acquire multiple perspectives (realities) --> emphasizes the active role of learners in building their own understanding of reality

Bias in experiments

Demand characteristics- when subjects know they are in an experimental condition an are being observed, they figure out that certain things are expected from them... and certain ps may answer in a way that makes them look better Expectancy effects- reactive effects produced by the experimenters, who have been shown to have biased findings in the direction of what they expect to happen

Case Study

Development of detailed, intensive knowledge about a single "case", or of a number of related "cases". Details of design emerge through data collection and analysis

Ethnographic Study

Development of detailed, intensive knowledge about a single "case", or of a number of related "cases". Details of design emerge through data collection and analysis Seeks to capture, interpret, and explain how a specific group, organization, or community live, experience and make sense of their own lives and the world

Feminist Research:

Feminist Research: • focus on gender imblanaces- female lack of power, privilege, oppressive society • more equal relationship between researcher and participant (so as to not recreate power relationship of males to females) • criticized for ignoring other imbalances

Non-experimental Strategy

Overall Approach is the same as the experimental strategy, but researcher doesn't intend to change the situation, circumstances, or experience of participants. • The Details of design are fully pre-specified before data collecting begins

Experimental Strategy

The Researcher deliberately introduces some form of change in the situation, circumstance, or experience of the participants with intention to change their behavior as a result.

What flexible designs are called and when it is not appropriate to use this name

This approach is sometimes referred to as qualitative designs but this isn't accurate when quantitative data is used.

Research Questions influence choice of strategy...

a. "What?" questions ("how many?", "how much", Who, and where) all suggest non-experimental fixed strategies (i.e. a survey) b. "What?" questions (What is going on here?) are flexible designs c. "How?" more open to either

If a proposed study is an EVALUATION..

a. Are you trying to establish worth or value of something like an intervention, innovation, or service? If so... i. Flexible designs→ mostly focus on process ii. Fixed designs → mostly focus on outcomes

C. The Purpose(s) help in selecting a strategy a. Flexible (qualitative) → ______ b. Non-experimental fixed strategies→ ___ c. Experiments→ ______ d. Real world studies are very commonly _____ based (assessing the value or worth of something) e. To initiate change or involve others→ ___

a. Flexible (qualitative) → Exploratory Work b. Non-experimental fixed strategies→ descriptive strategies c. Experiments→ explanatory studies d. Real world studies are very commonly evaluation based (assessing the value or worth of something) e. To initiate change or involve others→ Action strategies

Action Research

a.Typically involves direct participation b.The study intends to initiate change in the focused problem → involvement and improvement of issue at hand

advantage of fixed designs

ability to transcend individual differences and identify patterns and processes which can be linked to social structures and group or organizational features

Non-experimental fixed designs lack.....

active manipulation of the situation

sexual dichotomism

another instance of double standards treating the sexes as two entirely distinct social groups rather than as groups with overlapping characteristics.

Objectivity

being objective in research We want intersubjective agreement -we want an involved rather than detached investigator, but we also be very objective (no subjective bias opinions in research)

sex appropriateness

common form of double standards eg that child rearing is necessarily female activity

Emancipatory

create opportunities and the will to engage in social action. Exclusively flexible design

double standards

evaluating treating or measuring identical behaviors traits or situations by different means for males and females. Eg using female derived categories of social status for males

experimenter effect

experimenters have to remain very detached because beliefs values and expectations can influence process and results of study

Problem with fixed design

firm theoretical base that is called for often difficult to get hold of

familism

gender insensitivity consists of treating the family as the smallest unit of analysis when it would be possible and appropriate to treat individual as unit

predictive criterion validity

how well does it predict the performance on the criterion in question (how will test scores predict where students will be in 20 years

embeddedness

human actions can only be understood in terms of their place within different layers of social reality

Fixed Design Strategy

i. Calls for tight pre-specification before reaching the main data collection stage of the research. (Don't use the fixed approach if you can't pre-specify) ii. Often referred to as a quantitative strategy→ data is almost always in the form of numbers

Flexible design Strategy

i. Evolves during data collection ii. Referred to as a qualitative strategy→ Data is usually non-numerical A design can't be flexible and fixed at the same time, but can have a flexible phase followed by a fixed phase (called a multiple design). It can also be a fixed design with a flexible element.

internal validity

if a study can demonstrate a CAUSAL relationship between treatment and outcome

Validity

if the findings are really about what they appear to be about

gender insensitivity

ignoring sex as a possible variable

Combined strategy design

initial flexible design stage of primarily exploratory purpose before second fixed design phase

Combined Strategy Design

initial flexible design stage that is exploratory, pursues discussion with those involved or who know about study and gather information before second phase of highly focused fixed experiment

Positivism

looks for the constant relationship between two variables • objective knowledge (facts) can be gained from direct experience/ observation

Experiment-Speak

measuring the effects of manipulating one variable with another variable (and design is not yet pre-specified... like a pilot phase)

Client-professional exploration--> Approach to Problem Solving

o Client with a problem requests help from a researcher. Collection of new data is minimal. Advice/recommendation is based on researchers past experiences/ knowledge of the field.

Design: Think in terms of the purposes of your research—clarifying purpose

o Exploratory—what's happening, new insight, ask questions, assess phenomena in a new light, generate ideas for future research o Descriptive—portray accurate profile of persons events or situations, flexible or fixed o Explanatory—explanation of a situation, patterns, flexible / fixed o Emancipatory—create opportunities and the will to engage in social action. Exclusively flexible design

Building bridges between researcher and user--> Approach to Problem Solving

o Researcher believes work has practical implications and should be used, tries to present results in a accessible language and manner. o Researcher uses client collaboration, but provides feedback

True Design

o Two or more groups are set up o Random allocation of participants to groups o Manipulation for different treatments

Ethical reporting of research

o Values and value judgments closely linked to morals o Relations to scientific attitude and positivism discussed in ch.2...value judgments involved in almost every part of research even if we don't realize it

Theory verification vs. theory generation research

o Verification—positivist methodology which has traditionally formed the basis for fixed design experimental studies starts with a theory, deduces formal hypotheses from it and designs the study to test these. o Flexible design research quintessentially in grounded theory studies is theory generation. Aim to end up with a theory o What's most appropriate depends on particular circumstances and context of your research

Fixed designs should always be... (and Confirmatory task)

piloted (do a mini-version of experiment before committing to the big one) confirmatory task-what you are going to do with your data should be thought through in advance ... and there is nothing stopping you from carrying out exploratory data analysis to look for unexpected pattens

Descriptive purpose

portray accurate profile of persons events or situations, flexible or fixed

Straightjacket effect

previous research experience can influence decision but should beware of "straitjacket effect" / allowing these to limit what you look at

Scientific Attitude

research that is carried out systematically, skeptically, and ethically • systematically: being explicit about nature or observations, the circumstances in which they are made, and our role you take in making them (biases) • skeptically: accepting possible disconfirmation and scrutiny (by yourself and others) • ethically: follow code of conduct for research

Credibility

responsibility for making sure that everything you did was accurate in your study

Reliability

the consistency or stability of a measure (if study were repeated, would we get the same results?

Generalizability

the extent to which the findings of the enquiry are more generally applicable outside of specifics of the situation studied

Fixed Designs are ______......

theory-driven ⇒ variables are specified in advance, be knowledgeable on phenomenon, follows a model that emphasizes how some variables have direct or indirect effects on others • Should always be piloted • Regarded as a confirmatory task

clarificatory

translating problem presented into something researchable and do-able within limits of time resources, finance.

androcentricity

viewing the world from a male perspective. Ex when a test or other research instrument is developed and tested don males and then assumed to be suitable for use with females. (opposite = gynocentricity)

Exploratory purpose

what's happening, new insight, ask questions, assess phenomena in a new light, generate ideas for future research

overgeneralization

when a study deals with only one sex but presents itself as generally applicable (overspecificty can occur when single sex term used when both sexes involved

repeated measures

when individuals performance are compared under two or more conditions

Features of Grounded Theory Studies

• Applicable to a wide variety of phenomena • Commonly interview-based • Systematic, but flexible research strategy which provides detailed prescriptions for data analysis and theory generation

Researcher-client equality--> Approach to Problem Solving

• Researcher-client equality o Together researcher and client discuss problem area and form research design together.

Features of Experimental Strategy • Samples are individuals from _______ ____ • The samples are allocated to _______ • Plan to change _______ • Measuring _________ • ________ other variables • Involves _________ testing

• Samples are individuals from a known population • The samples are allocated to different experimental conditions • Plan to change one or more variables • Measuring a small number of variables • Controlling other variables • Involves hypothesis testing

Features of an ethnographic study

• Selection of a group, organization, or community of interest or concern • Immersion of researcher in that setting • Uses participant observation

Traditional Approach to Problem Solving

•Traditional approach- 'science only': o Basic research- application to problem solving in the real world not seen as an objective. o Less basic- application is not a high priority and usually left to others. o Research on practical problems- application seen as possible but not a necessary outcome.

The Place of Theory

- formal large scale systems developed in academic disciplines to informal hunches or speculations - may be able to make some small contribution to development of the theory itself.

Problems with carrying out field experiments

1. Random assignment 2. Validity 3. Ethical issues 4. Control

Threats to Generalizability (external validity)

1. Selection- finding being specific to the group 2. Setting- findings being specific to, or dependant on, the particular context in which the study took place 3. History- specific and unique historical experiences may determine or affect the findings 4. Construct effects- the particular constructs studied may be only specific to the group studied

insider vs. outsider dilemma:

Part of Action Research some argue that insiders can't carry out any credible or objective research when they are centrally involved. But others using collaborative, participatory action research, or participatory evaluation argue that outsider research is ineffective in terms of enacting change and development.