Chapter 4 - Methodological Issues
Interview schedules
a list of questions an interviewer prepares prior to an interview to be asked orally Done by researcher
Participant bias
a tendency for research participants to respond in a certain way because they know they are being observed, or they believe they know what the researcher wants
Researcher Bias
a tendency for researchers to engage in behaviors and selectively notice evidence that supports their hypotheses or expectations
Manipulated variables
are controlled, created, applied, enforced, or enacted by the researcher.
Researcher trustworthiness
because qualitative research does not involve the level of objectivity as quantitative studies, extra steps should be taken to establish validity in qualitative methodology
Halo effect
Tendency to allow one trait (usually irrelevant to research purposes) of a person to influence how you view other traits (usually relevant to research) of a person. Ex: attractiveness, sex, race, social class
Quantitative population sample
how many participants are needed to find significance that exists, without reporting significance that should not be detected. 300 may be adequate, 500 better, 1000 even better - Blaikie
Self-checklists
list of characteristics or activities for participants to check if it pertains to them
Methodology of a study
determines the exact steps to be taken to conduct the research
Quantitative Studies goal
discerning relationship, effects, and predictions and making generalizations to the larger population
Reflexive journaling (researcher trustworthiness)
helps qualitative researchers through the process of self-awareness, representing others, and understanding how knowledge is generated. These components significantly impact the analysis process
Replication
repeating the essence of a research study, usually with different participants in different situations, to see whether the basic finding extends to other participants and circumstances
Existing counseling literature is based on (demographics)
white, middle class, educated participants
Categorical variables
(qualitative variables) those that divide subjects into groups, but do not allow any sort of mathematical operations to be performed on the data
Summary of participant selection process needs to include:
- Definition of your target population - Description of overall accessible population - Description of your study participants - including number and any relevant demographic characteristics
When choosing an instrument consider
- consider how long it is/how long to complete it - its readability; will sample be able to understand it? - if you will have to make any changes to it; if so the appropriateness of its administration procedures; appropriateness for answering research questions; its psychometric properties; has it been validated for specific sample you are using - are directions for administration/scoring clear - the cost - does an equivalent instrument exist - does it require you or someone else to report info - is it paper-pencil version or based on performance behaviors - have others reported problems with this instrument?
Discussion on instrument choice should include
- description of constructs and variables it measures - how it is laid out in format - how it is scored - what different scores mean - info about reliability and validity
Considering diversity in research, consider the impact of culture on
- language and communications styles - willingness to discuss sensitive topics - level of trust in authority figures/researchers - conceptualization of mental and physical illness, - help seeking
Considering multicultural sampling you may
- purposeful sampling to include minorities - choose instruments that have adequate norming with minority pops - develop your own instruments - undertake measurement validation procedures for pre-established instruments using non majority participants
Common Methodological Errors
1. Failing to clearly define research population 2. Not collecting a large enough sample 3. Using the most convenient method of data collection rather than the best 4. Attempting to go through all stages of the study within one semester 5. Using inadequate or inappropriate instruments 6. Generally lacking a well-thought-out methodological plan of action 7. Overgeneralizing, or generalizing to a population different from the one sampled 8. Inadequately addressing issues of bias
Theoretical population
A group which, ideally, you would like to sample from and generalize to. This is usually contrasted with the accessible population.
Observation forms
A list of specific behaviors for the researcher to observe and evaluate, and to provide place to document frequency of said behaviors Done by researcher
Variables
Any trait, attribute, or characteristic that varies.
How to determine sample size in Qualitative designs
Do you anticipate finding a - Large effect? Then smaller sample size will work. - Small effect? Need a larger group to be able to generalize Review existing studies that used similar variables, statistical methods Pay attention to limitations to sample (size) in these studies
Purposeful sampling
Qualitative designs; choosing a meaningful sample that fits the parameters of the project's research questions and goals
Anecdotal records
Records containing specific and factual recordings, usually in paragraph form, of observations deemed important to the researcher Done by researcher
Independent variable
The experimental factor that is manipulated; the variable whose effect is being studied. Causes change.
Dependent variable
The measurable effect, outcome, or response in which the research is interested. Change depends on the other variable
Accessible Population
The portion of the target population that is accessible to the researcher, from which a sample is drawn
John Henry effect
The tendency for people based in a control group to increase performance AND outperform experimental group. Rivalry and competitiveness are the motives
Steps to establish validity in qualitative studies (researcher trustworthiness)
Triangulation, member checking, and reflexive journaling
Member checking (researcher trustworthiness)
When participants provide feedback regarding findings and make recommendations on interpretations
Sampling Bias
a flawed sampling process that produces an unrepresentative sample convenience sampling methods volunteerism access to populations (addictions) Where and when we collect data can bias the sample
Attitude scales
a set of statements that represent certain attitudes, depending on which ones he participant endorses
Flowcharts
a sheet for recording frequency counts of a behavior, as well as the intended direction of the behavior Done by researcher
Performance checklists
checklist to record IF a person is engaging in behaviors typically associated with performing a particular task Done by researcher
Instrumentation
entire process of data collection - instrument selection to instrument administration
Experimenter or observer expectancies
inadvertent behavior cues from experimenter that tips people off of what their expectations are. Self fulfilling prophecy - Pygmalion effect Shaping data outcomes in direction experimenter chooses
Questionaires
includes a list of questions that participant answers in writing or marking appropriate response completed by research participant
Triangulation (researcher trustworthiness)
incorporates multiple sources of data, methods, researchers, or theories
Qualitative population sample
is usually smaller due to the nature of the design
Qualitative studies goal
meaningful experiences among individuals can be interpreted towards a larger phenomenological idea
Achievement and Aptitude tests
measure a participant's knowledge, ability or propensity to achieve
Performance tests
measure ability to perform on a certain task
Personality Inventories
measures the characteristics and traits of the participant
Instrument
mechanism used to collect data
Selected Variables
observed as they occur or after they have occurred without researcher intervention
Operationalize
operationally define a variable to state its definition for purpose of the study, but also to state how it will be identified and measured
Rating Scales
provide a score in relation to how a person behaves; imply judgement rather than acknowledgment Done by researcher
Numerical variables
quantitative measurements that have magnitude on a numerical scale
Hawthorne effect
the tendency for people to behave differently when they know they are being studied
Extraneous Variables
undesirable variables that influence the outcome of the study. Goal of study is to control for these variables as much as possible
Experimenter effects
unintended changes in subjects' behavior due to cues inadvertently given by the experimenter
Time-and-motion logs
very detailed observations of a person or group that occur over a specified period of time in an effort to understand underlying reasons for behavior Done by researcher