Chapter 4 - Methodological Issues

Interview schedules

a list of questions an interviewer prepares prior to an interview to be asked orally Done by researcher

Participant bias

a tendency for research participants to respond in a certain way because they know they are being observed, or they believe they know what the researcher wants

Researcher Bias

a tendency for researchers to engage in behaviors and selectively notice evidence that supports their hypotheses or expectations

Manipulated variables

are controlled, created, applied, enforced, or enacted by the researcher.

Researcher trustworthiness

because qualitative research does not involve the level of objectivity as quantitative studies, extra steps should be taken to establish validity in qualitative methodology

Halo effect

Tendency to allow one trait (usually irrelevant to research purposes) of a person to influence how you view other traits (usually relevant to research) of a person. Ex: attractiveness, sex, race, social class

Quantitative population sample

how many participants are needed to find significance that exists, without reporting significance that should not be detected. 300 may be adequate, 500 better, 1000 even better - Blaikie

Self-checklists

list of characteristics or activities for participants to check if it pertains to them

Methodology of a study

determines the exact steps to be taken to conduct the research

Quantitative Studies goal

discerning relationship, effects, and predictions and making generalizations to the larger population

Reflexive journaling (researcher trustworthiness)

helps qualitative researchers through the process of self-awareness, representing others, and understanding how knowledge is generated. These components significantly impact the analysis process

Replication

repeating the essence of a research study, usually with different participants in different situations, to see whether the basic finding extends to other participants and circumstances

Existing counseling literature is based on (demographics)

white, middle class, educated participants

Categorical variables

(qualitative variables) those that divide subjects into groups, but do not allow any sort of mathematical operations to be performed on the data

Summary of participant selection process needs to include:

- Definition of your target population - Description of overall accessible population - Description of your study participants - including number and any relevant demographic characteristics

When choosing an instrument consider

- consider how long it is/how long to complete it - its readability; will sample be able to understand it? - if you will have to make any changes to it; if so the appropriateness of its administration procedures; appropriateness for answering research questions; its psychometric properties; has it been validated for specific sample you are using - are directions for administration/scoring clear - the cost - does an equivalent instrument exist - does it require you or someone else to report info - is it paper-pencil version or based on performance behaviors - have others reported problems with this instrument?

Discussion on instrument choice should include

- description of constructs and variables it measures - how it is laid out in format - how it is scored - what different scores mean - info about reliability and validity

Considering diversity in research, consider the impact of culture on

- language and communications styles - willingness to discuss sensitive topics - level of trust in authority figures/researchers - conceptualization of mental and physical illness, - help seeking

Considering multicultural sampling you may

- purposeful sampling to include minorities - choose instruments that have adequate norming with minority pops - develop your own instruments - undertake measurement validation procedures for pre-established instruments using non majority participants

Common Methodological Errors

1. Failing to clearly define research population 2. Not collecting a large enough sample 3. Using the most convenient method of data collection rather than the best 4. Attempting to go through all stages of the study within one semester 5. Using inadequate or inappropriate instruments 6. Generally lacking a well-thought-out methodological plan of action 7. Overgeneralizing, or generalizing to a population different from the one sampled 8. Inadequately addressing issues of bias

Theoretical population

A group which, ideally, you would like to sample from and generalize to. This is usually contrasted with the accessible population.

Observation forms

A list of specific behaviors for the researcher to observe and evaluate, and to provide place to document frequency of said behaviors Done by researcher

Variables

Any trait, attribute, or characteristic that varies.

How to determine sample size in Qualitative designs

Do you anticipate finding a - Large effect? Then smaller sample size will work. - Small effect? Need a larger group to be able to generalize Review existing studies that used similar variables, statistical methods Pay attention to limitations to sample (size) in these studies

Purposeful sampling

Qualitative designs; choosing a meaningful sample that fits the parameters of the project's research questions and goals

Anecdotal records

Records containing specific and factual recordings, usually in paragraph form, of observations deemed important to the researcher Done by researcher

Independent variable

The experimental factor that is manipulated; the variable whose effect is being studied. Causes change.

Dependent variable

The measurable effect, outcome, or response in which the research is interested. Change depends on the other variable

Accessible Population

The portion of the target population that is accessible to the researcher, from which a sample is drawn

John Henry effect

The tendency for people based in a control group to increase performance AND outperform experimental group. Rivalry and competitiveness are the motives

Steps to establish validity in qualitative studies (researcher trustworthiness)

Triangulation, member checking, and reflexive journaling

Member checking (researcher trustworthiness)

When participants provide feedback regarding findings and make recommendations on interpretations

Sampling Bias

a flawed sampling process that produces an unrepresentative sample convenience sampling methods volunteerism access to populations (addictions) Where and when we collect data can bias the sample

Attitude scales

a set of statements that represent certain attitudes, depending on which ones he participant endorses

Flowcharts

a sheet for recording frequency counts of a behavior, as well as the intended direction of the behavior Done by researcher

Performance checklists

checklist to record IF a person is engaging in behaviors typically associated with performing a particular task Done by researcher

Instrumentation

entire process of data collection - instrument selection to instrument administration

Experimenter or observer expectancies

inadvertent behavior cues from experimenter that tips people off of what their expectations are. Self fulfilling prophecy - Pygmalion effect Shaping data outcomes in direction experimenter chooses

Questionaires

includes a list of questions that participant answers in writing or marking appropriate response completed by research participant

Triangulation (researcher trustworthiness)

incorporates multiple sources of data, methods, researchers, or theories

Qualitative population sample

is usually smaller due to the nature of the design

Qualitative studies goal

meaningful experiences among individuals can be interpreted towards a larger phenomenological idea

Achievement and Aptitude tests

measure a participant's knowledge, ability or propensity to achieve

Performance tests

measure ability to perform on a certain task

Personality Inventories

measures the characteristics and traits of the participant

Instrument

mechanism used to collect data

Selected Variables

observed as they occur or after they have occurred without researcher intervention

Operationalize

operationally define a variable to state its definition for purpose of the study, but also to state how it will be identified and measured

Rating Scales

provide a score in relation to how a person behaves; imply judgement rather than acknowledgment Done by researcher

Numerical variables

quantitative measurements that have magnitude on a numerical scale

Hawthorne effect

the tendency for people to behave differently when they know they are being studied

Extraneous Variables

undesirable variables that influence the outcome of the study. Goal of study is to control for these variables as much as possible

Experimenter effects

unintended changes in subjects' behavior due to cues inadvertently given by the experimenter

Time-and-motion logs

very detailed observations of a person or group that occur over a specified period of time in an effort to understand underlying reasons for behavior Done by researcher