Bias in Research

Bias

- Any tendency which prevents unprejudiced consideration of a question or finding. Can occur at any phase of the project.

Sampling Bias

- Participants may be selected (or select themselves), in a way that biases the results.

Researcher Bias (Most common ways)

- Source Bias: The researcher may have not conducted a thorough review of the literature or has selectively chosen/ignored sources providing a biased view of existing evidence. - Design Bias: The researcher may have choses an inappropriate design, data collection technique, setting, or procedure that biases the result. - Sampling Bias: Participants may be selected (or select themselves) in a way that bias the results.

External/Internal Validity

Experimental validity can be broken into two broad categories and a lot of subcategories. Internal validity: The ability to determine if a causal relationship exists between one or more independent variables and one or more dependent variables. (In simple terms, it's asking "Did in fact the IV make a difference in the experiment?" or "Is what was observed true?". External validity: The ability to generalise the findings beyond the sample or study setting. (In other words, "Are the results of the study applicable to the broader target population?".

Experimental Validity

Often when discussing bias and it's effects relation to experimental methods, we discuss them as threats to the validity of the study. Experimental Validity: The manner in which variables that influence both the results of the research and the generalisability to the population at large.

Statistical Bias

The choice of the wrong statistical test or the misuse of tests (e.g., running many comparisons) may cause biased results.

Interviewer/Experimenter Bias

The interviewer (or more generally the person interacting with the participants), can influence the results either their interaction with the participants or through the way the participant reacts to him or her. - Often double-blind controls in which the researcher(s) do not know which group or condition the participants have been assigned to are used to minimise this form of researcher bias.

Participant Bias/Demand Characteristics

The most common form of participant bias comes in the form of various types of demand characteristics. - Demand Characteristics: occur when participants act differently simply because they know that they are in a study. They may try to guess the aims of the study and act accordingly. (In non-experimental designs, demand characteristics are often referred to as participant expectations). One way to minimise participant bias is to keep the true aim (and in the case of an experimental design, the hypothesis) of the study, secret, until after the participant is finished. - This requires some use of deception up front, which is generally addressed during the debriefing procedure. - In other cases, other procedures like cover observation may be used to minimise some forms of bias like the social desirability effect. - Keeping the participant blind as to what condition they are in, is also often used. (With drug studies, this often involves the use of a placebo).

Scorer/Observer/Analysis Bias

The person scoring or analysing the data may influence the results by selectively choosing what they analyse or how they score a particular response. - Double-blind controls work here as well. - Standardised scoring instructions and the use of measurement standards are helpful. - Statistical bias - the choice of the wrong statistical test or the misuse of tests (e.g., running many comparisons) may cause biased results.

Design Bias

The researcher may have been chosen an inappropriate design, data collection technique, setting, or procedure that biases that result. -In experimental designs, flawed designs can introduce confounding variables or lead to problems with reliability.

Common Forms of Demand Characteristics

These are: - Expectancy effect: The participant attempts to discern the experimenter's hypothesis with the goal of "helping" the researcher. This may result in acting a certain way or giving the "right answer". - Screw you effect: The participant attempts to discern the experimenter's hypothesis, but only in order to destroy the credibility or validity of the study. - Social desirability effect: This is when the participant answers or behaves in a way that makes him/her look good to the researcher. This is done to avoid embarrassment or judgement. (Social and cultural norms play a big role here).