Methods in Scientific Research
Introduction to the Scientific Process
a logical, problem-solving technique have a question, follow a process to get a good answer
Which to use? active or passive
active
HA
alternative hypothesis, temperature has an effect on batting
The example that follows is based on a study by Darley and Latané (1969). The authors were interested in whether the presence of other people has an influence on whether a person will help someone in distress. In this classic study, the experimenter (a female graduate student) had the subject wait in a room with either 0, 2, or 4 participants. The experimenter announces that the study will begin shortly and walks into an adjacent room. In a few moments the person(s) in the waiting room hear her fall and complain of ankle pain. The dependent measure is the number of seconds it takes the subject to help the experimenter. Name the test scenario:
ANOVA
an analysis of variation in an experiment is a test of the hypothesis that the variation in an experiment is no greater than that due to normal variation of individuals' characteristics and error in their measurement name the test:
ANOVA
6th step in Scientific Process
Analyze data
7th step in Scientific Process
Draw conclusions: reject or accept our hypotheses (use conclusions to develop new hypothesis)
3rd step in Scientific Process
Formulate a Hypothesis: "educated guess", review of literature
2nd step in Scientific Process
Gather information: review of literature, look for other experiments
When you finish writing, ask:
Have I said it?
step 3: formulate a hypothesis
Hypothesis: possible answer to a question that can be tested based on observations and knowledge "If then" statement
1st step in Scientific Process
Identify a Problem: specific, narrow, introduction and review of literature
Following the RER test: The medians show that, on average, caffeine appears to have reduced RER from about 98% to 94%, a reduction of 6%. However, there is a great deal of variation between the data values in both samples and considerable overlap between them. So is the difference between the two medians simply due to sampling variation or does the data provide evidence that caffeine does, on average, reduce RER? Name the test scenario:
Mann Whitney test
H0: population medians are equal H1 (non-directional alternative): population medians are not equal
Mann Whitney test
nonparametric equivalent to the independent samples t-test
Mann Whitney test
Step 4: develop and experiment
Materials and Procedure
Correlation is a technique for investigating the relationship between two quantitative, continuous variables, for example, age and blood pressure. ______________ _______________ ______________ (r) is a measure of the strength of the association between the two variables.
Pearson's Correlation Coefficient
Nine students held their breath, once after breathing normally and relaxing for one minute, and once after hyperventilating for one minute. The table indicates how long (in sec) they were able to hold their breath. ____________ _______________ _______________ attempts to answer whether or not there is an association between the two variables, and if so, how strong the association is.
Pearson's Correlation Coefficient
The first step in studying the relationship between two continuous variables is to draw a scatter plot of the variables to check for linearity. The correlation coefficient should not be calculated if the relationship is not linear. For correlation only purposes, it does not really matter on which axis the variables are plotted. However, conventionally, the independent (or explanatory) variable is plotted on the x-axis (horizontally) and the dependent (or response) variable is plotted on the y-axis (vertically). Name the test scenario:
Pearson's Correlation Coefficient
For example, two doctors may assess the condition of eight patients suffering from particular symptoms. To do this they rank the patients from 1 (best) to 8 (worst). A significant association between the sets of ranks by calculating ___________'s rank correlation coefficient (rs) is indicated by p = 0.05, as usual. The value of rs ranges from -1 to +1. Results indicate that there is evidence to suggest good agreement (rs = 0.86) between the doctors' assessments (p = 0.007). Name the test scenario:
Spearman's Coefficient
5th step in Scientific Process
Record and Organize data
Words:
Reduce dead weight words and phrases: get rid of jargon and repetition cut, cut, cut: learn to part with your words watch out for "to be", "there are", very, really, quite, etc. Be specific
ordered categories chronological ranks and do calculations based on them how closely two sets of rankings are related strength of association ***************for my proposal, most conspiracy to least
Spearman's Coefficient
The null and alternative hypotheses are the same as those for the paired samples t-test although they are often stated in terms of the median, thus: H0: in the population the median difference is zero H1: in the population the median difference is not zero
Wilcoxon
an assumption behind this test is that the population distributions are symmetrical, in which case the means and medians are identical
Wilcoxon
nonparametric equivalent to paired t-test
Wilcoxon
Pay attention to your _____! Strip your sentences to just the _____ that ____.
Words! words that tell
Qualitative Traditions: Narrative Designs
focuses on individual experiences reports a chronology of the experiences (use a time sequence of events, chronology sets narrative apart) collects the individual stories told to the researcher or gathered through field texts (autobiographies, interviews, journals) Restories the individual stories (researcher gathers stories and analyzes them for elements of the story, researcher rewrites the story to place it in a chronological sequence, restorying provides a casual link among ideas, info would include interaction, continuity, and situation) Describes the context or setting for the individual stories ( people involved, physical setting, setting may described before events or actions, or can be woven throughout the study) Corroborates throughout the process of research with the individuals whose stories are being reported (participants are activelly involved in the inquiry as it unfolds, relationships between research and participant are negotiated to minimize the potential gap)
Sentences:
follow subject + verb+object (SVO) use strong verbs eliminate negatives, use positive constructions instead
Grounded theory: how does this process work?
general or abstract theory derived from process, action, or interaction views "grounded" in views of participants multiple stages of data collection constant comparison of data theoretical sampling of groups to maximize similarities and differences
Read, pay attention, and intimate
good writing
does not fall into any other literature category the MO dept. of conservation government resources pamphlets
gray literature
choose this when you want to develop a theory grounded in the views of participants
grounded theory
study when the researcher inductively develops or elaborates on a theory by collecting primarily interview data, making multiple visits to the field (theoretical sampling), attempting to develop and interrelate categories (constant comparison) of information, and writing a discursive theory
grounded theory
Some components of a good writer: 3
having something to say logical and clear thinking few simple, learnable rules of style
possible answer to a question that can be tested
hypothesis
Steps of the Scientific Process: 7
identify a problem gather information formulate hypothesis design and experiment record and organize data analyze data draw conclusions (use conclusions to develop new hypothesis)
Dead weight phrases
in the event of in the nature of it has been estimated that it seems that
The variable I (the scientist) change or manipulate
independent variable
a logical interpretation of events based on prior knowledge or opinion, educated guess
inference
A set of measurements can almost always be regarded as measurements on a sample of items from a population of these items, as it is usually impractical or impossible to measure every item in the population. Thus we have to make inferences about the population from the sample.
inferential statistics
sample population sample size-the larger the better
inferential statistics
Independent or paired t-test Chi-squared (difference or association) Spearman Pearson Mann-Whitney Wilcoxon ANOVA
inferential statistics tests
Although 95% confidence intervals are most often reported, you will sometimes see 99% confidence intervals, in which case the confidence interval contains the population parameter with probability 0.99 and will, consequently, be wider than the corresponding 95% confidence interval.
inferential statistics: confidence
This can only be true if the sample is representative of the population, and even then the sample is very unlikely to reflect the population exactly in all respects. That is, there is uncertainty as to how well the sample results reflect the population. Statistical methods have been developed to reduce and quantify this uncertainty.
inferential stats: sample population
To avoid bias the sample items should be selected from the population at random. This means that all members of the population have an equal chance of being in the sample.
inferential stats: sample population
Criteria for choosing a tradition: 4
intent or focus audience personal training personal comfort level
Coding Process
interview transcribe categorize repeated patterns
Qualitative research
involves people in some way, questions about experiences, lots of samples, take data and make big inferences, deep interaction with participants
often used in science published by those involved in the research
journals
a list of all things you need, supplies
materials
Materials and Methods must include: 5 things
materials, methods, IV, DV, control
3 types of measures of central tendency (descriptive statistic)
mean, median, mode
average
mean- measure of CT (descriptive stat)
assign actual number to things, measurable and quantitative height, weight, age
measurement variables
descriptive statistics: averages
measures of central tendency
2 types of descriptive statistics
measures of central tendency measures of dispersion
stats that describe how data varies, or is dispersed show how data differs
measures of dispersion (descriptive stat)
middle
median (measure of CT, descriptive stat)
occurs most
mode (measure of CT, descriptive stat)
collaborates throughout the process of research with the individuals whose stories are being reported
narrative design
collects the individual stories told to the researcher or gathered through field texts
narrative design
describes the context or setting of the individual stories
narrative design
focuses on individual experiences
narrative design
reports a chronology of the experiences
narrative design
restories the individual stories
narrative design
What are the 5 Qualitative Traditions?
narrative research, phenomenology, ethnography, grounded theory, case study
H0
null hypothesis, void, had no effect, inapplicable
What is this about? What patterns exist? Coding type:
open coding
Consider a study in which standing and supine systolic blood pressures were compared. This study was performed on twelve subjects. Their blood pressures were measured in both positions. It is therefore, a paired samples design. Name test scenario:
paired t-test
The difference between the means of the samples is unlikely to be equal to zero (due to sampling variation) and the hypothesis test is designed to answer the question "Is the observed difference sufficiently large enough to indicate that the alternative hypothesis is true?".
paired t-test
used to compare the values of means from two related samples, for example in a "before and after" scenario
paired t-test
most widely respected journals are...
peer reviewed
scientific articles are reviewed by other scientists before being accepted for publication
peer reviewed
as much a philosophy of meaning as a method
phenomenology
identify essence of human experience as described by participants, develop patterns or relationship of meaning
phenomenology
researcher puts aside or "brackets" his/her ideas
phenomenology
seeks to understand human experiences through the meaning of experiences held for each individual
phenomenology
uses small number of subjects over prolonged period of time through lengthy interviews
phenomenology
Step 6: analyze data
pie, line, bar graph
measured by calculating the standard error of the estimate or a confidence interval (usually the 95% confidence interval)
precision-sample size of inferential stats
Journal of Experimental Zoology, Journal of the History of biology
primary lit
all papers that present novel scientific research, but also may include literature reviews, perspectives, and analyses that advance new concepts and viewpoints about data generated by others
primary literature
comprises all scientific literature that presents new scientific data or ideas
primary literature
written by scientists who have done the research themselves, generally published in scholarly journals
primary literature
Vague nouns
problem, situation, approach, method, reaction, component, technique, solution, challenge, difficulty
step by step instructions (some knowledge is assumed), identifies variables used in the experiment
procedure
uses our 5 senses to gather info
qualitative observation
uses numbers to gather info
quantitative observation
comprises summaries of results and ideas from the primary literature written for an audience of scientists with some understanding of the topic
secondary lit
journals that only publish review articles
secondary lit
review articles and perspectives in other journals, many journals publish papers that review topics of current interest: papers in Science published as "articles" and "perspectives"
secondary lit
compile primary sources, citations
secondary sources
What implications does it have? coding type:
selective coding
Observation uses our _______ to gather information
senses
2 most commonly used measures of dispersion
standard deviation and range
Research Proposal Application-Introduction should include: 4 things
statement of the problem-not prose purpose of the study research questions and hypotheses (null and alternate for every question) limitations of the study (age, gender, etc)
Qualitative traditions: grounded theory
study when the researcher inductively develops or elaborates on a theory by collecting primarily interview data, making multiple visits to the field (theoretical sampling), attempting to develop and interrelate categories (constant comparison) of information, and writing a discursive theory
scientists within a discipline often meet to present scientific info focused upon a particular topic. these presentations are often published together as a book
symposia, secondary lit
When two samples are taken from the same population it is unlikely that the means of the two samples will differ significantly. When two samples are taken from two different populations, it is likely that the means of the two samples will differ. Our problem is how to differentiate between these two situations using only the data from the two samples
t-test
enables us to determine the probability that two sample means could have been drawn from the same population of numbers if the probability is sufficiently low (5%) we can claim a "statistically significant difference" 2 groups, 1 with intervention and 1 without
t-test
most important statistical test used in relation to biological variation
t-test
used to compare the values of the means from two samples and test whether it is likely that the samples are from populations having different mean values
t-test
Qualitative Traditions: case study
the study of a "bounded system" or a case/multiple cases, over time through detailed, in -depth data collection involving numerous sources and rich in context, in order to present a substantive picture of the case some researchers consider "the case" an object of study, while others consider it a methodology
Qualitative Traditions: Ethnography
the study of an intact cultural group of an individual, using primarily observational methods of data collection, based on a prolonged period of time in the field, listening and recording the voice of informants in order to generate a cultural portrait both a process and a product Key question: what is the culture like?
passive voice
the subject is acted upon, and the subject doesn't act
Step 2: Gather Information
use references to do background research books, journals, magazines, internet, TV, videos, interview, experts
Identify a Problem
using observations, identify a problem you would like to solve This is a question you do not know the answer to and can't look up. If you can google your question and many thorough answers come up, the question needs tweaking. Finding contradictions or someone disproving your theory is ok! "Why" and "what would happen if" are good ways to start
Step 5: record and organize data
write all observations and measurements use table to organize your data IV on left side, DVs on right side if you have more than one DV, use a new column for each DV
What makes good scientific writing? 2 things
communicates an idea clearly and effectively (takes having something to say and clear thinking) elegant and stylish (time, revision, good editor)
a variable that can represent any value on the scale used to measure the variable (typically interval or ratio variable)
continuous variable
number of Coke drinks per week (1, 1.5, 2) "how much" amount
continuous variable
typically interval or ratio variables
continuous variables
the normal condition that you compare the other conditions to, recreate the conditions you first observed
control variable
Key question: what is the culture like?
ethnography
both a process and a product
ethnography
contextual response to "lived" realities in field setting
ethnography
interaction is group in natural setting over prolonged period of time, primarily observational data
ethnography
the study of an intact cultural group of an individual, using primarily observational methods of data collection, based on a prolonged period of time in the field, listening and recording the voice of informants in order to generate a cultural portrait
ethnography
4th step in Scientific Process
Design and Experiment
7 inferential statistics tests
Independent or paired t-test chi-squared (difference or association) Spearman Pearson Mann-Whitney Wilcoxon ANOVA
first hand accounts, citations
Primary sources
Before you start writing, ask:
What am I trying to say?
Qualitative traditions: Phenomenology
as much a philosophy of meaning as a method the researcher seeks to understand human experiences through the meaning of experiences held for each individual to do this, the researcher puts aside, or "brackets" his/her own ideas "what does this mean?" identify "essence of human experience as described by participants" small number of subjects over prolonged period of time through lengthy interviews develop patterns of relationship of meaning
What is the relationship that explains how the process works? Coding type:
axial coding
Asks "what has happened?", explore event, process or 1 or more individuals in depth bounded by time and activity variety of data collection procedures over time
case study
The study of a "bounded system" or a case/multiple cases, over time through detailed, in -depth data collection involving numerous sources and rich in context, in order to present a substantive picture of the case
case study
gender, race assign a number to, but number represents a type rather than an amount, thus qualitative
categorical variables
A group of students were classified in terms of personality (introvert or extrovert) and in terms of color preference (red, yellow, green or blue) with the purpose of seeing whether there is an association (relationship) between personality and color preference. Data was collected from 400 students and presented in the 2 (rows) x 4 (cols) contingency table. Name the test scenario:
chi-square
The sample data is used to calculate a single number (or test statistic), the size of which reflects the probability (p-value) that the observed difference between the 2 variables has occurred by chance, i.e., due to sampling error. The maternity wards of two hospitals had different preparation for childbirth schemes. A study of mothers who had participated in the schemes asked them to assess their satisfaction with the scheme with specific results. To answer the question 'Is there any evidence of a difference in the satisfaction of the mothers between the two schemes at the two hospitals?', the _________ test is used.
chi-square
used to determine whether an association (or relationship) between 2 categorical variables in a sample is likely to reflect a real association between these 2 variables in the population. In the case of 2 variables being compared, the test can also be interpreted as determining if there is a difference between the two variables. also for nominal, categorical data
chi-square
In the case of 2 variables being compared, the test can also be interpreted as determining if there is an association (or relationship) between the two variables.
chi-square test
The sample data is used to calculate a single number (or test statistic), the size of which reflects the probability (p-value) that the observed association between the 2 variables has occurred by chance, i.e. due to sampling error.
chi-square test
for nominal, categorical data: which test? this related to that *************** for my proposal
chi-square test
used to determine whether a difference between 2 categorical variables in a sample is likely to reflect a real difference between these 2 variables in a population *****************for my proposal
chi-square test
test the relationship between your independent variable and dependent variable depends on the 'type' of data you have
choosing the correct statistical test
is measured in the experiment changes because of the independent variable "depends" on the independent variable
dependent variable
sample size, max/min values, averages and measures of variation of the data about the average many studies use this as a first step, prior to inferential statistics
descriptive statistics
only 2 possible outcomes (yes/no, male or female)
dichotomous categorical variables
can only represent a few specified values on a designated scale (typically nominal, ordinal variables)
discreet variables
male (1) or female (2) "how many" type
discreet variables
typically nominal or ordinal variables
discreet variables
2 things to do with hypotheses:
disprove null hypothesis and accept alternative hypothesis The warmer it is, the better the bat performs (directional alternative hypothesis)
A study of the effect of caffeine on muscle metabolism used eighteen male volunteers who each underwent arm exercise tests. Nine of the men were randomly selected to take a capsule containing pure caffeine one hour before the test. The other men received a placebo capsule. During each exercise the subject's respiratory exchange ratio (RER) was measured. (RER is the ratio of CO2 produced to O2 consumed and is an indicator of whether energy is being obtained from carbohydrates or fats). Name the test scenario:
t-test The question of interest to the experimenter was whether, on average, caffeine changes RER. The two populations being compared are "men who have not taken caffeine" and "men who have taken caffeine". If caffeine has no effect on RER the two sets of data can be regarded as having come from the same population
Example of null hypothesis
temperature has no effect on batting
generally written for a nonscientific audience or for scientists in other disciplines
tertiary lit
science magazines, lay magazines, text books
tertiary lit
no citations, use common knowledge
tertiary sources
