Chapter 1

Test-retest Reliability

the ability of a measure to produce consistent results when the same entities are tested at two different points in time.

Falsification

the act of disproving a hypothesis or theory

Deviance

the difference between the observed value of a variable and the value of that variable predicted by a statistical model aka observed value minus the mean

Measurement Error

the discrepancy between the numbers used to represent the thing that we're measuring and the actual value of the thing we're measuring (i.e., the value we would get if we could measure it directly).

Probability Density Function (PDF)

the function that describes the probability of a random variable taking a certain value. It is the mathematical function that describes the probability distribution.

Interquartile Range

the limits within which the middle 50% of an ordered set of observations fall. It is the difference between the value of the upper quartile and lower quartile.

Median

the middle score of a set of ordered observations. When there is an even number of observations the median is the average of the two scores that fall either side of what would be the middle value.

Mode

the most frequently occurring score in a set of data.

Tertium Quid

the possibility that an apparent relationship between two variables is actually caused by the effect of a third variable on them both (often called the third-variable problem)

Randomization

the process of doing things in an unsystematic or random way. In the context of experimental research the word usually applies to the random assignment of participants to different treatment conditions.

Range

the range of scores is the value of the smallest score subtracted from the highest score. It is a measure of the dispersion of a set of scores. See also variance, standard deviation and interquartile range.

Levels of Measurement

the relationship between what is being measured and the numbers obtained on a scale

Z-scores

the value of an observation expressed in standard deviation units. It is calculated by taking the observation, subtracting from it the mean of all observations, and dividing the result by the standard deviation of all observations. By converting a distribution of observations into z-scores a new distribution is created that has a mean of 0 and a standard deviation of 1.

Upper Quartile

the value that cuts off the highest 25% of ordered scores. If the scores are ordered and then divided into two halves at the median, then the upper quartile is the median of the top half of the scores.

Lower Quartile

the value that cuts off the lowest 25% of the data. If the data are ordered and then divided into two halves at the median, then the lower quartile is the median of the lower half of the scores.

Unsystematic Variance

this is variation that isn't due to the effect in which we're interested (so could be due to natural differences between people in different samples such as differences in intelligence or motivation). We can think of this as variation that can't be explained by whatever model we've fitted to the data.

Kurtosis

this measures the degree to which socres cluster in the tails of a frequency distribution. Kurtosis is calculated such that no kurtosis yields a value of 3. To make the measure more intuitive, SPSS Statistics (and some other packages) subtract 3 from the value so that no kurtosis is expressed as 0 and positive and negative kurtosis take on positive and negative values, respectively. A distribution with positive kurtosis (leptokurtic, kurtosis > 0) has too many scores in the tails and is too peaked, whereas a distribution with negative kurtosis (platykurtic, kurtosis < 0) has too few scores in the tails and is quite flat.

Quantiles

values that split a data set into equal portions. Quartiles, for example are a special case of quantiles that split the data into four equal parts. Similarly, percentiles are points that split the data into 100 equal parts and noniles are points that split the data into nine equal parts (you get the general idea)

Systematic Variation

variation due to some genuine effect (be that the effect of an experimenter doing something to all of the participants in one sample but not in other samples or natural variation between sets of variables). We can think of this as variation that can be explained by the model that we've fitted to the data)

Nominal Variable

where numbers merely represent names. For example, the numbers on sports players shirts: a player with the number 1 on her back is not necessarily worse than a player with a 2 on her back. The numbers have no meaning other than denoting the type of player (full back, center forward, etc.).

What are (broadly speaking) the five stages of the research process?

1. Generating a research question: through an initial observation (hopefully back up by some data) 2. Generate a theory to explain your initial observation. 3. Generate hypotheses: break your theory down into a set of testable predictions. 4. Collect data to test the theory: decide on what variables you need to measure to test your predictions and how best to measure or manipulate those variables. 5. Analyze the data: look at the data visually and by fitting a statistical model to see if it supports your predications (and therefore your theory). At this point you should return to your theory and revise it if necessary

Assuming the same mean (39.68) and standard deviation (7.74) for the ice bucket example above, what's the probability that someone posted a video within the first 30 days of the challenge?

1st convert 30 to a z-score: (30-39.68)/7.74 = -1.25 We want the area below this value (because 30 is below the mean), but this value is not tabulated in the Appendix. However, because the distribution is symmetrical, we could instead ignore the minus sign and look up this value in the column labelled 'Smaller Portion' (i.e. the area above the value 1.25). You should find that the probability is 0.10565, or, put another way, a 10.57% chance that a video would be posted within the first 30 days of the challenge. By looking at the column labelled 'Bigger Portion' we can also see the probability that a video would be posted after the first 30 days of the challenge. This probability is 0.89435, or a 89.44% chance that a video would be posted after the first 30 days of the challenge.

Compute the mean of: 22 40 53 57 93 98 103 108 116 121

81.1

Say I own 857 CDs. My friend has written a computer program that uses a webcam to scan the shelves in my house where I keep my CDs and measure how many I have. His program says that I have 863 CDs. Define measurement error. What is the measurement error in my friend's CD-counting device?

863-857 = 6 CDs Measurement error is the difference between the true value of something and the numbers used to represent that value. In this trivial example, the measurement error is 6 CDs. In this example we know the true value of what we're measuring; usually we don't have this information, so we have to estimate this error rather than knowing its actual value.

Compute the range of: 22 40 53 57 93 98 103 108 116 121

99

Leptokurtic

A distribution with positive kurtosis (kurtosis > 0) has too many scores in the tails and is too peaked.

Based on what you have read in this section, what qualities do you think a scientific theory should have?

A good theory should do the following: - explain the data - explain a range of related observations - allow statements to be made about the state of the

Repeated-measures Design

An experimental design in which different treatment conditions utilize the same organisms (i.e., in psychology, this would mean the same people take part in all experimental conditions) and so the resulting data are related (aka related design or within-subject design).

Between-groups design

Another name for independent design

Between-subjects design

Another name for independent design

What is the level of measurement of the following variables? - the instruments played by the band members

This variable is categorical and nominal too: the instruments have no meaningful order but their names tell us something useful (guitar, bass, drums, etc).

What is the fundamental difference between experimental and correlational research?

In a word, causality. In experimental research we manipulate a variable (predictor, independent variable) to see what effect it has on another variable (outcome, dependent variable). This manipulation, if done properly, allows us to compare situations where the casual factor is present to situations where it is absent. Therefore, if there are differences between these situations, we can attribute cause to the variable that we manipulated. In correlational research, we measure things that naturally occur and so we cannot attribute cause but instead look at natural covariation between variables.

Twenty-one heavy smokers were put on a treadmill at the fastest setting. The time in seconds was measured until they fell off from exhaustion: 18, 16, 18, 24, 23, 22, 22, 23, 26, 29, 32, 34, 34, 36, 36, 43, 42, 49, 46, 46, 57 Compute the mode, median, mean, upper and lower quartiles, range and interquartile range.

Mode: 18, 22, 23, 34, 36, 46 (multimodal and not particularly helpful to us) (had frequencies of 2) Median: 32 ((n+1)/2th score) = 22/2 = 11th meaning the 11th score in our ordered list is the median) Mean: 32.19 Upper quartile: 42.5 Lower Quartile: 22.5 Range: 57-16 = 41 Interquartile range: 42.5-22.5 = 20

What is the level of measurement of the following variables? - the time they had spent learning to play their instruments

This is a continuous and ratio variable. The amount of time could be split into infinitely small divisions (nanoseconds even) and there is a meaningful true zero (no time spent learning your instrument means that, like 911, you can't play at all).

What is the level of measurement of the following variables? - the number of downloads of different bands' songs on iTunes

This is a discrete ratio measure. It is discrete because you can download only whole songs, and it is ratio because it has a true and meaningful zero (no downloads at all). It is also a continuous variable because entities get a distinct score. - Continuous -- ratio variable

What is the level of measurement of the following variables? - the names of the bands downloaded

This is a nominal variable. Bands can be identified by their name, but the names have no meaningful order. The fact that Norwegian black metal band 1349 call themselves 1349 does not make them better than British boy-band has-beens 911; the fact that 911 were a bunch of talentless idiots does, though.

What is the level of measurement of the following variables? - their positions in the iTunes download chart

This is an ordinal variable. We know that the band at number 1 sold more than the band at number 2 or 3 (and so on) but we don't know how many more downloads they had. So, this variable tells us the order of magnitude of downloads, but doesn't tell us how many downloads there actually were.

What is the level of measurement of the following variables? - the gender of the people giving the bands their phone numbers

This variable is categorical and binary: the people dishing out their phone numbers could fall into one of only two categories (male or female)

What is the level of measurement of the following variables? - the phone numbers that the bands obtained because of their fame

This variable is categorical and nominal too: the phone numbers have no meaningful order; they mights as well be letters. A bigger phone number did not mean that is was given by a better person.

What is the level of measurement of the following variables? - the type of drugs bought by the bands with their royalties

This variable is categorical and nominal: the name of the drug tells us something meaningful (crack, cannabis, amphetamine, etc.) but has no meaningful order.

What is the level of measurement of the following variables? - the weight of drugs bought by the bands with their royalties

This variable is continuous and ratio. If the drummer buys 100 g of cocaine and the singer buys 1 kg, then the singer has 10 times as much.

What is the level of measurement of the following variables? - the money earned by the bands from the downloads

This variable is continuous and ratio. It is continuous because money (pounds, dollars, euros or whatever) can be broken down into very small amounts (you can earn fractions of euros even though there may not be an actual coin to represent these fractions)

What is the difference between reliability and validity?

Validity is whether an instrument measures what it was designed to measure, whereas reliability is the ability of the instrument to produce the same results under the same conditions

Negative Skew

When the frequent scores are clusters at the higher end of the distribution and the tail points towards the lower or more negative scores, the value of skew is negative. See skew.

Binary variable

a categorical variable that has only two mutually exclusive categories (e.g. being dead or alive)

Probability Distribution

a curve describing an idealized frequency distribution of a particular variable from which it is possible to ascertain the probability with which specific values of that variable will occur. For categorical variables it is simply a formula yielding the probability with which each category occurs.

Bimodal

a description of a distribution of observation that has two modes

Platykurtic

a distribution with negative kurtosis (kurtosis < 0) has too few scores in the tails and is quite flat.

Concurrent validity

a form of criterion validity where there is evidence that scores from an instrument correspond to concurrently recorded external measures conceptually related to the measured construct

Predictive Validity

a form of criterion validity where there is evidence that scores from an instrument predict external measures (recorded at a different point in time) conceptually related to the measured construct.

Experimental Research

a form of research in which one or more variables are systematically manipulated to see their effect (alone or in combination) on an outcome variable. This term implies that data will be able to be used to make statements about cause and effect. Compare with cross-sectional research and correlational research.

Cross-sectional Research

a form of research in which you observe what naturally goes on in the world without directly interfering with it by measuring several variables at a single time point. In psychology, this term usually implies that data come from people at different age points, with different people representing each age point. See also correlational research, longitudinal research.

Longitudinal Research

a form of research in which you observe what naturally goes on in the world without directly interfering with it, by measuring several variables at multiple time points. See also correlational research, cross-sectional research.

Correlational Research

a form of research in which you observe what naturally goes on in the world without directly interfering with it. This term implies that data will be analyzed so as to look at relationships between naturally occurring variables rather than making statements about cause and effect. Compare with cross-sectional research, longitudinal research and experimental research.

Histogram

a frequency distribution

Central Tendency

a generic term describing the center of a frequency distribution of observation as measured by the mean, mode and median.

Quartiles

a generic term for the three values that cut an ordered data set into four equal parts. The three quartiles are known as the first or lower quartile, the second quartile (or median) and the third or upper quartile.

Frequency Distribution

a graph plotting values of observations on the horizontal axis, and the frequency with which each value occurs in the data set on the vertical axis (aka histogram)

Skew

a measure of the symmetry of a frequency distribution. Symmetrical distributions have a skew of 0. When the frequent scores are clustered at the lower end of the distribution and the tail points towards the higher or more positive scores, the value of skew is positive. Conversely, when the frequent scores are clustered at the higher end of the distribution and the tail points towards the lower or more negative scores, the value of skew is negative

Positive Skew

a measure of the symmetry of a frequency distribution...when the frequent scores are clustered at the lower end of the distribution and the tail points towards the higher or more positive scores.

Normal Distribution

a probability distribution of a random variable that is known to have certain properties. It is perfectly symmetrical (has a skew of 0), and has a kurtosis of 0.

Counterbalancing

a process of systematically varying the order in which experimental conditions are conducted. In the simplest case of there being two conditions (A and B), counterbalancing simply implies that half of the participants complete condition A followed by condition B, whereas the remainder do condition B followed by condition A. The aim is to remove systematic bias caused by practice effects or boredom effects

Hypothesis

a proposed explanation for a fairly narrow phenomenon or set of observations. It is not a guess, but an informed, theory-driven attempt to explain what has been observed. A hypothesis cannot be tested directly but must first be operationalized as predictions about variables that can be measured (see experimental hypothesis and null hypothesis)

Mean

a simple statistical model of the center of a distribution of scores. A hypothetical estimate of the 'typical' score.

Percentile

a type of quantile; they are values that split the data into 100 equal parts.

Noniles

a type of quantile; they are values that split the data into nine equal parts. They are commonly used in educational research.

Confounding variable

a variable (that we may or may not have measured) other than the predictor variables (aka independent) in which we're interested that potentially affects an outcome variable (aka dependent)

Continuous Variable

a variable that can be measured to any level of precision. (Time is a continuous variable, because there is in principle no limit on how finely it could be measured.)

Discrete Variable

a variable that can only take on certain values (usually whole numbers) on a scale

Predictor Variable

a variable that is used to try to predict values of another variable known as an outcome variable.

Outcome Variable

a variable whose values we are trying to predict from one or more predictor variables.

Theory

although it can be defined more formally, a theory is a hypothesized general principle or set of principles that explain known findings about a topic and from which new hypotheses can be generated. Theories have typically been well-substantiated by repeated testing.

Variance

an estimate of average variability (spread) of a set of data. It is the sum of squares divided by the number of values on which the sum of squares is based minus 1. aka sum of squared errors divided by the degrees of freedom

Standard Deviation

an estimate of the average variability (spread) of a set of data measured in the same units of measurement as the original data. It is the square root of the variance.

Sum of Squares (SS)

an estimate of total variability (spread) of a set of observations around a parameter (such as the mean). First the deviance for each score is calculated, and then this value is squared. The SS is the sum of these squared deviances.

Independent Design

an experimental design in which different treatment conditions utilize different organisms (e.g., in psychology, this would mean using different people in different treatment conditions) and so the resulting data are independent (aka between-groups or between-subjects designs)

Ratio Variable

an interval variable but with the additional property that ratios are meaningful. For example, people's ratings of this book on Amazon.com can range from 1 to 5; for these data to be ratio not only must they have the properties of interval variables, but in addition a rating of 4 should genuinely represent someone who enjoyed this book twice as much as someone who rated it as 2. Likewise, someone who rated it as 1 should be half as impressed as someone who rated it as 2.

Independent Variable

another name for a predictor variable. This name is usually associated with experimental methodology (which is the only time it makes sense) and is used because it is the variable that is manipulated by the experimenter and so its value does not depend on any other variables (just on the experimenter). I just use the term predictor variable all the time because the meaning of the term is not constrained to a particular methodology.

Within-subject Design

another name for a repeated-measures design

Reliability

the ability of a measure to produce consistent results when the same entities are measured under different conditions.

Dependent Variable

another name for outcome variable. This name is usually associated with experimental methodology (which is the only time it really makes sense) and is used because it is the variable that is not manipulated by the experimenter and so its value depends on the variables that have been manipulated. To be honest, I just use the term outcome variable all the time - it makes more sense (to me) and is less confusing

Second Quartile

another name for the median

Sum of Squared Errors

another name for the sum of squares

Categorical variable

any variable made up of categories of objects/entities. The university you attend is a good example of a categorical variable: students who attend the University of Sussex are not also enrolled at Harvard or UV Amsterdam, therefore, students fall into distinct categories.

Variables

anything that can be measured and can differ across entities or across time.

Interval Variable

data measured on a scale along the whole of which intervals are equal. For example, people's ratings of this book on Amazon.com can range from 1 to 5; for these data to be interval it should be true that the increase in appreciation for this book represented b ya change from 3 to 4 along the scale should be the same as the change in appreciation represented by a change from 1 to 2 or 4 to 5.

Ordinal Variable

data that tell us not only that things have occurred, but also the order in which they occurred. These data tell us nothing about the differences between values. For example, gold, silver and bronze medals are ordinal: they tell us that the gold medallist was better than the silver medallist, but they don't tell us how much better (was gold a lot better than silver or were gold and silver very closely completed?).

Multimodal

description of a distribution of observations that has more than two modes.

Validity

evidence that a study allows correct inferences about the question it was aimed to answer or that a test measures what it set out to measure conceptually. See also content validity, criterion validity.

Criterion Validity

evidence that scores from an instrument correspond w/ (concurrent validity) or predict (predictive validity) external measures conceptually related to the measured construct.

Content Validity

evidence that the content of a test corresponds to the content of the construct it was designed to cover

Ecological Validity

evidence that the results of a study, experiment or test can be applied, and allow inferences, to real-world conditions

Qualitative Methods

extrapolating evidence for a theory from what people say or write (contrast with quantitative methods).

Journal

in the context of academia a journal is a collection of articles on a broadly related theme, written by scientists, that report new data, new theoretical ideas or reviews/critiques of existing theories and data. Their main function is to induce learned helplessness in scientists through a complex process of self-esteem regulation using excessively harsh or complimentary peer feedback that has seemingly no obvious correlation with the actual quality of the work submitted.

Quantitative Methods

inferring evidence for a theory through measurement of variables that produce numeric outcomes (cf. qualitative methods)

Why is randomization important?

it is important because it rules out confounding variables (factors that could influence the outcome variable other than the factor in which you're interested). For example with groups of people, random allocation of people to groups should mean that factors such as intelligence, age and gender are roughly equal in each group and will not systematically affect the results of the experiment.

Practice Effect

refers to the possibility that participants' performance in task may be influenced (positively or negatively) if they repeat the task because of familiarity with the experimental situation and/or the measures being used.

Boredom effect

refers to the possibility that performance in tasks may be influenced (the assumption is a negative influence) by boredom or lack of concentration if there are many tasks or the task goes on for a long period of time. In short, what you are experiencing reading this glossary is a boredom effect