Psych 440 Midterm 1
Culture
"the socially transmitted behavior patterns, beliefs, and products of work of a particular population, community, or group of people" (Cohen, 1994) History suggests cultural bias in testing can have an adverse impact: • Immigration restrictions • Forced sterilization
Patrick's Key Points
All psychological tests involve collecting a sample of behavior • Often used to predict behaviors that are very different from tested behaviors Tests are evaluated on the basis of reliability, validity, and adequate norms Remember to stay calm and composed during job-related interviews
Error
Deviation for some measurement from the true standing of an individual on some characteristic Many sources of error: • effects of the environment • precision of the measurement device • confounding variables Error influences estimates of both central tendency and variability
Culture and Testing
Many early tests had NO minority individuals in standardization samples, Items culturally grounded in the dominant American culture: • "Who was the first person to discover America?" Translation problems: no corresponding object/word, changes in meaning
Measures of Central Tendency
Mode (most frequently observed) Mean (average score) Median (50th percentile score)
The Mode
Most frequently observed score Only measure of central tendency that can be used with Nominal data Examples: -3, 4, 4, 5, 5, 5, 6, 8 - mode = 5 • 3, 4, 4, 4, 5, 5, 5, 8 - modes = 4, 5
The Median
Point which divides the group in half so that 50% of scores fall above it and 50% fall below it Better measure of central tendency than the mean when the data are skewed because it is unaffected by extreme scores.
Scales of Measurement
Scales, or levels, of measurement help determine what statistical analyses are appropriate Enable test users to make accurate score interpretations Four levels: • Nominal • Ordinal • Interval • Ratio
The Mean
The "Average" of a set of scores Found by summing all values and then dividing that sum by the total number of observed values Requires Interval or Ratio Data Sensitive to every score in the sample, and may be inappropriate with skewed data
Describing Data
Three methods: • Pictorially • MeasuresofCentralTendency • Measures of Variability (or Dispersion)
Ordinal Scale Statistics
Values imply nothing about magnitude of differences between one level to the next Numbers are not units of measurement Statistical operations are limited to non- parametric tests
Correlation
A statistical technique which allows us to make inferences about how two (or more) variables relate (co-relate) to each other (linearly) Expressed using a correlation coefficient • statement about the direction of a relation statement about the strength of the relation
The Normal Distribution
A symmetrical, mathematically defined frequency distribution curve Highest at the center (most frequent scores are at the mean) and tapering on both sides Asymptotic towards the abscissa Mean, median and mode are equal Area under the curve is divided in terms of standard deviation units and can aid in the interpretation of test scores
Norm-Referenced Evaluation
A way of interpreting test scores by comparing an individual's results to the scores of a group of test takers Interpretation is relative Alternative is Criterion-Referenced Evaluation
Professional Standards
APA 1895 - formed committee on mental measurement APA 1954 - published Technical Recommendations for Psychological Tests and Diagnostic Tests Collaboration of APA and other organizations (AERA) have lead to publication of sound practices in the field of testing and assessment
Coefficient of Determination
Accurate interpretation of correlation coefficients requires another statistic, the coefficient of determination Calculated by squaring the correlation coefficient (r2) The coefficient of determination tells how much variance in one variable is accounted for by the variance in the other
Z-Scores: Pros and Cons
Advantages: Indicates each person's standing as compared to the group mean Can easily be converted to percentiles Disadvantages: Negative z values can be difficult to work with and explain Dealing with fractional z values can be a hassle
Percentile Pros and Cons
Advantages: can be used to interpret performance in terms of various groups and are easily understood Disadvantages: units are not equal on all parts of the scale *Percentiles are an ordinal scale *Differences between individuals near the middle are magnified and differences at the extremes are compressed
Why is the Normal Distribution so important in Psychometrics?
Many psychological and educational variables are distributed approx. normally Examples: reading ability, introversion, job satisfaction, and memory Allows for the use of many types of inferential statistics
Deviation Scores
Measure of how far the raw score is from the mean of its distribution (X - μ)
Inferential Statistics
Methods for making inferences about a population of objects based on information from a sample from that population Examples: • chi-square test of association • t-test and ANOVA • correlation and regression
Interval Scale
Numbering includes order, but intervals between each successive level represents equal differences No absolute zero point in the scale Examples: Fahrenheit Scale IntelligenceTestScores
Review: Getting Percentiles from a Z-Score Probability Table
On a recent exam, the class average was 65 with a standard deviation of 10. Student A and Student B scored 82 and 70 respectively, what are their percentile ranks? Step 1 - Calculate the z-scores ZA = 82 - 65 / 10 = 1.7
Correlation Coefficient
Pearson's r No Relation = r = 0.00
Types of Norms
Percentiles Age Grade National Anchor Subgroup Local
Predictors of Risk Taking Behavior
Positive Predictors • Confidence • Risk Propensity • Sensation Seeking • Gender (M>F) • Extraversion Negative Predictors • Age • Social Desirability • Neuroticism • Risk Assessment
Prediction Examples
Predicting job performance from aptitude test scores Predicting success in school from college entrance exam scores Predicting response to therapy based on severity of disorder and length of treatment Predicting length of marriage from measure of relationship satisfaction
Prediction
Predicting values of one variable based on knowledge of scores on other variables is a practical use of correlation Simple Linear Regression is used when one variable is used to predict values Multiple Regression is used when multiple predictors are used Logistic Regression is used when the variable being predicted is dichotomous (ex. gender)
Descriptive Statistics
Procedures for organizing, summarizing, and describing quantitative information Academic performance can be described using descriptive statistics Examples: • Batting average • Census data • Horsepower
Federal Testing Legislation
Public interest in educational testing sparked by Sputnik (1957) National Defense Education Act (1958) provided money for aptitude testing in attempt to identify gifted children Increased use of tests led to concerns about value and effect of psychological testing on students
Advantages of Standardized Measurements
Quantification Communication Economy Scientific Generalizability
Other Samples
Random - each individual from the population has an equal chance of being included in the sample Purposive - arbitrarily selecting a sample because it is believed to represent some population Convenience - a sample that is convenient or available for use
Range
Range is the difference between the highest and lowest scores Is sensitive to outliers A: 2, 5, 7, 7, 8, 8, 10, 12, 15, 17, 20 Range = 18 B: 2, 2, 2, 3, 4, 4, 5, 5, 5, 6, 6, 20 Range = 18
Variance and Standard Deviation
Reflects the variability of scores about the mean of the group Variance is the average of the sum of the squared deviations of each score from the mean The standard deviation is the square root of the variance ***is expressed in the same units of measurement as the original scores.
Coefficient of Determination
Relationship between self-esteem and job satisfaction; r = .5 r2 = .25 (percent of shared variance)
Characteristics of an Effective Test
Reliability: • Does the test produce consistent measurement results? Validity • Does the test measure effectively what it purports to measure? Adequate norms • Was the test developed using samples similar to the people taking the test?
Stratified Samples
Sampling individuals from subgroups in the population in the same proportion as the population they are part of Best when population includes subgroups that differ on some potentially meaningful characteristic Helps prevent sampling bias
Two types of measurement
Scaling- represent quantity of an attribute numerically ex: Physical Attributes and other Quantities such as weight, height, age Classification - define when objects fall into the same or different categories with regards to an attribute. Ex: Types of objects, College majors
Standard Deviation Formula
Standard Deviation - the average deviation of each score from the mean
Error and Prediction
Standard Error of the Estimate (SE) Indicates magnitude of errors in estimation Higher correlations produce smaller SE Lower correlations produce larger SE
Why Use Standard Scores?
Standard Scores are more easily interpretable than raw scores We can tell where a score falls in relation to other scores using a standardized scale Allow for easier comparisons of both similar and dissimilar scores
Z-Scores
A standard score where the mean of the scores is set at zero (0) and standard deviations are set at intervals of one (1)
James McKeen Cattell
First American to systematically study assessment of individual differences A student of Wundt, but more influenced by Galton's methods Studied differences in reaction time Coined the term "mental test" Named his daughter "Psyche"
Assessment Assumptions
1. Psychological States or Traits exist, and can be quantified and measured --If these things are not real, then what's the point of trying to measure them? --If we can't quantify and measure them then psychologists are screwed 2. Different approaches to measuring aspects of the same thing can be useful --Because we are making inferences it is better to have convergent evidence 3. Various sources of error are part of the assessment process --Our goal is to manage error sources instead of ignoring the problem 4. Test-related behavior can predict behavior in other settings --Otherwise the test would be useless 5. Present-Day behaviors can predict future behaviors --Or there would be no point to testing
Origins of Testing: Early Psychology
1859 Darwin's Origin of the Species raised issue of individual differences --Provides theoretical basis for animal models in medical and psychological testing Wilhelm Wundt was a German medical doctor who studied how individuals were similar instead of different (Leipzig School) --• Described human abilities with respect to reaction time, perception and attention span
Scale
A set of numbers whose properties model empirical properties of the variables to which the numbers are assigned Discrete - categorical labels or integers, no meaningful middle grounds between categories Continuous - numbers do not represent categories, middle ground between units possible
Origins of Testing: Psychology Precursors
-Chinese civil service exams initiated in the Chang Dynasty over 3000 years ago
Nominal Scale
-Nominal (or Naming) Level -Lowest level of measurement -Ordering is not important, only the label attached to designate a mutually exclusive and exhaustive category Examples: • MedicalDiagnoses • Gender • Political party affiliation
Chapter 3: Descriptive Statistics Standardized Scores
...
Chapter 3: Scales of Measurement Descriptive Statistics
...
Chapter 4: Correlation and Regression, Tests and Testing
...
Testing Assumptions
1. Psychological States and Traits exist 2. Psychological states and traits can be quantified and measured 3. Test-related behavior predicts non-test- related behavior 4. Measures have both strengths and weaknesses 5. Various sources of error are part of the assessment process 6. Testing and assessments can be conducted in a fair and unbiased manner 7. Testing and Assessment benefits society
Normative Sample
A group of people whose performance on a particular test is analyzed for reference in evaluating the performance of individual test-takers. Sample must be representative or typical of the intended population of interest Inadequate norms makes it difficult to make proper interpretations
Interpreting Percentiles
A percentile difference of 10 near the middle of the group often represents a smaller difference in performance than a difference of 10 near the tails in terms of skills, a difference of a few percentile points near the tails means more change has taken place than the same size difference near the middle of the group
Standard Scores
A raw score that has been converted from one scale to a new (standardized) scale with a prescribed mean and SD Typically expressed in terms of number of standard deviations from the mean **All standard scores have equal unit sizes across the distribution
Percentiles
A raw score that has been converted into the percentage of a distribution that falls below that particular raw score Widely used in test manuals as well as other literature on commercially published standardized tests
Age and Grade Norms
Average performance of test-takers at various ages/grades Scores do not represent equal units of measurement Scores often (incorrectly) used as evaluative standards Not effective with very young or adult test-takers
Interval Scale Statistics
Because of equal intervals between values some mathematic operations are meaningfully appropriate: • Addition and Subtraction • Multiplication and Division not appropriate because the is no true zero • Statistical tests based on mean scores and/or variance
Review: Working with Negative Z-Scores
Calculating percentile ranks for scores below the mean is slightly more difficult Student C and Student D were in the same stats class as Student A and Student B average score = 65, std. dev. = 10 • Student C scored 51 • Student D scored 60 What were their percentile ranks?
Multiple Regression
Can be used when more that one predictor variable is available Multiple regression takes into account the correlation between each of the predictor scores and what is being predicted Also taken into account are the correlations among the predictors Y=a+b1X1 +b2X2
Central Tendency vs. Variability
Central Tendency measures are used to describe the typical response seen in a sample of observations Variability measures are used to describe how much fluctuation in scores there are in a sample of observations We need both to interpret a person's score
Variable
Characteristics or attributes of objects (people, places, things, animals, etc.) in a population that are not constant Measurement is the process of assigning numbers or symbols to a characteristic or attribute according to a set of rules
Alfred Binet
Commissioned by France's education system to help identify "subnormal" children Developed first intelligence test in 1905 with Theodore Simon Mental Age proposed as criterion for evaluation Test revised by Lewis Terman at Stanford, current revisions still widely used
Scales and Descriptive Statistics
Data must be measured on an interval or a ratio scale for the computation of means and other parametric statistics to be valid. Therefore, if data are measured on an ordinal scale, the median but not the mean can serve as a measure of central tendency.
Simple Linear Regression
Describes the relationship between one Independent Variable (X) and one Dependant Variable (Y) Least-Squares approach is used to minimize the differences between observed and predicted scores Regression line is the straight line which comes closest to the greatest number of points on the scatterplot of X and Y
Kurtosis
Describes the steepness of a distribution in its center Platykurtic = flat Leptokurtic = peaked with fat tails Mesokurtic = somewhere in between
Skewness
Distributions can be characterized by the extent to which they are asymmetrical or "skewed" Positive skew: only a few extremely high scores and many low scores Negative skew: only a few extremely low scores and many high scores
Quartiles
Dividing points between the four quarters of a distribution of test scores Interquartile range is equal to the difference between Q3 and Q1 * The relative distance of Q1 and Q3 from the median (Q2) gives an indication of skewness of the distribution Semi-interquartile range equals the interquartile range divided by 2
Nominal Scale Values?
Good = 1 Bad = 2 Ugly = 3 2.5 = ?
Patrick's Key Points
History suggests that scientific methods can be manipulated to achieve desired results Scaling using standardized methods is desirable to the alternatives Choice of scale depends on characteristics of the thing you are measuring But choice of scale may also limit the types of statistical inferences you can do
Issues with Prediction
How do we deal with the fact that the predictors (X) and the variable to be predicted (Y) are often on different scales of measurement? Prediction technique must take into account both the scales of measurement and the correlation between the two variables Linear regression does just that!
Correlation ≠ Causation
Ice cream sales and the number of shark attacks on swimmers are correlated. The number of cavities in elementary school children and vocabulary size have a strong positive correlation. Number of churches in a community and the crime rate are positively correlated. Other factors are 'causing' the effects
Nominal Scale Statistics
If numbers are assigned, they cannot be meaningfully manipulated mathematically Appropriate arithmetic operations: -counting -proportions -percentages -chi-square tests
Coefficient of Alienation
If r2 = .25, the coefficient of alienation = .75 (1 - r2 the percentage of unexplained variance)
Z-Scores and the Normal Distribution
If we have a normal distribution we can make the following assumptions: Approximately 68% of the scores are between a z-score of 1 and -1 Approximately 95% of the scores will be between a z-score of 2 and -2 Approximately 99.7% of the scores will be between a z-score of 3 and -3
Satisfaction With Life Scale
In most ways my life is close to my ideal The conditions of my life are excellent I am satisfied with my life So far I have gotten the important things I want in life If I could live my live over, I would change almost nothing
Ratio Scale
Includes ordering, equal intervals AND an absolute zero Examples: • length and weight • Kelvin scale All mathematical operations can be meaningfully performed
Ordinal Scale
Individuals or things are ranked or ordered on the basis of some criteria Intervals between ranks are not consistent Examples: • Grade level • Ranking from shortest to tallest • Gold, Silver, Bronze • Movie sequels
Questions to Ask
Is the number statistically significant? • Could this have occurred merely by chance? How strong is the relation? • What proportion of the variance in variable Y is accounted for by variation in variable X? • Coefficient of Determination (r2) Is the relation linear? Has the value been affected by a restricted range of values? Do outliers affect the result?
Types of Samples
Stratified Random Purposive Convenience
Negative Relations
Strong Negative (r = -.7) **Political Affiliation and Willingness to vote for another party's candidate Moderate/Weak Negative Relation (r = - .4) **Brushing teeth and cavities r2 = proportion of variance shared by variables
Positive Relations
Strong Relation (r = .7 or higher) **Height and Weight **Age and Job Experience Moderate/Weak Relation (r = .4 or lower) **Chemotherapy and Cancer remission **GRE Scores and Grad student success r2 = proportion of variance shared by variables
Sir Frances Galton
Studied genetic influence using pedigree charts Attempted to quantify individual differences by classifying people Developed first correlation coefficient later refined by Karl Pearson Created Anthropometric Laboratory in London in 1884 Major Proponent of the Eugenics Movement
Measures of Variability
Synonyms for variability are spread and dispersion Each term refers to differences among scores within a sample or population Three common types are: • Range • Deviation Scores • Variance and Standard Deviation
T-Scores
T-scores represent one transformation of z which overcomes the disadvantage of working with negative scores T-score = (z score X 10) + 50 T score mean = 50 T Score SD = 10
Who's Involved in Assessment?
Test Developers • Psychologists required to adhere to ethical standards (APA, AERA) Test Users • Counselors, other therapists, teachers, human resources, researchers The Test Taker Society at Large
Sampling and Norms
Test administered to members of the sample under the same conditions • Environment • Instructions • Time restrictions • Et cetera -Developers calculate descriptive statistics Provide precise description of sample
Testing vs. Assessment
Testing - the process of measuring variables by means of devices or procedures designed to obtain a sample of behavior Assessment - the process of gathering and integrating data for the purpose of making an evaluation **Most effective when information is obtained using multiple techniques
Population Z-Score Formula
The population z-score is calculated by subtracting the population mean from the individual raw score and then dividing by the population standard deviation. Where: • x is the individual score • μ is the population mean • δ is the population standard deviation
Sample Z-Score Formula
The sample z-score is calculated by subtracting the sample mean from the individual raw score and then dividing by the sample standard deviation Where: • x is the individual score • x is the sample mean • s is the sample standard deviation
Testing in the U.S.
U.S. military developed Army Alpha & Beta during WWI (Yerkes & Brigham) --Used to identify intellectual abilities of recruits and personality risk factors for "shell shock" In 1939 the Wechsler-Bellevue Intelligence Scale (now WAIS) developed for adults. • Later other versions were developed for use with preschool (WPPSI) and school age children (WISC).
Spearman's Rho (Ρ or ρ)
Used if sample sizes are small OR If Ordinal Scale data is used
Working Backwards
We can also use the z-score to work backwards and figure out a raw score. E.g., Kim is in the same stats class as Students A, B, C, & D. She finished in the 65th percentile. What was her score? • X=65; s=10 Plug this z-score into the formula and solve for X
Number of Students in a Range?
We can calculate the number of students who finished between Student A (96th percentile) and Student C (8th percentile). Multiply total N by the percentage of Student A and Student C **60 students in the class ** #between A and C= 60 x 0.88 = 53
Inference Example
Who has the stronger bite? Cheetah or small groundhog animal? (small animal)
Multiple Regression Example
X1= Interview, X2= Ability Test, Y = Graduate GPA rX1Y = .50 rX2Y = .75
Patrick's Key Points
Z-Score table can be used to translate observed scores into standardized scores *Assumes a normal distribution Standardized scores allow for the more effective interpretation of test results • Also assumes a normal distribution Test interpretation is often dependent on assumption of normality - but not always
Z-Scores and Percentile Ranks
Z-scores can be used to calculate percentiles when raw scores have a normal distribution When used in conjunction with a Z-Table, the z-score reveals the area of the normal distribution below the score in question
Using Central Tendency and Variability to Interpret Bad Ideas
video of people doing dumb things
