RM 2 exam 1 - reading charts

Réussis tes devoirs et examens dès maintenant avec Quizwiz!

discussion - what's the nmnemonic

Run Little Doggy --reverse funnel-small to large specfiic hypot --different/simillar findings with theory/lit --Limitations

discussion first points

Run Little Doggy --reverse funnel-small to large specfiic hypot --different/simillar findings with theory/lit --Limitations

sum of squares and cross products

SSCP = the covariance for each pair of dv's

between groups SSCP total

SSCP total = Between/ within --Between - group mean x grand mean --Within = indvidual score/group mean = error

Univariate normality

Scores normal for -each group(IV/level) for ANOVA

interpreting non signifigant results in terms of power

Sig results --low power no sig = maybe not enough pt ---high power no sig = likely not there

Cohen's D 1 - start at 2 and go up to 5 then up to 8 .2 . 5 . 8 small medium large: Left off at 8 Step 2 ? <img src="c:\users\chelsey\dropbox\current classes\flashcards for all classes\classes\research\research stats\keyboard.png">

Step 2: Count 8: Go down 1 to 5 = 85% overlap .2: Left off at 5<img src="c:\users\chelsey\dropbox\current classes\flashcards for all classes\classes\research\research stats\colored keys.png">

step 3 metanalysis

Step 3 - coding - translate to # - best way to split not too broad to overlook and not to small -probles and translate in by self problems did you capture the essential features? missing data acknowledge trouble and how impact future findgins translate everything to quan want general enough for gist, but specific enough to capture unique variable report interrater relatibility for coding should be at least 80%

Step 4 cohen's after finding . 2 =85%, .5=67%, .8<img src="c:\users\chelsey\dropbox\current classes\flashcards for all classes\classes\research\research stats\keyboard.png">

Step 4: From 7 go over 1 to right & down 1 to = 5 : (box 2 complete) From 5 go down again and over to the right = 3 (1/2 a rectanlgle complete) .8 = 53% overlap<img src="c:\users\chelsey\dropbox\current classes\flashcards for all classes\classes\research\research stats\colored keys.png">

test for sig of mean effect size - metaanysi

Stoffer method (f) --convert to Z scores and add compare to p value

homogeneity of covariance MANOVA

MANOVA -- use pilla's if violated, wilks if not --homogeneity of covariance --vectors of means = variance and normal distrubtion? (all f) variation between groups not = if violated reduce power slightly when groups = impacts power/type1 error

metanalysis vs research syntehiss - differences

META - > external validy and has fail safe N RS -- not good for external validiy , no fail safe n

METAanalysis IV

METAanalysis IV data that has been coded into categorical data often representing study features remember you're analysis level is 1 up from regular bc your looking at study features and effect sizes (dv) not indvidual participants per se

SSCP matrix

Matrix with SS on diagnoals and SSCP for every DV pair filling the rest in SS on diagnoal = the variance of each DV (f) SSCP = the covariance for each pair of dv's (F)

Multiple ANOVA'S follow up analysis for sig multivarate effects alpha for sig and not

Multiple ANOVA'S protect type 1/experimental alpha when sig. If not sig use bonferoni see debate: manova

nesting - just what is it. if yes and if no in seperately

Nesting yes are the same group of people giving the intervention for both groups or are they split if nested finding could be result nonsepfic factors no random faciliators or mixing it up - nonspecfici factors wash out makes it hard to keep IV seperate - you should always do MANIPULATION CHECK

Non-sig Q test

Non-sig Q test = good fit homogeneity of effectshomogenity of distriubtions homogeniety is good allows you to see IV is having effect not error (f)means studies are from sampe pouplation and you can use group mean effects

f test which to use of the 4

Pilla's if homogeneity of variance violated Wilk's if not typically doesn't make much of a difference

metanalysis combine 1 study for effect size pro con

Pro --each study doesn't get unfair extra weighting - you control weighting for study qualities or whatever you want downfall might obscure differences

Metanalysis mean effect

Quan DV metanalysis indicate strength of assocation between study feature/outcome standardized measure

anova manova debate - if manova not sig?

---some say if MANOVA not sig not protected use bonferoni

random vs fixed -- error ?

--Both --error due to subject level sampling Random --error also due to study sampling

When you look at your alpha what 3 things do you need to remember that ti's associated with --confidence level & what does this mean

--I want to be 95% confident that I found my effect 1-.95= alpha .05, so if p = < .05 95% confident something happened. so if you were to do the same study again, with a random selection from the same poulation you'd find simillar results with the similar chance of type 1 error as you did in your study

Mahalnous / Mahal's distance - each pt deviation from centroid compared to cv which is compared to a critical value to check for outliersrs Those that are sig = outlier , those that are not = not outlier whatsa centroid

--centroid = pint created from remaining sample mean of all variables

unit of analysis - has to figure out how to combine dvs effect sizes to get mean effect -- difference for the 3 that deal with particular DV's

--each dv represented -each concept represented -each dv represented that youre interested in

--manova can do a few as long as there aren't too many or too extreme

-CV comes from MAhal's distance table - residula stats

/Multivariate normality -

-MANOVA- normal distribution for --indvidual DV --linear combo DV -subset variables

why can't you intereptet well if your sample is hetrogenous - meatanlysi

-say your looking at med vs med and CBT and you find that med and cvt is hetrogenous -- well whats cuasing lower depression med or cbt when hetrogenous = 2_ populations so u divide further or not interpret as well bc u don't know which populations are causing your eeffect

if you had N^2 .012 what would that mean what about .14

.12 -- 1.2 % of the variance in our DV is explained by our IV .14 -- 14% of the variance in our DV is explained by our IV

if you use a covariate that didn't meet assumptions

RESULT if not satifised group x covariate interaction that could be overlooked

step 1-3 metanalysis

1 hypot/research domain 2) lit review 3) coding 4) index es 5) analyzie ES 6) draw conclusions

steps 4-6 metanalysis

1 hypot/research domain 2) lit review 3) coding 4) index es 5) analyzie ES 6) draw conclusions

1 way

1 way MANOVA 1 + categorical IV, 2 + continous DV's iv categrocial could be continous thats been chunked Dv DVS which are necessairly CORRELATED with each other and ideally theoeritcally correlated too because it has 2+ dv = multiariate analysis 1 way anova - 1 iv 1 dv

covariate only ues if

1) stat correlated with DV(s) 2) homogeneity of reggression satisfied = relationship between covariate and dv must be = for all groups (f) --RESULT if not satifised group x covariate interaction that could be overlooked (F)

covariate only use if correlated & --whats > homogeneity of reggression satisfied

2 way

2 way manova could have a 2 way manova with 2 ivs and 2 + dvs could have a 2 way manova with 2 ivs and 1 dv

When you look at your alpha what 3 things do you need to remember that ti's associated with --confidence level & what does this mean

--I want to be 95% confident that I found my effect 1-.95= alpha .05, so if p = < .05 95% confident something happened. so if you were to do the same study again, with a random selection from the same poulation you'd find simillar results with the similar chance of type 1 error as you did in your study

fiex effects model

--error due to subject level sampling --thinks underlyiging, united effect being measured -hopes to generalzie to those like it --think if sample sizes were large enough all effect sizes would be =

When you look at your alpha what 3 things do you need to remember that ti's associated with used to evalute p value

--what level sig needed to reach to reject null --how extreme socre needs to be to reject null

manova debate - criticism against hummel and sligo who said okay to do anovas after manova with out > experimenter wise alpha -- criticims point 2

2) manova only protects type 1 error if null is true so even if manova is sig doing anovas can still incrase experimterwise alpha

F test # groups need to compare

3 + Ratio of variance between groups/within groups -stats test need 3 + groups to compare. like which age group runs fastest --look at differences between groups --older kids run faster/ --look at diff within groups - could have a super fast 4 year old Sig - difference between groups > within groups = old kuds stiill run faster even if there are some oultliers

Run Little Doggy lats point

--reverse funnel-small to large specfiic hypot --different/simillar findings with theory/lit --Limitations

unit of analysis - finding effect size for the whole study

somtimes good bc each study isn't getting unfair weight but could obscure differences

anova manova debate - HUMMEL AND SLIGO

ANOVA KEPT experimterwise lowest - didnt > it looked at all diff ways to analyize manova

between groups effect size

Effect size = between / within --between = SS between / between subject variance --within = SStotal = error - so within has to look at indvidual scores to group means and group means tot tal mean for all

q stat specfically looks ay -- yes wants to see if you sample studies variation are representive of the poulation variation but how does it do this

SSQ = between groups/ residual variability for q = within then looks at distribution of effect sizes with chi square and u evalute with CV

When you look at your alpha what 3 things do you need to remember that ti's associated with -type 1 error

chance type 1 error alpha .01 = 1/100 for every test chance type 1 error for alpha .05 = 1 in 20 chances out of the # of test you do in your sample or in the general population? -- general population if same poplulation, random sample same alpha

Step 3 - coding - translate to # - best way to split not too broad to overlook and not to small -problems

did you capture the essential features? missing data acknowledge trouble and how impact future findgins

unit of analysis - finding effect sizefind effect sizes for each DV in 1 study downfall

downfall: # of dv's in 1 study could get unfair weighting

follow up analysis for sig multivarate effects - dv variable contiribution remove 1 variable at a time: technique Huberty : f to remove - test sig for each dv when removed from whole set THEN?

rank orders dv by impt

Step 3 - coding - translate to # - best way to split not too broad to overlook and not to small - translate everything to quan

want general enough for gist, but specific enough to capture unique variable report interrater relatibility for coding should be at least 80%

<img src="c:\users\chelsey\dropbox\current classes\flashcards for all classes\classes\research\research stats\keyboard.png"> 1 - 2.5.8 2- .2 =85 % left off at 5 step 3: % overlap for . 5 = ?

<img src="c:\users\chelsey\dropbox\current classes\flashcards for all classes\classes\research\research stats\colored keys.png"> --From 5 go over 1 = 6 --now go all the way left (4) and up 1 = 7 --.5 =67% - Box 1 completed and working on bottom of box 2

bonferoni inequeatility example for 6 test and nominal alpha .05

= 6 *.05 = 30% or less chance type 1 error for those 6 test (f all under) multiplies # of test and nominal alpha = max value of alpha for set of test -used when manova NOT sig but still want to do ANOVA'S

MANOVA looks for diff between groups based on DV by looking at between groups/within groups what goes into within groups groups?

= amt of error for all pt over GRAND MEAN find with sscp -- compares pt score to GROUP mean, then group mean scores to GRAND mean. considers SSCP

within groups -- find it how what is it -- just find it

= error for all pt over grand mean (found with SSCP total which looks at indvidual score to group mean, then gets you to group mean x grand mean)

within group manova

= error for all pt over grand mean find with (f) SCCP total - looks at indvidual scorex group mean

what to think of faciliators were nested

= good same group of people doing both intervention ometimes hard to keep straight need manipulation check washes out nonsepcfic factors

Mahalnous / Mahal's distance - each person gets a deviation score from the centroid which is compared to a critical value to check for outliersrs Those that are sig = , those that are not =

> CV = outlier < CV = not an outlire

publication bias

> effect sizes in published vs not published articles --fail safe n helps but ntoes say bias can't be estimated or corrected for just compare studies

? She expect 4x - 4x / sd pooled effect what would negative tell you. ***********

? say 20 - 10 / 10 = outcome psychothreapy fan not good this d = + 1 tells you psychotherapy didn't do as well because their scores are on depression

cohens d . 8

? small = .2 or 85^ overlap between two distrubtions ? moderate .5 67% overlap between two distrubtions ? large - .8 or 53% overlap ..

F =

Ratio of variance between groups/within groups (F)-stats test need 3 + groups to compare. like which age group runs fastest --look at differences between groups --older kids run faster/ --look at diff within groups - could have a super fast 4 year old Sig - difference between groups > within groups = old kuds stiill run faster even if there are some oultliers

homogeneity of variance ANOVA

ANOVA - Levene's test --homogeneity of variance variance in test population = sample? homoschedastic (all f) variation between groups not = if violated reduce power slightly when groups = impacts power/type1 error --if unequal = type 1 error up or down

familywise alpha/compairsonwise alpha

ANOVA ALPHA trouble correct for how lots of test like tukey, sheffe, least sig dfifference, newman - keuls, and duncans multiple range (not as impt) chance of type 1 error within all test

DV - in regards to test type

ANOVA -1 dv t test 1 dv manova 2 + dvs that are correlated

manova assumptions

ASSUMPTIONS indpencne of observations - nessary okay if violated homogeniety of covriance look at box's m use pilla's if violated, wilks if not multivaraiate normality sphericiphicity -- for repeated measures dv's must be sig correlated with each other

metanalysis vs research syntehiss - similiarites

Both could do divide studies into suuport hypot vs not cant always guage effect sizes -- experimental could but could be hetrogenous so obscure

Whats a linear compposite cannonical variable forumula?

CV = a or constant + w1v1 +w2v2+w3v3 - each in indviduallyuses w = weight a weight for each DV per each IV analogous to slope (rise/run) higher scores have more weight meaning that that particular IV is discriminanting better other names discriminant function weight discriminant function coefficent a = discriminant function constant (y axis/intercept when making a line) v - scores for each person on orginal DVs example CV = constant + .8 (v1) + .03 (v2) v1 selective attention v 2 stm v 1 = selective attention and it has the most weight meaning that it can tell the difference better for gender here - greater difference between the iv groups her answer -- each group of IV's are weighted according to between group variability

linear question shows men and women differ most on selective attention vs stm (linear equation = a line)

CV = constant + .8 (v1) + .03 (v2) --- show that the weight is higher for the one we say has greater difference between groups. .. whats the weight called discriminant function. Variable 1 = selective attention more difference. V2 = stm

manova debate - criticism against hummel and sligo who said okay to do anovas after manova with out > experimenter wise alpha -- criticims point 1

Criticism 1) dat used in simulation not representive of real data 3) follow up ANOVAS ignore correlations between DVs doesnt matter still > experimeter error 2) manova only protects type 1 error if null is true so even if manova is sig doing anovas can still incrase experimterwise alpha

metanalysis - mnemonic

DICED d- dispense knowledge or whats most effective i - identify effective variables c - disadvantage can only compare post tx e - examine effect sizes for ivs draw conclusions like if gap is in lit summarize findings

DV - / criterion variable

DV - / criterion variable t-test 1 continous DV t - test manova 2 + correlated dvs anova- 1 continous dv metanalysis effect sizes from indvidual studies remember you're analysis level is 1 up from regular bc your looking at study features and effect sizes (dv) not indvidual participants per se

Covariateimpact f 2 ways

F larger --your IV had a larger impact than you thought that covariate was taking up too much variance F smaller --you're covariate seems to be really accounting for your impact

Introduction first two

FRIENDS FROM GPC :D FRIENDS Fill the gap FROM Funnel effect G Grounded in theory P Pertitent/Current literature C Conceputalize variables FRIENDS FROM GPC :D fftcc fftpc FFTPC

last three introduction

FRIENDS FROM GPC :D grounded in theory pertient.current literature conceptualization of variables

phillai barlett trace - what it is

Good for --small samples unequal cell size homogeneity of covariance one of the strongest for protecting against type 1 with small samples

IV --factor-- predictor 1 way 1 iv anova and manova 2 way 2 iv anova manova t -test 1 iv 2 levels metanalysis - no specificed # but usually decent amt unless your just comparing groups bc your looking at IVS data that has been coded into categorical data often representing study features remember you're analysis level is 1 up from regular bc your looking at study features and effect sizes (dv) not indvidual participants per se

MANOVA iv dv

IV/DV 1+ & 2 + continous DV's it's an extension of the MANOVA dv's must be sig correlated with each other

anova iv dv

IV/DV Anova - 1 Way 1 Iv 3+ Lcategorical Levels & 1 Continous Dv 2way Anova 2: Iv's Categorical Ivs And 1 Continuous Dv

incrase power how and what is the result

Increase power - seat >sample size >effect size >less stringent alpha >1 tail Results in > chance of type 1 error

step 4 meta anlyis :

Index ES - find mean effect by looking at effect sizes dv and variability - chose proper unit anayis within 1 study find effect size for 1 of these unis of aanlayss

Whats a linear compposite cannonical variable forumula? CV = a + w1v1 +w2v2+w3v3--whats v

Indvidual dependent variable scores for each person DVsexample CV = constant + .8 (v1) + .03 (v2) v1 selective attention v 2 stm v 1 = selective attention and it has the most weight meaning that it can tell the difference better for gender here - greater difference between the iv groups her answer -- each group of IV's are weighted according to between group variability

Whats a linear compposite cannonical variable forumula? CV = a + w1v1 +w2v2+w3v3--whats w

It's the weight given to each DV as far as how well it discriminates between the IV's. If you had .69stm +.2ltm then .69 would discrimainte between genders better than ltm

sum of squares

Variance = sum of squared eviations about a mean for each pt's score on each dv

purpose of linear composte/ cannonical variable

Weights = maxamize difference between groups CV # used to look group differences -accounted for correlation of DV

Whats a linear compposite cannonical variable forumula? CV = a + w1v1 +w2v2+w3v3--whats a

a = discriminant function constant (y axis/intercept when making a line)example CV = constant + .8 (v1) + .03 (v2) v1 selective attention v 2 stm v 1 = selective attention and it has the most weight meaning that it can tell the difference better for gender here - greater difference between the iv groups her answer -- each group of IV's are weighted according to between group variability

between subjects

a variable on which each subject can be found on only one level of the variable, such as age or gender

methods spasm A

a answer choices

confidence interval (CI)

accounts for error and variation in effect sizes showing you what range of scores, that are distributed around the mean could occur depending on alpha level set --influnces probability -- 95% confident your true value fall under alpha .05

additional assumptios mixed design

additional assumption homogeneity of intercorrelation (f) pattern of intercorrelations should be the same for between and within subjects test with box's m but its senstive for this so use .01 in hopes u dont violate

assumption homogeneity of intercorrelation

additional assumption mixed design pattern of intercorrelations should be the same for between and within subjects test with box's m but its senstive for this so use .01 in hopes u dont violate (f)

assumption homogeneity of intercorrelatio test how

additional assumption mixed design\ test with box's m but its senstive for this so use .01 in hopes u dont violate

q stat and goodness of fit

almost like a model of goodness of fit bc you had to chose which unit of analysis to go with (F) Non-sig Q test = good fit homogeneity of effects homogeniety is good allows you to see IV is having effect not error means studies are from sampe pouplation and you can use group mean effects (f)sig Q test = bad fit hetrogeneity of effects bad -- more error than IV having impact means yur group of studies contains 2 + distinct subpopulations whicu should divide further means you've got a lot of error and you'r iv in there

sum of cross-products**

amt of shared covariance between DVS --so the variance for each subject variation from the mean on 1 DV & their variance on the other DV's and so on variance - squared deviation from the mean

homogenity of effects/distrubtion

are your mean effects/splts of iv = variability to poulation? use Q for homogenity / hetrogenity often people can't get full homogeniety

assumptions anova

assumptions necessary indpendence of observcations can still possibly be okay if violated univariate normality homogeniety of variance look at levene's test if violated other wise no need to mention anything

what to think of faciliators were NOT nested

bad, different groups of people doing different interventions = results not as clear now = nonspecfic factors may be accounting for result = confound - they might not realize/identify it & even if they do they might not measure it.

within groups SSCP total

between looks at group means to grand means so within has to look at indvidual scores to group means and group means tot tal mean for all

moderator vs interventing variable

both started as confound where they both share identify measure Moderator - accounted for Intervneing variable (couldn't acount for it

Mean effect statistically sig??

c sometimes magnitude more interesting than stat sig (like if ci doesn't include 0) need to know about characterstics of studies they should translate it for you

phillai barlett trace - what it is caculates

caculates variance in DV accounted for by greatest seperation of IV sum of explained variance on discrimiant varibles

weighted effects metanalysis

can weight your IV's based on sample size --helps adjust for sampling error --larger more weight bc better estimate of pouplation or maybe you choose study quality or both

interaction effect

cell means going in different directions for each column? if plotted would lines cross? Means level of 1 iv impacting level in other iv

power

chance find sig result when it's really there (f) -violate assumptions > alpha error < power Homogeneity variance --reduce power > alpha error if groups = --if unequal = type 1 error up or down (F) Sig results --low power no sig = maybe not enough pt ---high power no sig = likely not there (f) Increase power >sample size >effect size >less stringent alpha >1 tail Results in > chance of type 1 error

MANOVA procedures looks at between within - elaborate on between

compares differences group mean and grand mean produces single # = determinant considering variance & covariance the larger the # the more your IV had an effect

follow up analysis for sig multivarate effects -multivarate contrast / contrast analysis complex

complex contrast - can compare multiple groups of 2's in al combinations on DV instead of using DV's this looks at IV's comapres groups over set of dV simultaneously simple/pairwise -like huge t-test - 2 groups on combined DV can only compare 2 at a time

step 5 step 5 - analyize ES distrubtion (MR for - requires

continous data think 1+ sig predictor is found and u want to evalute which predictor is most impt hierachiecial prefered - theory enter in order simultaneous not liked as much

Covariate

continous variable sig correlation with DV used to -factor out ANCOVA/MANCOVA --look at for main interest --impacts F 2 ways (f) use only if -- (f)

covariance

covariance the variation in one variable that is shared by another variable Sum of Cross Products = sum of variance for one DV and sthe variance of another variable = index of covariance

2. what's a canonical variable? And how is it created. 1. HER ANSWER

creates linear equatuon weights for each DV each pt get score for that set of variables . Greater the diff between the groups the larger the weight

df IV 4 groups (different climates) DV happy. n = 10 participants - repeated f (3, 36) = 3.43 Wilks or Pilla's value, p <.05, ?2 = . 69 effect size 3, # = df = # of groups -1 4 groups - 1 = 3 #, 6 = df =( # groups * n) - total # of groups 4*10 - 4 note # of groups is 4 not 3

MANOVA looks for diff between groups based on DV by looking at between groups/within groups what goes into between groups?

diff between group mean x grand mean - produces single # called dterminantn larger # better IV

discriminant function coefficent

discriminant function coefficent MANOVA finds a weight for each DV called a discriminant function coefficent . Computer does it . Composite variable created using weights called a canonical variable = composite variable - makes them into 1 see procedures combining dv manova

metanalysi mnemonic -- DICE full think ignore others

disementnaties knowledge identifies impt variables, gaps and future research conclusions drawn but limited to just post tx effect sizes

variance

dispersion of values around a mean aVG of (mean - pt score)^2

q stat sig means what abou tpopulation

distinct subpopulations whicu should divide further means you've got a lot of error and you'r iv in there

look at mean differences what do u do

divides all studies into 2+ groups based on IV thought to contribute to DV

follow up analysis for sig multivarate effects - dv variable contiribution remove 1 variable at a time: technique Wilkson's

do MANOVA'S with 1 DV left out at a time - change in F score shows you which DV contributes the most

follow up analysis for sig multivarate effects - dv variable contiribution 0 what does it do and what does it allow you to see

dv contributuon looks at decrease in multivareate effe as dvs are being taken out shows: lets you see which DV contributes the most

dv contributuon while step down looks at

dv contributuon looks at decrease in multivareate effe as dvs are being taken out while step down looks at effect while adding dvs with account for previous dv

cv is done for

each participant and each dv is weighted

Mahalnous / Mahal's distance - what does it do not what it means

each person gets a deviation score from the centroid which is compared to a critical value to check for outliers

main effect

effect of single iv on dv look at marginal means --column 1 vs 2 on age --diff? main effect --row 1 vs 2 on weight - diff ? main effect

how to interpret cohens d

effect size given in sd units like a z score Interpreted In Sd Units - Like A Z Score looks At Mean Differences - Tells You How Different The Distrubtions Are From One Another Like Grousp Seperated By 1 Sd (F) .2 small .5 medium .8 large ? small = .2 or 85^ overlap between two distrubtions ? moderate .5 67% overlap between two distrubtions ? large - .8 or 53% overlap ..

Cohen's D

effect size given in sd units like a z score looks at mean differences .2 small .5 medium .8 large (f) to find = mean group 1 - mean group 2 / sd spooled sd spooled is the avg sd for two groups

finding effect sizes in studies without means or SD

effect sizes can be caculated without means or sd for signifigant test

DV - / criterion variable metanalysis

effect sizes depending on how you split them could just be (f) 1) each dv in the study 2) each construct in the study 3) the study as a whole 4 )combine your dvs like linear compsosite --

step 5 mean differences - if split wrong

errors if split wrong say you looked at lumpectomy vs mastecoymy and find both - good but then you take that same group and dvide them by family history finding an itneraction if u find a diff u should use the new construct

alpha trouble for anova

experimeterwise -- looks at all of the test your doing to see chance type 1 error Familywise/commonwise - looks at chance type 1 error for 1 test you run -more common in ANOVA correct for: ike tukey, sheffe, least sig (f) dfifference, newman - keuls, and duncans

experimentwise alpha

experimeterwise -- looks at all of the test your doing to see chance type 1 error bc comparing multiple levels of IV with DV

metanalysis typeexploratory

exploratory uses standardzied group mean differences as the index of effect -what relationsips between iv dv are supported

F test just what does it look at not the resulting F

f = within group variance/ total group variance = amt NOT EXPLAINED by DV (wilk's Pilla's) effect size = 1 -wilks or ss between/ss total

file drawer problem

file drawer problem the tendency for authors not to submit and journal editors not to accept for publication the results of experiments that fail to achieve statistically significant findings

random vs fixed -- undelrying theme

fixed - thinks underlying uniting theme - random doesn't

manova debate - criticism against hummel and sligo who said okay to do anovas after manova with out > experimenter wise alpha -- criticims point 3

follow up ANOVAS ignore correlations between DVs doesnt matter still > experimeter error

What's a linear composite / cannonical varriable

formula that combines all dvs into one while weighting ivs taht discriminate the best'(f) all in indvidually after here formula - done for each person CV = a + w1v1 +w2v2+w3v3 uses w = higher the weight the more the DV discriminates between groups IV -- analogous to slope (rise/run) higher scores have more weight meaning that that particular IV is discriminanting better other names discriminant function weight discriminant function coefficent a = discriminant function constant (y axis/intercept when making a line) v - scores for each person on orginal DVs example CV = constant + .8 (v1) + .03 (v2) v1 selective attention v 2 stm v 1 = selective attention and it has the most weight meaning that it can tell the difference better for gender here - greater difference between the iv groups her answer -- each group of IV's are weighted according to between group variability

inferential stats follow...

genreal linear modelline y = a +b (x) b slope, a y intercept x = data point not really what we're doing just as example

inclusion exclusion critiera - part of step 2 metanalysis review randomized vs not : randomized

gold standard - random assignment = > internal validity < external validity bc strict inclusion/exclusion critiera

Wilk's looks for differences where

group means type of F test used when homogeneity of variance not violated MANOVA looks at within group variance/total group variance 1- Wilks = effect size (between/within) --so the smaller the wilks the larger the effect size Looks for differences in group means

test homogeneity of effects

homogeneity of effect sizes with Q sort of like testing for homogeneity of variance - is it error or IV? procedure by looking at SSQ = q between groups / residual variability Q qithin looks at distribution of effeft sizes for group of studies like a chiq squrae variable with # of studies -1 = df use critical value to find sig result needed

one of the requirements for using a covariate relationship between covariate and dv must be = for all groups -- what is it

homogeneity of regression

additional assumption for follow up after manova step down

homogeneity of regression - impt to look at if you do a step down analysis

criticismmetaanlysis

how know u splt right - might compare apples to oranges --accounting for differences in studies metanalysis never better than the indvidual studies that comprise them

adjusted (or unbiased) effect

if have < 30 use adjusted effect which reduces effect size

if mixed design sig interaction

if it was sig you would have to be super careful interperitng main effect bc interaction = impact of 1 influenced by other so general conclusions like main effect might not be approriate pic pic no interaction between program type and time wilks .87, f =2.03 p.15

confidence interval how to know if sig

if range doesnt include 0 --ES of .45 has a CI ranging from .20 to .70), then the mean ES is considered to be statistically different from zero

box m and levene's test if it's sig = if its not sig =

if sig your bad if it's nto sig ogod didn't violate homogeniety of variance

so dvs need to be correlated for MANOVA whats too high and too low what should u do

if they aren't correlated consider seperate anovas for DV's if too highly correlated = multicolineary - happens like if one variable is a combo of your other variables = singularity just do a correlation matix. if abocve .8 or .9 bad

step 2 can't be totally comprehenisive lit review - fail safe n

impossible to be totally comphrensive - file draw/publicatoin bias use fail safe n there is no aboslute standard for assessing what value of the fale safe n is critical do fail safe n after data analysized estimates the # of additional studies that would have to be located to change your result

which is the only assumption that is really bad if you violated

independence of observation - DON'T VIOLATE multivariate normality homogeneity variance, repeated sphericity ok

MANOVA procedures looks at between within - elaborate on within

indvidual scores compared to their group's mean error variance & covariance Between: diff between group x vs total mean

each study should contribute at least 1 effect size if u just do 2+

indvidual studies dvs are often correllated so when u include more than 1 effect size your not accounting for overlap which results in overweighting of study if you think some of measures in single study represent diff constructs that are idnepent of one another then it might be okay to d o more than 1 effect size

chi-square (x2)

inferental stat can transform into probbility level

follow up analysis for sig multivarate effects -multivarate contrast / contrast analysis

instead of using DV's this looks at IV's comapres groups over set of dV simultaneously simple/pairwise (F) -like huge t-test - 2 groups on combined DV can only compare 2 at a time (f) complex contrast - can compare multiple groups of 2's in al combinations on DV

interaction effect

interaction effect 1 iv influences the other you'd want your cell means for each column going in opposite directions -looking at cell means not marginal means if lines eventually corss = interaction 1 main effect an an interaction pic pic explanation main effect you want 1 simillar and 1 dissimillar marginal mean 6 & 5.5 arent that diff from one another but 7 and 4.5 are interaction you'd want your cell means for each column going in opposite directions when u look down the yound colum # get smaller, down the old > larger = interaction you can do a simple effects analysis - see simple effects be careful to remember that if you have sig interaction and you only have 2 levels for one of your IVS you have to look at descriptives, post hoc is for 3 + levels of IV

Expresses mean effect as r would be appropriate when

iv's dvs are contonous

iv dv metaanylis

iv/dv IV data that has been coded into categorical data often representing study features DV (F)effect sizes from indvidual studies remember you're analysis level is 1 up from regular bc your looking at study features and effect sizes (dv) not indvidual participants per se

iv dv other named mixed between subject anova

iv: 2(Cog, Behavioral) x 2 (time period scores on fear) dv fear scores split plot anova SPANOVA

When you look at your alpha what 3 things do you need to remember that ti's associated with

just list top categories - rest in (f) used to evalute p uses confidence intervals assocaited with type 1 error

follow up analysis for sig multivarate effects -multivarate contrast / contrast simple/pairwise

like huge t-test - 2 groups on combined DV can only compare 2 at a time instead of using DV's this looks at IV's comapres groups over set of dV simultaneously simple/pairwise -like huge t-test - 2 groups on combined DV can only compare 2 at a time complex contrast - can compare multiple groups of 2's in al combinations on DV

step 2manova which 3 must u do

lit review - coms search, manua serac, review references - use fail safe n - not use 3 and you could miss something

looking at mean differences step5

looking at mean differences divides all studies into 2+ groups based on IV thought to contribute to DV Controversy on how to split no way to know you grouped right making to broad of categories is bad bc maybe the iv should have been split to see difference bc some other variable accounting errors if split wrong say you looked at lumpectomy vs mastecoymy and find both - good but then you take that same group and dvide them by family history finding an itneraction if u find a diff u should use the new construct

procedures Manova for F

looks at SSCp/between and SSCp within Between - compares group mean and grand mean produces single # = determinant considering variance & covariance the larger the # the more your IV had an effect within (F) looks at indviduals score compared to their group mean considering variance & covariance = error cacculate F to find effect size (f) f = within group variance/ total group variance = amt NOT EXPLAINED by DV (wilk's Pilla's) effect size = 1 -wilks or ss between/ss total

follow up analysis for sig multivarate effects stepdown analysis

looks at effect while adding dvs with account for previous dv get to see unique contribution order put in is impt - need causal logical order (f) dv contributuon looks at decrease in multivareate effe as dvs are being taken out while step down looks at effect while adding dvs with account for previous dv -assumption homogeniety of regresion

doubly multivariate repreated measure manova

looks at mean differences across levels of IV so u can see pre/post test effects so time is one of the ivs

procedure ANOVA

looks at means and mean differences to find out how much your IV worked (to see how much total variation is expalined by your maniupliation) you look at the DIFFERENCE BETWEEN GROUPS attributed to your iv maniuplation looks at group mean compared to grand mean divided by WITHIN GROUPS attributed to natural error compares indviduals score to their group mean

manova assumption: multivareate normality - check for with mahalnovus diance -- Wha'ts it basically doing?

looks at the variation of each persons score from the remaining samples mean to check for outliers

method spasMs

m measures your using, incude demographics and recrutiment

purpose

manova/anova - stat DIFFERENT between groups? main effects/interactions? for DV or composite DV t-test groups stat DIFFERENT from each other? metanalysis (f) summarize research findings shows u what eveyone else is finding impt technique allowing u to draw conclsuons / dissementate knoweldge public poicy implaictions can identify effective programs and highlight gaps limitations in research

follow up analysis for sig multivarate effects -discrimintant analysis

maxamize seperation between groups on IV with linear composite you're basically doing a linear composite for each signifigant effect get to see subsets of construct for which groups differ

metanalysis types

metanalysis types exploratory descriptive treatment effectivness

coding

method by which study features (types of subjects contro group etc) tranformed into quant iv for the stat analysis coding yield combo continous variable for study features like # of med sx and categorical variability respresetning other featur want > 80% interereater realibilit yes like meds meds + cbt

mixed design

mixed design looks at between and within variables

roy's largest root bad or not suitable for

multiple dimensions if it's sig and others aren't then it's ignored bc it shows the upper bound of F

bonferroni inequality - what does it do and why would you need it

multiplies # of test and nominal alpha = max value of alpha/type 1 error for set of test -used when manova NOT sig but still want to do ANOVA'S example (f) true alpha = example for 6 test and nominal alpha .05 = 6 *.05 = 30% or less chance type 1 error for those 6 test

step 2 metanalysis -

must do all 3 ) coms serach, manula search, review of reference -- need to consdier fail safe n

n vs N metanalysi

n in metanalysis refers to # of stuides revieed where N =# subjects unless noted otherwise

What's independence of observations

necessary condition - not in indvidually pt score on DV not influenced by other group don't want 1 level of your IV or another IV influenced by the other find out how use an interclass correlation if correlated what does it mean actual alpha can raise up to 7x nominal level even for a little correlation for manova if violated use group means instead of indvidual scores in your analysis

step 5 mean differences - controversy how to split

no way to know you grouped right making to broad of categories is bad bc maybe the iv should have been split to see difference bc some other variable accounting

types of alphas

nominal - what you set actual - what really occurs experimenterwise - anova/manova etc- type 1 error chance for all test familywise/common wise -- type 1 error for 1 test you run in the family

cohens d score range

not limited to | 1|

inclusion exclusion critiera - part of step 2 metanalysis review randomized vs not : not randomized

not randomly assigned - possible more external validity since randomized often stric inclusion/exclusion critiera oulation

not sig q- good homogenity g - now what

not sig q- good homogenity g can more clearly see IV effect examing mean effects is apporpriate

nonsig box m and levene

not non sig Box's m = didn't violated assumption of homogeniety of vovariance- use wilks Levene's violated assumption of homogeneity of variance - just not it

null hypothesis H0

null hypothesis H0 the hypothesis that states that there is no difference in the mean values of one or more DVs across levels of one or more IVs MANOVA no differences between groups/vectors of means for groups the people who created hope you'd do a random sample so it'd be population means difference = 0 but we use samples so its we expect no difference between group means/vectors of means anova considers indvidual means to look at differences repeated designs consider mean differences

random effects mdoel

often this model is used --random error due to subject level sampling study level sampling doesn't propose single underlying effect size assumes randomly distributed with averages being represted

effect size interpretation large small metaanlysis

often we think the larger effect size the better but not always the case = here a small effect shows that they were close together which is good for psychotherapy

omnibus test

omnibus test a test of the null hypothesis that none of the IVs has an effect on any of the DVs omnibus test test the null hypot can refer to diff test like ANOVA F test - between groups/repeated measures multiple regression chi squaresa more likely to hapeen when testing overall hypothetsis on quadratic statistic (looks at vectors, variance etc.) / maybe > likely with multivariate test types Family: Omnibus test: \ "a test statistic to which multivariate indices can be transformed, to derive a probability levle." AKA looks = between group variance / within group variance Large F = > between = IV had desired effect example say design with 1 iv and 3 dvs --If you did an ANOVA you'd have to do 3 of them bc you only can do 1 DV @ a time > error -- MANOVA = just 1 dig omnibus test (F) with the all those DV's turned into a composite variabe < error in compairson to ANOVA USED IN used in: MANOVA (1+ categorical IV & 2 + continous DV) & ANOVA (1 or 2+ categorical IVS's & 1 continous DV)

on diagnoal = the variance of each DV

confound vs moderator

once was a confound but now identified measured accounted for ---(f) if you couldn't account for it then it's an intervening variable as like as the other conditions are met (identified, measured)

moderator what is it

once was a confound but now identified measured accounted for if you couldn't account for it then it's an intervening variable as like as the other conditions are met (identified, measured)

unit of analysis - when deciding on which IV's to include you need to decide how you're going to combine those DV's for the IVs your interested in..what are your options

option A --1 effect size per study option b - Multiple effect sizes per study --all dv's in one study --each construct presentedi n 1 study --each dv that you're interested is represtented in the study

outlier

outlier an extreme value in a distribution of effects. authors should specify how an outlier is defined. even tho outliers are excluded from eventual analyses so as not to distort the findings, they should be inspected for their heuristic value: outliers might suggest particularly successful or unsuccessful programs

methods spams P

p procedures proper headings?

one of the strongest for protecting against type 1 with small samples

pilla's

p value

probability level --you're result occured by chance -shows you how extreme you're scores are based of your sample. smaller the p more like iv

COHEN'S D can only comare

prupose: allows you compare ONLY 2 GROUPS

anova purpose

purpose Lets You Look At Main Effects, Interactions Groups Sig Different From Each Other? do post hoc to see where differences are Anova Test For Effect Sizes By Looking At Ss Between / Ss Within

manova purpose

purpose can look at main effects and interactions have to do post hoc to see where exact differences are manova test the differences between groups based on the cannonical/linear composite variable diff between subjects SSCP- between / within SSCP subjects looks at vectors of means while accounting for correlation in DV

purposed mixed design

purpose whether there a re main effects for iv's and an itneraction

have to test for what before analysizing effet sizes metaanlysis

q sta required before analyzie - no one gets perfect but can't interpret as well if f -variability in effect sizes for the studies you're looking at compared to the poulation f non sig= homogenous - studies/divisons variation = population f sig = hetrogenous - samples/divisons = 2+ population - divide further or can't interert as confidently bc you don't know which of the populations is causing your effect

effect size r

r by itself (f) ?effect size same as eta <.9 small, .9-2.35 medium tells u strength, direction,

r^2

r^2 effect size % of varriance accounted for r by itself (f) ?effect size same as eta <.9 small, .9-2.35 medium tells u strength, direction,

random vs fixed - think what about their sample studie effecy sizes?

random: --think obtained effect sizes represent avg falling on nomral bell curve fixed think if sample sizes were large enough all effect sizes would be =

if homogeneity of variance is violated what happens (yes I know u know the Wilk's pilla's thing tell more)

reduce power slightly when groups = impacts power/type1 error --if unequal = type 1 error up or down

repeated measures

repeated measures see assumptions manova/anova repeated designs consider mean differences see - doubly multivariate repreated measure manova an experimental design and corresponding analysis in which each subject is measured on the DV for more than one level of an IV (eg. time 1 and time 2) / repeated measures an experimental design and corresponding analysis in which each subject is measured on the DV for more than one level of an IV (eg. time 1 and time 2)

homogeneity of regression

requirements of covariate relationship between covariate and dv must be = for all groups

anova manova procedue

same procedures except manova looks at variance and covariance and anova just looks at variance looks at SSCp/between and SSCp within Between - compares group mean and grand mean produces single # = determinant considering variance & covariance the larger the # the more your IV had an effect within looks at indviduals score compared to their group mean considering variance & covariance = error

Follow up analysis manova choices SC

scam -step down -(dv) Contribution -Anovas or discriminant analysis -multivarate contrast

Follow up analysis manova choices mnemonic

scam -step down -(dv) Contribution -Anovas or discriminant analysis -multivarate contrast

Follow up analysis manova choices scAM

scam -step down -(dv) Contribution -Anovas or discriminant analysis -multivarate contrast

Sig q hetrogeneity bad- what do u do

sig - hetrogeneity bad- your catching error and iv effect and u've got 2 populations - divide further try to idenitfy variability contributing to hetrogeniety

Result of Q non sig

sig Q test = bad fit hetrogeneity of effects means yur group of studies contains 2 + distinct subpopulations whicu should divide further your sample is hetrogenous stat sig q you've got 2+ populations in your sample - yoou need to split futher means you've got a lot of error and you'r iv in there bad - / you're sample is homogenous homogeniety is good allows you to see IV is having effect not error means studies are from sampe pouplation and you can use group mean effects varaibility around effect size is no greater than you'd expect from error good fit 0- ths is what u want

sig Q tes

sig Q test = bad fit hetrogeneity of effectshomogenity of distriubtions bad -- more error than IV having impact means yur group of studies contains 2 + (f) distinct subpopulations whicu should divide further means you've got a lot of error and you'r iv in there

sig> meta

sig> magnitude of effecct can can be sometimes more interesting thatn stat sig (wehther CI includes 0) practical sig depends on several factors stat mag not related to practical impt impt to know charactersitics of otucome measure in which es based they should translate it for u

Univariate/Multivariate normality -what happens if violated

small impact type 1 < power by flatening bell curve - people still run test

method mncmeonic

spasms suitable for current sample procedure/proper heading answer choices sample# questions measures used, demographics, recruitment scoring

Method mnemonic

spasms s scoring p procedures proper headings? a answer choices s suitability for current sample/ realiability of scale validity is a bonus m measures your using, incude demographics and recrutiment s sample item # questions

stanadrized scores

stanadrized scores z d # units away from the mean see effect sizes cohens d

Expresses mean effect how? & consideration of scale of measurement - Cohen's d

standard deviation units so you can compare and combine not limited to | 1 | though most fall within this range Cohens d- for 2 groups only (F) r - for continous variables x^2 for dichotomous variables

what ways can you express mean effect--------- and what considerations should u use for your variables (F)

Cohen's D 0 first step to find # and what they mean<img src="c:\users\chelsey\dropbox\current classes\flashcards for all classes\classes\research\research stats\keyboard.png">

start at 2 and go up to 5 then up to 8 .2 . 5 . 8 small medium large: Left off at 8 <img src="c:\users\chelsey\dropbox\current classes\flashcards for all classes\classes\research\research stats\colored keys.png">

statistical signfigance

statistical signfigance ?compaare p and alpha ?p = < alpha = reject null yay! ?P = > alpha = boo nothing found ?smaller the alpha the harder it is to get statistical significance anova/manova bigger differences between groups than within

step 1 metanalysis

step 1 - hypot and research domain formulate hypot clearly define relevant research domain

step 4 metanlaysis

step 4: Index ES - find mean effect by looking at effect sizes dv and variability - chose proper unit anayis

step 5 meta

step 5 - analyize ES distrubtion (MR for continous data -heirachical better) (Mean difference catgeorical)

step 6 draw conclusions - meta anlsysis

step 6 draw conclusions offer suggestions specific to the lit you evaluted consistent with database limitations offer reccomendations improve future research - useful indicate hwo study artificats in studies could have impacted current findings

non siqn q means what about poulation and generalizability

studies are from sampe pouplation and you can use group mean effects - > generliaze

study artificats

study artificats any error or bias in studies- always assess in conclusions/interpretations

methods spaSms

suitable for popualtion sample item scoring # questions

sum of cross products

sum of cross-products = covariance sum of variance for one DV and sthe variance of another variable = index of covariance find sd for each person (indvidual score deviation/grand mean)

metanalysis typedescriptive

summarize broad doesnt consider implications / draw conclusuions

q staistic looks @

test like homogeneity of variance -- is the variance from systematic differences or sampling error (f sall below) by looking at SSQ = q between groups / residual variability Q qithin looks at distribution of effeft sizes for group of studies like a chiq squrae variable with # of studies -1 = df use critical value to find sig result needed

Stoffer method

test mean effect sig (f) --convert to Z scores and add compare to p value

determinant

the single # that is produced for a numerator and denominator when looking @ b/w diff

effect sizes show you

the strength between iv/dv

moderator variable theoretically

theoretically studies with diff level of same moderate should differ in effect size

what kind of variables are usaully moderators

those that are unmaniplated like ses

how to find cohens d

to find = mean group 1 - mean group 2 / sd spooled sd spooled is the avg sd for two groups

total group variance

to find the overall total group variance for all DVS for between/within on MANOVA look at SSCP total which compares indvidual scores to group mean

metanalysis typetreatment effectivness

treatment effectivness impact of tx type questions framed in realtive terms like does a help b better than nothing at all

alpha trouble familywise correct for how

tukey sheeff duncan kehuls

type 2 error

type 2 error accepting the null hypothesis when it is really false

roys largest root looks at and as a result - good

type of F test looking at largest eignvalue if u have a single dimension & strong correlations > power specificity

hotellings lawel trace T^2 type purpse

type of F test 2 groups compared on linear composite = most sig linear combination of the DV for 2 groups

Wilk's

type of F test used when homogeneity of variance not violated MANOVA looks at within group variance/total group variance 1- Wilks = effect size (between/within) --so the smaller the wilks the larger the effect size Looks for differences in group means (f)

simple effects

type of analysis done for signifigant interactions on ANOVA. you do anova's across and if sig t test down

types anova

types 1 way ANOVA - 1 way 1 IV 3+ lcategorical levels & 1 continous dv 2 way 2way anova 2: IV's categorical IVS and 1 continuous DV REPEATED ANCOVA

types of metanalysis

types exploratory descriptive treatment effectivness

types of manova

types mancova doubly repeated measures manova 2 way 1 way manova

follow up analysis for sig multivarate effects - dv variable contiribution remove 1 variable at a time: technique Huberty

use F to remove statisic for each DV then test sig of decrease when removed from entire set rank orders dv by impt

x^2 chi square Expresses mean effect how? & consideration of scale of measurement

used when iv/dv dichotomous result chi square

CV = constant + .8 (v1) + .03 (v2) v1 selective attention v 2 stm whats this mean?

v 1 = selective attention and it has the most weight meaning that it can tell the difference better for gender here - greater difference between the iv groups her answer -- each group of IV's are weighted according to between group variability

SD variability unit of metaanlaysis

variability allows you to see how much IV working ES .05 sd = 98% of all effects range between -.5 and 2.5

homogeneity-heterogeneity

variability distrubtion of ESS for group of studies just sampling error variance = homogenous-good hetrogenoug = sample error and IV = bad

moderator variable

variable accounts for sig variability in effect size theoretically studies with diff level of same moderate should differ in effect size Usually exist when the variable is unmanipulated. like ses or degree relationship to confound/intervening variable (F)

q compares variability in what to what

variaiblity in effect sizes for studies/ivs you haave created compared to the poulation

sphericity assumption Violation =

variance in all differences between all combo's of related groups isn't = required for repeated manova test for with maulchys test of sphereicity > type 1 error

between groups

variation accounted for by IV (f) SSCP total = between/within where between looks at difference between group mean x grand mean (F) Effect sizes = between/within where between = SS between/between subject variance

Homogeneity of variance (anova) or covariance (manova - just general information

variation between groups not = if violated (F) reduce power slightly when groups = --if unequal = type 1 error up or down (f) MANOVA -- use pilla's if violated, wilks if not --homogeneity of covariance --vectors of means = variance and normal distrubtion? (f) ANOVA - Levene's test --homogeneity of variance variance in test population = sample? homoschedastic

weight etaalysis

weights each study often has dif sample sizes larger sample size better estimate of poulation than small so need to consider sample size when avg by weighting each effect size you could also weight by other impt indices like by quality of study

within subject

within subject an IV used in such a way that DV values are obtained for every level of the within subject variable

type 1 error

yay i found it ./.. but not really there

metanalysis - you'll be looking at diff types of effect sizes in each study so what do u do

you can combine them since they are standardized even if different types like cohens or R

homogenity of distriubtions - METAANLYSIS

you want to make sure the varaince in your sample/way ou parced it for IVS is = to the poulation use q stat homo good

Voir tous les ensembles d'études

RM 2 exam 1 - reading charts

Ensembles d'études connexes

10.4.14 Application Development and Security Section Quiz

Prepositions, adverbs, and verbs

unit 15 terms great depression

integrated bus quizzes

Vital Signs Chapt. 29

Biology 1110 Exam #2

Natural Language Processing (AI)

Chapter 15 (Critical Thinking)

Introduction to Management

Accounting Chapter 8-Proprietorship, Partnerships and Corporations

Exam 2 - Practice Problems, Quizzes

Question Bank Quizzes

Promulgated Froms

MidTermCOP

Guide To Computer Forensics and Investigations 5th Ed Chapter 1 Review Questions

Chapter 11: Statement of Cash Flows

annuities

Chapter 43 - Restorative and Esthetic Dental Materials

Chapter 8 Checkpoint Questions

Exam 5 Psychology Part 2