Evaluation

Validity (Brown & Abeywickrama, 2010) 5 types of Validity:

1. Content-related 2. Criterion-related 3. Construct-related 4. Consequential Validity 5. Face Validity

Interactive Reading Types of Interactive Reading tasks?

1. cloze tasks 2. short-answer tasks 3. Impromptu reading/ comprehension 4. editing

Apply Interactive Reading task- ***********************************

1. students put in groups/ pairs and given two paragraphs worth of text cut up and separated. 2. students required to put the strips of paper in order to create a story.

How many institutions/ countries accept the TOEFL?

10,000 ; 130 countries

Validity (Brown & Abeywickrama, 2010) 5. Face Validity

"How do people view the test?" "the degree to which the test looks right" (Mousavi, 2009). -cannot be empirically measured -cannot be theoretically justified - simply how the test-taker/giver perceives a test *many test experts do believe that face validity can have an effect. ***Psychological states can also affect test performance (i.e. confidence, anxiety) "a pernicious fallacy that should be purged from the technician's vocabulary" (Bachman, 1990).

Validity (Brown & Abeywickrama, 2010) 3. Construct-related Validity "attempts to explain ____________________ phenomena."

"attempts to explain *observed phenomena."

Validity (Brown & Abeywickrama, 2010) Define

"the extent to which a test measures what it is supposed to measure"

Validity (Brown & Abeywickrama, 2010) 1. Content-related Validity ?

*Test content are only based on the materials/textbooks in class. - tests assess REAL course objectives -test taker PERFORMS the behavior being measured i.e. Low Content Validity = oral test asking students to do cloze task. High Content Validity= oral test triggering oral production previously practiced in class.

Validity (Brown & Abeywickrama, 2010) 3. Construct-related Validity

*attempts to explain observed phenomena. -not really used with classroom teachers! "Does this test actually tap into the theoretical construct as it has been defined? i.e. Oral Fluency test situation: *Scoring analysis includes pronunciation, fluency, grammatical accuracy, and vocabulary use. Low Construct Validity: only asked to evaluate grammar and fluency. High Construct Validity: asked to evaluate ALL factors in the construct.

Genres for Interactive Reading?

- narratives - directions -recipes -questionnaires/ announcements

Extensive Speaking (Brown, 2010) Examples of Extensive Speaking tasks

- oral presentations: presenting a report, an idea, talking about one's family or hometown. - picture- cued tasks: learner given a sequence and prompted to narrate the story using the pictures (important for teacher to utilize new vocabulary and grammar structures in doing this) - speeches - retelling a story: Learner reads or hears a story and they have to retell/ summarize it.

Who takes the TOEFL?

- students planning to study at a higher institution - English language learning programs entrance/ exit - scholarship opportunities - students who want to keep track of their English learning - people applying for visas or for jobs

Validity (Brown & Abeywickrama, 2010) 2. Criterion-related Validity

-learners who pass the test actually have the skills they are intended to have. the right criteria! -"extent to which the criterion has actually been reached" Asks the question of whether or not passing tests actually mean mastery of content.

Validity (Brown & Abeywickrama, 2010) Characteristics of a valid test:

-measures what it is supposed to measure -does not measure irrelevant variables - relies on empirical evidence - supported by theoretical rational

Extensive Speaking (Brown, 2010) Define Extensive speaking activities are activities that that include ___________________________________________.

Extensive speaking activities are activities that that include extensive oral production.

Extensive Speaking (Brown, 2010) Extensive speaking tasks include complex and lengthy stretches of _______________________. The language is *deliberately planned.

Extensive speaking tasks include complex and lengthy stretches of *discourse.

Extensive Speaking (Brown, 2010) Apply!

Picture cued task for students about making an appointment; goes through all the steps/ vocabulary for going to the doctor (medical unit) *student describes what is happening in the pictures. Oral Presentation- presentation about their hometowns or families, hobbies, etc.

Validity (Brown & Abeywickrama, 2010) 4. Consequential Validity Micro-level vs. Macro-level?

Related to the impact that a test will have on a student (micro), or on a school/ society (macro). macro- impact on education/ society a. standardized admissions tests deprive students of opportunities to learn. b. socioeconomics deprive some learners of the costly test-prep opportunities. micro- impact on learner * WASHBACK! - influence of testing on teaching. a. negative washback- students only want to learn what is on the test. b. positive washback- TOEFL exercises such as writing samples are independent writing tasks-> simply writing an essay enhances skills.

TOEFL iBT what does it stand for?

Test of English as a Foreign Language (internet based)

Extensive Speaking (Brown, 2010) in these types of oral productions the listeners have limited _______________________ or only respond in a nonverbal way.

in these types of oral productions the listeners have limited *interaction or only respond in a nonverbal way.

How many people have taken the TOEFL iBT?

over 30 million people

Validity (Brown & Abeywickrama, 2010) How to apply Validity in CR assessment

Content- use direct testing, examine how tests assess course objectives. Criterion- use comprehension checks after the test to reflect results. Construct- N/A not measurable-> "observed phenomena" Consequential/ Face Validity - ensure students are familiar with test formats - give clear directions - give ample time to complete tests.

Interactive Reading Define Interactive reading is a type of reading in which the reader must ______________ with the text

Interactive reading is a type of reading in which the reader must *interact with the text

Interactive Reading Reading is a process of ___________________ meaning

Reading is a process of *negotiating meaning -reader brings knowledge to the text.

How long is each section?

Reading: 60 minutes (36-42 questions) Listening: 60 minutes (34 questions) Speaking: 20 minutes (6 questions) Writing: 60 minutes (2 questions; 1 integrated, 1 prompt)

Interactive Reading Texts are ____________ in length ( - pages)

Texts are *medium in length ( 1-2 pages)

How is the TOEFL scored?

The TOEFL is scored from 0-120 ; not pass/fail. Each institution has a different score requirement (i.e. UCF requires an 80 for entrance into the UCF TESOL program).

What does the TOEFL measure?

The TOEFL measures the test taker's ability to use and understand English at the university level. There are four sections on the TOEFL: Reading, Listening, Speaking, and Writing.

Validity (Brown & Abeywickrama, 2010) Mihai's example: a placement test would have ___________ construct validity, but __________ content validity.

a placement test would have *high construct validity, but *low content validity.

Validity (Brown & Abeywickrama, 2010) 2. Criterion-related Validity (2 types of criterion-related evidence)

a. Concurrent- results supported by previous performance. b. Predictive- predicts/determines a student's readiness to "move on" to the next unit. **Purpose is to predict a student's future success. ** the predictive validity of a standardized test is VERY important.

Interactive Reading more focused on top-down processing or bottom up?

top-down -focus on word meaning

How long is the TOEFL?

up to 4 hours