Assessment and Learning Analytics

Formative Assessment

"Formative assessment is a planned, ongoing process used by all students and teachers during learning and teaching to elicit and use evidence of student learning to improve student understanding of intended disciplinary learning outcomes and support students to become self-directed learners."

As part of an introductory icebreaker activity on an online discussion board, students must write a complete sentence about their morning. The sentence must use correct spelling, grammar, and punctuation. One student writes, "i ate breakfast B4 school". Initially, the teacher provides this feedback: "The requirements state that you must use correct spelling, grammar, and punctuation." Which additional statement would make this feedback more effective?

"Rewrite your response with the first letter of the sentence capitalized, proper spelling, and a period at the end."

Students in an ethics course are assigned to read about various moral theorists and principles and then select the philosophy they believe is correct, defending their selection. One student explains that the philosophy of pleasure-seeking is the best choice. The instructor responds by commenting, "Your selection is selfish. I disagree." Why is this instructor's evaluative feedback biased?

Because it implies there is a correct answer, when morals are personally determined

A learning goal for a group of kindergarten students is "Students will be able to differentiate how objects are alike and different." A teacher provides each student with a sorting mat and a bag filled with tiles in a variety of colors. The teacher asks students to sort the tiles by color on their mats. Why does this performance-based assessment align with the learning goal and the Analyze level of the revised Bloom's taxonomy?"

Because students demonstrate their ability to distinguish attributes of tiles

A teacher of a first-year German class is interested in determining the impact of using online learning games on vocabulary knowledge. How could this teacher use learning analytics with the purpose of determining the efficacy of the learning games?

By comparing assessment scores from students who did and did not use the online learning games

5 practices of formative assessments

Clarifying learning goals and success criteria within a broader progression of learning, Eliciting and analyzing evidence of student thinking, Engaging in self-assessment and peer feedback, Using actionable feedback, Responding by adjusting learning strategies or next instructional steps

Comprehensive-Type Assessment Methods

Comprehensive-type assessments provide various ways for the instructor to monitor a learner's academic achievement and progress. These assessments include benchmark, formative, summative and diagnostic. An example is a multiple-choice test covering an outcome and all its learning objectives.

In a college chemistry course, students construct a model of a benzene ring. The learning objective is "Investigate the properties of a benzene ring." Which cognitive level of the revised Bloom's taxonomy does the students' construction of the model reflect?

Create

One of the learning objectives of a ninth-grade anatomy unit about the systems of the body is "Students will be able to describe the functions of the respiratory system." Which project-based assessment would determine whether students have met this objective?

Creating a song with lyrics about the role of the respiratory system

Criterion-Referenced Assessment

Criterion-referenced assessments measure student learning based on a concrete learning standard, objective, or outcome. These assessments measure each learner on individual performance, not against the performance of other learners (norm-referenced)

A fourth-grade teacher is piloting an enrichment unit on fractions with like and unlike denominators for remote learners. The modules are accessed through a learning management system (LMS) and are arranged in a specific instructional sequence. The teacher is uncertain whether the modules are effectively sequenced. Which data measure should be used to determine if the learning modules are effectively sequenced?

Current scores from end-of-module quizzes

A special education school administrator is concerned that students with special needs are not making adequate progress in developing literacy skills. The administrator sets a goal that the literacy scores of students with disabilities will increase by 12% by the end of the following school year. Which type of learning analytics should this special education administrator use when reviewing performance levels during the year?

Descriptive

A principal is concerned that 14% of students in a high school are absent five or more school days per month. The principal sets a goal to decrease the overall absenteeism rates by 10% between the third and final term. Which type of learning analytics should this principal consider when trying to determine why some students have been chronically absent?

Diagnostic

What are the three types of tests that will be used by instructional designers?

Diagnostic, Formative, and Summative

Direct Assessment Methods

Direct assessments are used to evaluate a learner's understanding of a concept, achievement of a learning objective, or completion of a goal. The instructor evaluates the learner's work to reach a conclusion about the learner's achievements. Direct assessment involves evaluating work produced by a learner. Formative assessments such as weekly quizzes and worksheets are examples of direct assessment. Summative assessments such as capstone projects, presentations, essays, and tests are also examples of direct assessment. Learners must demonstrate knowledge to the evaluator.

Discussion Board

Discussion boards allow learners to create posts (in writing, audio, or video) on topics relevant to a course's learning objectives. When used well, a discussion board can create a sense of community, increase collaboration among learners, and deepen thinking, analysis, use of evidence, and other skills.

A site-based decision-making team for an urban school works with a principal to establish a campus improvement plan that will target academic areas that need improvement based on academic performance data. Which ethical obligation should this team consider in the allocation of the funds?

Ensuring a portion of the funds are used to address achievement gaps

An individual has been hired to train an organization's administrative staff to use a new automated scheduling system. The training runs for three days. The individual wants to know the progress of each participant at the end of the first day and use that information to plan the second day's instruction. Which type of assessment should be used to obtain this information?

Formative

An instructor is teaching an eight-week clinical health course to college students. At the beginning of the course, each student performs a complex clinical procedure while being recorded. At the end of the course, students perform the same procedure while being recorded. The students compare these performances and reflect on how their ability to perform the complex procedure has progressed. Which assessment strategy does this describe?

Ipsative

ipsative assessment

It measures student learning against past knowledge or performance. Ipsative assessment can help motivate learners by showing the distance progressed, even if the learner's performance is not yet at the level of an expert or some classmates.

Students in a business writing course complete a series of writing practice quizzes using a self-paced approach. Quizzes are automatically scored in a learning management system (LMS), but no grade is entered into the grade book until an instructor reviews and releases the score. The instructor commits to releasing the score and providing effective feedback within 24 hours after the LMS scores a quiz. The following feedback is provided to the first student who completes a quiz: "Awesome work! You missed only 2 of the 35 questions." How should this feedback be modified in order to be more effective for students?

It should describe in some detail how the student's performance can be improved.

4 Tips for Summative Assessments

Make grading rubric available in advance, Analyze the results of the summative assessment to identify trends and areas of improvement, Distinguish between formatvie and summative assessments, Use a wide range of assessments to cater to different learning styles.

In a nursing course, students examine body systems, including the nervous system. The learning objective is "Interpret terms related to the nervous system." An instructor for the course wants to determine students' ability to interpret these terms accurately. Which skill demonstration aligns with this learning objective?

Matching nervous system elements on an image of the human body

A high school history teacher administered a summative exam that required students to answer short-response items focused on the causes of the Revolutionary War. The responses were graded with a rubric on the following categories: accuracy, rationale, and length. One student received the following written feedback from the teacher: "The information you provided for each question minimally met the standards. However, you had some spelling errors. For example, the following words were misspelled: revolutionary, adequate, colonies, and applicable. Please revise and resubmit." Is this feedback effective?

No, because spelling was not a category in the rubric

Students in a healthcare capstone course prepare for a required postgraduation licensing exam by completing a nationally validated competency exam that serves as practice. The students' practice exam performance results are available within 10 days of the exam date to the program chair but not to the capstone instructor. The program chair briefly reviews the exam scores and tells the capstone instructor that the results are around the expected average for healthcare students. Was the program chair's decision to provide a summary interpretation of the practice exam scores ethical?

No, because the interpretation does not provide enough information for the instructor to share specific performance insights with each student

norm-referenced assessment

Norm-referenced assessments use assessment score results to create a comparative score of how learners did relative to the scores of other learners. For example, a grade of 85, normally considered a B for above average, might be recorded as a C if 85 is the average score for all learners taking the assessment.

Competency-Based Assessement

One trait of competency-based assessments is the focus on skills more than knowledge. These assessments are often an example of authentic assessment because they center on learners applying skills and knowledge by creating the kinds of artifacts and work used in the field being studied. These assessments are often constructed via scenarios or simulations that place the work students must do in a context that requires students to use judgment, weigh approaches, and hone critical thinking and other higher-order skills.

5 Types of Summative Assessments

Online Multiple Choice Exams, Online Presentations, Creating a Website or Blog, Learners' Online Portfolios, Online Group Projects

Freshman college students are required to take a course that introduces them to the college experience during their first semester. The course includes an initial assessment that measures how the students' knowledge of resources supports their academic performance. The assessment is administered online and includes questions that require students to press and hold the Control key while selecting answers in a drop-down menu. Specific directions are provided for these items. Additionally, a graphic that demonstrates the process is provided. Which high-level principle of functional accessibility is a concern in this assessment?

Operable

A lesson objective for a ninth-grade history class is "Students will be able to explain how the preamble serves as an introduction to the United States Constitution." Which assessment would align with this objective and the Understand level of the revised Bloom's taxonomy?

Paraphrasing the preamble to the United States Constitution

A history curriculum department for a K-12 school district has created a history fair project: Sixth-grade students will present a historically significant character as part of a living museum. To determine how prepared students are for this project, teachers in the district have asked students to demonstrate their speeches before the fair. The teachers use a rubric to assess each student's presentation as meets expectations or below expectations in four categories: accuracy, presentation, scope, and bias. The scores are entered into a learning management system (LMS) for review by a history department coordinator to determine which schools need additional support. Which type of learning analytics does this process represent?

Prescriptive

Project-Based Assessment

Project-based assessment is a concrete way to assess the learner other than a test. Creative outlets also open the door to all learning types in this method. An example would be to ask students to create a website.

A third-grade reading teacher is developing a reading assessment to predict student preparedness for the state standardized test in making inferences. To ensure that the assessment is aligned with state standards, the teacher selects items from past state assessments. The teacher plans to provide the English learners (ELs) with a bilingual dictionary to use during the assessment. Which action does this teacher take to address the accessibility of this test?

Providing EL students with bilingual dictionaries

Reflection-Focused Assessment

Reflection-focused assessment allows the learner to look back and reflect on the learning experiences. The reflection can also become the basis for planning what to focus on in the future. This method promotes self-regulation of learning, as learners self-evaluate and make plans for accomplishing future goals. Because learners are accessing personal performance, this is a form of indirect assessment.

Several students who are enrolled in an online business communications course do not have access to a computer and instead interact with the course using their smartphones. An assessment in the course requires students to access five websites from links provided on the course site and compare the communication styles of the sites' content. Some of the linked websites have not been optimized for viewing on a smartphone, and several students are unable to experience the features of the websites using their smartphones. Which high-level principle of functional accessibility is missing in this course site?

Robust

Standards-Based Assessment

Standards-based assessment is often used interchangeably with criterion-referenced. Criteria, after all, are based on standards. To help tell these terms apart, consider the difference between the hyphenated words referenced and based. A criterion will often reference a standard. So, in a rubric, for example, there might be a criterion for the ability to use evidence in a presentation.

Summative Assessment

Summative assessment in eLearning is used to determine whether or not a learner achieved the learning objectives and reached the desired level of proficiency. Typically, a summative assessment is administered at the end of an eLearning course

Traditional Assessment

The traditional assessment describes assessment practices you are most likely used to from your years in school. It is a catchall term for common assessment options such as multiple-choice, true/false, fill-in-the-blank items. What makes these options traditional instead of authentic or competency-based is how the assessments are used. Frequently there is one right answer, even for written tests. The format often encourages students to cram for the test (and then, after cramming, to forget what was learned.) When used in what are meant to be formative contexts, the tests can become rote or perceived by learners as busywork.

What are diagnostic assessments used for?

These assessments can also identify incorrect learning behaviors, misconceptions the learner may have, and skill sets that need to be developed. It also gives you the opportunity to learn as much as possible about your audience, so that you can create more meaningful and memorable eLearning experiences.

A goal of a second-grade science unit is "Students will be able to categorize a variety of objects according to observable characteristics." After demonstrating how certain objects placed in an aquarium float and how others sink, a teacher decides to assess the students' conceptual knowledge. Before placing the next item into the aquarium, the teacher asks students to stand if they predict it will float and to remain seated if they think it will sink. Which cognitive levels of the revised Bloom's taxonomy do this learning goal and direct assessment align with?

Understand and Apply

Indirect Assessment Methods

Unlike direct assessment, the indirect assessment does not look at learners' actual work but instead uses information gathered from other sources. With some learning management systems, information for indirect assessment increases: attendance is mapped to log in data and time on task measures, for example.

A state board of education requires that all schools in the state determine the languages spoken in student homes to identify their language needs. In one elementary school, data from a "Home Language Survey" indicate that 11% of students speak a language other than English in their homes. The school's principal asks an English language teaching specialist to administer an English language proficiency (ELP) assessment to determine whether these students are eligible for language support services in school. Did the principal fulfill the ethical obligation to act on the knowledge about students that was gained through the initial survey?

Yes, the principal attempted to gather additional information to understand the students' specific needs.

Diagnostic Assessment

often referred to as pre-assessment, is designed to test a learner's knowledge BEFORE they begin an eLearning activity.

What are the 6 types of diagnostic assessments?

online journal, online quizzes, online scenarios and simulations, Mind Mapping, Surveys, focus groups and online interviews.