Assessment and Learning Analytics D293: Pre-Assessment

A third-grade reading teacher is developing a reading assessment to predict student preparedness for the state standardized test in making inferences. To ensure that the assessment is aligned with state standards, the teacher selects items from past state assessments. The teacher plans to provide the English learners (ELs) with a bilingual dictionary to use during the assessment. Which action does this teacher take to address the accessibility of this test?

Providing EL students with bilingual dictionaries

Several students who are enrolled in an online business communications course do not have access to a computer and instead interact with the course using their smartphones. An assessment in the course requires students to access five websites from links provided on the course site and compare the communication styles of the sites' content. Some of the linked websites have not been optimized for viewing on a smartphone, and several students are unable to experience the features of the websites using their smartphones. Which high-level principle of functional accessibility is missing in this course site?

Robust

Students are assessed with an at-home science project in which they must use the steps in the scientific method to test whether bean pods can grow without direct sunlight. The students will report the results of their projects in a month. What is a content-validity bias issue in this assessment?

Some students may not have access to proper supplies.

A short-answer analogy test includes the following item: House is to mansion as boat is to ____. (The correct answer is yacht.) Which modification to this assessment would eliminate bias

Square is to four sides as triangle is to ____ (three sides).

Second-grade students are learning about common and proper nouns in their English language arts class. At the end of a lesson, a teacher distributes electronic clickers to the students, lists one noun at a time on a whiteboard, and asks students to select whether the noun is a common or proper noun. Which learning objective aligns with this assessment and the Analyze cognitive level of the revised Bloom's taxonomy?

Students will be able to differentiate between common and proper nouns in sentences.

A short-answer analogy test includes the following item: House is to mansion as boat is to ____. (The correct answer is yacht.) What is the bias issue in this item?

The content relies on language suited to specific interests and experiences.

An online college course uses a digital textbook and online practice quizzes that students can take at any time to determine their competency. The responses are available to both the students and their instructor. What can be determined from the learning analytics data measures from these practice quizzes to help the instructor decide how to help students meet the competency goals of the course?

The content that students have mastered

Learning analytics from an online course indicate that some students are successfully completing discussion board assignments and exams without first reading the course materials in the modules. How can the learning analytics data be interpreted for the purpose of adjusting this course?

The data indicate that the course materials should be better aligned with the assignments and assessment activities

During the first week of a 12-week semester, students in an online accounting course complete an orientation. The orientation includes a knowledge check of the students' ability to locate specific course information and access navigation menu items. However, three weeks into the course, students are asking the instructor how to access assignment information. The instructor would like to provide further direction to these students. Which data measure should the instructor review to determine how to help these students?

The frequency of students access to course activity content

Students in a management principles course develop a presentation related to the application of four principles. The course instructor reviews each student's presentation draft to ensure it adheres to the presentation requirements. Then the instructor enters feedback in the form of a letter grade in each student's grade book. The instructor wants to make the feedback more effective by adding a descriptive comment. What should the comment in this modified feedback describe?

The student's progress toward the learning goal and how the presentation might be improved

The learning goal of a fifth-grade math unit is "Students will be able to correctly identify place values from one to ten million." A teacher for the course conducts informal checks for understanding by asking students to complete exit tickets, submit self-reflections, and share their responses using individual whiteboards. What is the teacher's purpose in collecting this formative assessment data?

To monitor student learning to guide instruction

A high school yearbook teacher teaches six classes with a total student count of 130 students. Students are required to submit monthly proposals for yearbook articles. The proposals represent students' monthly grade. Students' proposals must be accepted to receive a passing grade for each assignment. Which monthly process would provide meaningful formative guidance for the students?

Two weeks before the proposal due date, the teacher plans a peer-review session for students to score and discuss the proposals.

A goal of a second-grade science unit is "Students will be able to categorize a variety of objects according to observable characteristics." After demonstrating how certain objects placed in an aquarium float and how others sink, a teacher decides to assess the students' conceptual knowledge. Before placing the next item into the aquarium, the teacher asks students to stand if they predict it will float and to remain seated if they think it will sink. Which cognitive levels of the revised Bloom's taxonomy do this learning goal and direct assessment align with?

Understand and Apply

College applicants who score below acceptable limits on the math portion of the ACT are required to take a computerized math basic skills assessment to avoid remedial math courses. To pass the assessment, students must answer 70% of the questions correctly. Students have three opportunities to pass the test, and they receive feedback electronically after the first and second attempts. One student received the following feedback with attached models and practice problems in deficient areas (i.e., areas below 70%): Quantity 75% Algebra 64% Functions 86% Geometry44% Statistics and Probability44% Is this quality feedback?

Yes, because students are provided with specific and timely information regarding their performance and practice problems

Students in a business course are required to calculate an organization's break-even analysis. The students are given a formula, via a learning management system (LMS), and they use the formula throughout the course. Several weeks after the formula is introduced, an instructor notices a student who continues to submit assignments with incorrect calculations. Consequently, the instructor recommends that student support services and the student meet to review the student's inability to calculate correctly. Was the instructor's decision to contact support services ethical?

Yes, because the instructor is obligated to recommend support services for students who demonstrate a lack of understanding

A physical education teacher challenges remote learners to improve their speeds when completing a one-mile run. Students wear fitness trackers that automatically record their times in an online database where the students can review their progress. At the end of each week, the teacher sends an email to each student. One email reads, "You have been committed to the process of improving your speed. This week's data show that you have improved your speed by 28 seconds—wow! For next week, try to maintain a steady pace until the last two-tenths of a mile. Then try sprinting to the finish line." Is this feedback specific and actionable?

Yes, it mentions the learning goal, the progress made, and a plan for moving forward.

. During one simulation, a dental assistant student teaches a patient about oral hygiene and plaque-control techniques but does not provide an opportunity for the patient to ask questions. The instructor offers the following feedback to the student:"You gave a model demonstration for providing oral care information to a patient using easy-to-understand language, and you enhanced your verbal description with a simulation of techniques. Patients are more likely to apply techniques when they fully understand them. They must be invited to ask questions at the time the information is presented. How might you ask patients whether they understood the presented technique?" Is this feedback specific and actionable? Why, or why not?

Yes, the instructor identified the student's progress toward the learning goal and an opportunity for improvement.

A state board of education requires that all schools in the state determine the languages spoken in student homes to identify their language needs. In one elementary school, data from a "Home Language Survey" indicate that 11% of students speak a language other than English in their homes. The school's principal asks an English language teaching specialist to administer an English language proficiency (ELP) assessment to determine whether these students are eligible for language support services in school. Did the principal fulfill the ethical obligation to act on the knowledge about students that was gained through the initial survey?

Yes, the principal attempted to gather additional information to understand the students' specific needs.

An online music class teacher directs students to play a specific song in unison using their recorders. Informal observation data indicate that several students are not keeping the beat while playing. Which teacher feedback meets the goal of providing formative guidance?

"I noticed that several students were effectively keeping the beat by nodding their heads to the beat. I would like everyone to try nodding their heads as they play next time."

Students in a math class submit a homework assignment about adding decimal numbers on an online learning platform. While reviewing their work, the students' teacher noticed they did not line up the numbers and decimal points in the assignment correctly. The online learning platform automatically marks the incorrect answers with an X. The teacher follows up with these students by emailing a comment: "Your numbers and decimal points were not aligned." Which statement should be added to this feedback to improve its effectiveness?

"Please use the graph paper technique when completing the next problem set to ensure the numbers and decimal points are aligned."

As part of an introductory icebreaker activity on an online discussion board, students must write a complete sentence about their morning. The sentence must use correct spelling, grammar, and punctuation. One student writes, "i ate breakfast B4 school".Initially, the teacher provides this feedback: "The requirements state that you must use correct spelling, grammar, and punctuation." Which additional statement would make this feedback more effective?

"Rewrite your response with the first letter of the sentence capitalized, proper spelling, and a period at the end."

Fifth-grade students are learning about the three branches of government during a social studies class. At the end of a lesson, a teacher asks students to complete a 3-2-1 writing assignment in which they list the three branches of government, two facts about each branch, and one question. With which learning objective would this direct assessment method align?

"Students will be able to recall the features of the three branches of the United States government."

Students in a web design course are assigned to develop a custom logo for their website. One student uses an offensive image as the basis for the logo. Which instructor feedback would help motivate this learner to recognize the importance of inclusivity in design?

"The purpose of your website is to promote your business. Reconsider the basis of your logo so that you do not offend potential customers."

An American Sign Language instructor designs an assessment activity where pairs of students meet on a video conferencing platform to participate in a friendly conversation using basic signs. The instructor will use a checklist to assess students' use of basic signs. Which feedback meets the goal of motivating learners?

"Your skillful use of facial expressions will continue to serve you well in expressing the meaning of the signs."

Students in a second-grade class are learning how to tell time. The learning objective of a lesson is "Students will be able to identify the correct time on an analog clock." At the end of the lesson, a teacher distributes a scaled survey to the students so they can self-assess their confidence in reading an analog clock. Why does this indirect assessment method align with the learning objective?

Because it allows the teacher to gather information on the students' perceptions about their own learning

Students in an ethics course are assigned to read about various moral theorists and principles and then select the philosophy they believe is correct, defending their selection. One student explains that the philosophy of pleasure-seeking is the best choice. The instructor responds by commenting, "Your selection is selfish. I disagree." Why is this instructor's evaluative feedback biased?

Because it implies there is a correct answer, when morals are personally determined

A learning goal for a group of kindergarten students is "Students will be able to differentiate how objects are alike and different." A teacher provides each student with a sorting mat and a bag filled with tiles in a variety of colors. The teacher asks students to sort the tiles by color on their mats. Why does this performance-based assessment align with the learning goal and the Analyze level of the revised Bloom's taxonomy?"

Because students demonstrate their ability to distinguish attributes of tiles

An organization provides a six-week conceptual customer service training program to its customer service specialists. After four weeks, participants apply the presented concepts by completing a multiple-choice assessment in which customer service scenarios are randomly presented. While 65% of participants earned at least the minimum score to pass the assessment, 35% were unsuccessful. How could this organization redesign the test to improve its fairness?

By allowing participants to select from a set of presented scenarios and answer the corresponding questions

Freshman college students are required to take a course that introduces them to the college experience during their first semester. The course includes an initial assessment that measures how the students' knowledge of resources supports their academic performance. The assessment is administered online and includes questions that require students to press and hold the Control key while selecting answers in a drop-down menu. Specific directions are provided for these items. Additionally, a graphic that demonstrates the process is provided. How could the instructor improve the functional accessibility of this assessment?

By changing the format of the questions to allow for a single keyboard click to select answer choices

A teacher of a first-year German class is interested in determining the impact of using online learning games on vocabulary knowledge. How could this teacher use learning analytics with the purpose of determining the efficacy of the learning games?

By comparing assessment scores from students who did and did not use the online learning games

Students in an introduction to philosophy class are struggling to understand the complex information provided in the course readings. The students' instructor asks teaching assistants to facilitate tutoring sessions so students can receive support while completing their reading assignments. How can this instructor use learning analytics for the purpose of determining the impact of the tutoring sessions on the students' comprehension of course readings?

By comparing the students' grades from before and after the tutoring sessions

A trainer identifies several participants in an advanced, self-paced Excel course who have not completed a complex table activity at the end of the first unit. The trainer examines the performance dashboard in the learning management system (LMS) and determines that these participants attempted the complex activity but quit after several unsuccessful attempts. How should the trainer use data measures to foster successful learning for all participants?

By identifying participants who should participate in an adaptive learning experience in the LMS that includes customized instruction

An instructor identifies several students in a college math course who have not completed their weekly assignments at the end of the first three weeks. The instructor determines that these students begin the weekly activities but stop at the point where they must apply the presented math concept. The instructor would like to help foster successful learning for these students. How should this instructor use data measures to do this?

By identifying students who should participate in an adaptive learning experience in the LMS that includes additional practice

In an online college speech course, an instructor encourages students to work with their peers to develop speech topics and to rehearse their speeches. How could this instructor use diagnostic learning analytics to determine whether students are participating in group work?

By identifying the number and type of student interactions

A learning goal for a middle school music class unit is "Students will create music using computer-based composition tools." After the class spends several weeks practicing using the composition tools, an instructor for the course assigns a project in which students must compose an original song that is one minute long, incorporates at least two instruments, and includes a repeating chorus. How would this competency-based assessment strategy determine whether students met the learning goal?

By providing an opportunity for students to demonstrate their computer-based composition skills

Students at a high school are encouraged to enter the school's annual essay contest, which will be judged by a committee composed of an English teacher, guidance counselor, and student council advisor. The English teacher is pleased to see that many of the students who have entered are in the teacher's class. Before the committee reviews the essays, the teacher announces that one particular student deserves to win because the student is a good writer and a hard worker. How should this committee proceed to avoid confirmation bias?

By redacting students' names from the essays before the review

Students in a drama class are participating in a unit on character portrayal. To assess their ability to act in character, the students' teacher drafts brief character descriptions and asks them to randomly select one and then assume that role. Some examples of these descriptions include a stay-at-home mom baking cookies for her children, an Asian math student celebrating a perfect math test score, and a male driver who refuses to ask for directions when lost. How could the teacher modify this assessment to minimize bias?

By revising the character descriptions so they do not include generalizations

In a college speech course, students must prepare a speech in which they compare the field goal percentage performance of the current NBA championship team's point guard to the field goal percentage of the team's power forward. How could this assessment be redeveloped to reduce the barrier to fairness?

By using language that is suited to broad interests and experiences

Following a teacher's presentation of a science lesson, students access an online module that reviews the content presented and captures their responses to related questions. After reviewing the students' responses, the teacher determines that most students have not correctly answered questions associated with the lesson's learning objective. How could the teacher use prescriptive learning analytics for the purpose of creating a modified version of this lesson that will have a greater impact on this same group of students?

By using the identified weaknesses of the students' collective performance to adapt the original lesson presented

The learning goal of a seventh-grade geometry unit is "Students will be able to calculate the surface area of given geometric figures." The students' teacher is planning an assessment that aligns with the Apply cognitive level and the Procedural knowledge dimension. Which assessment would align with this learning goal, the Apply cognitive level, and the Procedural knowledge dimension?

Calculating the surface area of several shapes

To meet the learning goal of starting an IV, nursing students must understand the different functions of two types of blood vessels: veins and arteries. Which assessment aligns with the Understand taxonomy of this learning objective?

Classifying vein and artery characteristics

In a college chemistry course, students construct a model of a benzene ring. The learning objective is "Investigate the properties of a benzene ring." Which cognitive level of the revised Bloom's taxonomy does the students' construction of the model reflect?

Create

The learning goal for a sixth-grade language arts unit is "Students will be able to compose various types of poetry." Which competency-based assessment strategy would determine whether students met the competency requirements for this learning goal?

Creating a portfolio of poems the students have written

One of the learning objectives of a ninth-grade anatomy unit about the systems of the body is "Students will be able to describe the functions of the respiratory system." Which project-based assessment would determine whether students have met this objective?

Creating a song with lyrics about the role of the respiratory system

A fourth-grade teacher is piloting an enrichment unit on fractions with like and unlike denominators for remote learners. The modules are accessed through a learning management system (LMS) and are arranged in a specific instructional sequence. The teacher is uncertain whether the modules are effectively sequenced. Which data measure should be used to determine if the learning modules are effectively sequenced?

Current scores from end-of-module quizzes

A data analyst for a school district is asked to provide information to the district's superintendent on how each school in the district performed on the math portion of a state standardized test. Specifically, the superintendent wants to know which schools had fifth-grade students solve two-step math word problems with at least 70% accuracy on the test. The data analyst plans to summarize the schools' performance data using graphs. Which type of learning analytics does this assignment represent?

Descriptive

A special education school administrator is concerned that students with special needs are not making adequate progress in developing literacy skills. The administrator sets a goal that the literacy scores of students with disabilities will increase by 12% by the end of the following school year. Which type of learning analytics should this special education administrator use when reviewing performance levels during the year?

Descriptive

In an introduction to programming course, students analyze the structure and components of a program. The learning objective is "Create a simple program." Which direct assessment method would determine whether students have met this objective?

Developing program code

A principal is concerned that 14% of students in a high school are absent five or more school days per month. The principal sets a goal to decrease the overall absenteeism rates by 10% between the third and final term. Which type of learning analytics should this principal consider when trying to determine why some students have been chronically absent?

Diagnostic

The parents of an elementary school student are concerned that their child is being bullied and decide to contact the principal for more information. They ask to review the discipline records of the two students in their child's class to determine if the students have histories of bullying behavior. Which ethical consideration should lead the principal to deny this request?

Discipline records are confidential and private.

A high school chemistry teacher has developed several assessments that measure four different objectives. The teacher plans to use the assessments in the upcoming year to measure the students' preparedness for the summative state chemistry assessment. What could this teacher do to decrease the predictive-validity bias of the created assessments?

Ensure students take the assessments under the same conditions that the state assessment is administered

A site-based decision-making team for an urban school works with a principal to establish a campus improvement plan that will target academic areas that need improvement based on academic performance data. Which ethical obligation should this team consider in the allocation of the funds?

Ensuring a portion of the funds are used to address achievement gaps

A curriculum specialist is developing an online assessment that checks students' preparedness for a state math assessment. Which feedback provides the most meaningful information?

For each incorrect answer, students are provided with a video that demonstrates the correct way to answer the question.

An individual has been hired to train an organization's administrative staff to use a new automated scheduling system. The training runs for three days. The individual wants to know the progress of each participant at the end of the first day and use that information to plan the second day's instruction. Which type of assessment should be used to obtain this information?

Formative

The learning goal for a fifth-grade language arts unit is "Students will be able to use context clues to identify the meanings of unfamiliar words." Using a traditional assessment strategy, how should an instructor determine if students met this learning goal?

Have students complete a multiple-choice vocabulary quiz at the end of each chapter

An instructor is teaching an eight-week clinical health course to college students. At the beginning of the course, each student performs a complex clinical procedure while being recorded. At the end of the course, students perform the same procedure while being recorded. The students compare these performances and reflect on how their ability to perform the complex procedure has progressed. Which assessment strategy does this describe?

Ipsative

A high school chemistry teacher has developed several assessments that measure four different objectives. The teacher plans to use the assessments in the upcoming year to measure the students' preparedness for the summative state chemistry assessment. What is a construct-validity bias issue with this plan?

It has not been determined that the created assessments measure the students' knowledge of the objectives.

Students in an online creative writing class must develop an original story about a superhero. A teacher for the class reviews the drafts and finds that one student did not include all of the required information. The teacher inserts a comment into the student's live, shared document that says, "You successfully named your superhero and explained how she discovered her powers. To earn full credit, please add two more details about your superhero's appearance and give an example of how she uses her powers for good." Why would this comment be considered quality feedback?

It provides specific information about revisions that are needed.

The learning goal for a research foundations course is "Apply a variety of approaches to identify and solve problems." The summative assessment for the course involves applying these approaches to a case study. Why would this authentic assessment strategy be effective in measuring students' achievement of the learning goal?

It requires students to apply acquired knowledge to realistic situations.

Students in a business writing course complete a series of writing practice quizzes using a self-paced approach. Quizzes are automatically scored in a learning management system (LMS), but no grade is entered into the grade book until an instructor reviews and releases the score. The instructor commits to releasing the score and providing effective feedback within 24 hours after the LMS scores a quiz. The following feedback is provided to the first student who completes a quiz: "Awesome work! You missed only 2 of the 35 questions." How should this feedback be modified in order to be more effective for students?

It should describe in some detail how the student's performance can be improved.

The undergraduate students in a developmental psychology course are learning about genetics. The objective of a lesson in the course is "Students will be able to identify whether traits are genetic or not genetic." Before class is dismissed, an instructor for the course distributes a list of 10 traits and asks students to circle the traits that are genetic and cross out the traits that are not. Why would this direct assessment be effective in determining whether students met the learning objective?

It would allow students to demonstrate their ability to differentiate traits.

A trainer has been hired to facilitate a digital image editing skills workshop. The learning objective is "Identify the basics of image file formats." Participants use software and images provided by the workshop sponsors. Which direct assessment method would determine whether participants met this objective?

Matching common image file formats to their purpose

In a nursing course, students examine body systems, including the nervous system. The learning objective is "Interpret terms related to the nervous system." An instructor for the course wants to determine students' ability to interpret these terms accurately. Which skill demonstration aligns with this learning objective?

Matching nervous system elements on an image of the human body

A high school history teacher administered a summative exam that required students to answer short-response items focused on the causes of the Revolutionary War. The responses were graded with a rubric on the following categories: accuracy, rationale, and length. One student received the following written feedback from the teacher: "The information you provided for each question minimally met the standards. However, you had some spelling errors. For example, the following words were misspelled: revolutionary, adequate, colonies, and applicable. Please revise and resubmit." Is this feedback effective?

No, because spelling was not a category in the rubric

Students in a healthcare capstone course prepare for a required postgraduation licensing exam by completing a nationally validated competency exam that serves as practice. The students' practice exam performance results are available within 10 days of the exam date to the program chair but not to the capstone instructor. The program chair briefly reviews the exam scores and tells the capstone instructor that the results are around the expected average for healthcare students. Was the program chair's decision to provide a summary interpretation of the practice exam scores ethical?

No, because the interpretation does not provide enough information for the instructor to share specific performance insights with each student

Marketing administrative staff are required to participate in a self-paced software training course. There are two components to the final assessment. One is an objective, multiple-choice test that assesses knowledge of vocabulary and resources. The other is a hands-on practical application of software skills. At the end of the course, a trainer must provide all performance results to a marketing manager. The trainer notices that course participants earned high scores on the multiple-choice test but low scores on the practical application, suggesting participants may be unable to apply the software skills effectively in their work environment. To highlight the strengths of course participants, the software trainer reports only the multiple-choice scores to the marketing manager. Was this trainer's decision to limit the reported results ethical?

No, because the trainer has an obligation to provide complete assessment information

Freshman college students are required to take a course that introduces them to the college experience during their first semester. The course includes an initial assessment that measures how the students' knowledge of resources supports their academic performance. The assessment is administered online and includes questions that require students to press and hold the Control key while selecting answers in a drop-down menu. Specific directions are provided for these items. Additionally, a graphic that demonstrates the process is provided. Which high-level principle of functional accessibility is a concern in this assessment?

Operable

A lesson objective for a ninth-grade history class is "Students will be able to explain how the preamble serves as an introduction to the United States Constitution." Which assessment would align with this objective and the Understand level of the revised Bloom's taxonomy?

Paraphrasing the preamble to the United States Constitution

A history curriculum department for a K-12 school district has created a history fair project: Sixth-grade students will present a historically significant character as part of a living museum. To determine how prepared students are for this project, teachers in the district have asked students to demonstrate their speeches before the fair. The teachers use a rubric to assess each student's presentation as meets expectations or below expectations in four categories: accuracy, presentation, scope, and bias. The scores are entered into a learning management system (LMS) for review by a history department coordinator to determine which schools need additional support. Which type of learning analytics does this process represent?

Prescriptive

Employees in an organization are not completing a training course on their first attempt, resulting in additional costs for the organization. A training manager wants to anticipate the likelihood of participants' failure while participants are engaged in the course and recommend customized remediation that is embedded throughout participants' first attempt at the course and is based on each participant's needs. Which type of learning analytics should this trainer use to suggest various actions to participants to help them complete the course successfully on their first attempt?

Prescriptive