D293 - Assessment and Learning Analytics Quiz

Students are instructed to predict what will happen if they attempt to grow bean seeds without direct sunlight for a specified growth period. One student group hypothesizes that their bean seeds will not grow without sunlight. Which instructor feedback statement would be meaningful?

"List the steps you can take to test your hypothesis."

Predictive-validity bias (or bias in criterion-related validity) -

refers to a test's accuracy in predicting how well a certain student group will perform in the future.

Construct-validity bias -

refers to whether a test accurately measures what it was designed to measure

Students in an online humanities course are assigned to listen to and compare samples of three types of music: opera, classical, and jazz. What should be provided to students with hearing loss to increase the accessibility of this assignment?

A description of each sample

A fifth-grade health teacher forgets to make paper copies of the midsemester multiple-choice assessment. The teacher is required to administer the test by the end of the day; consequently, the teacher decides to project the assessment using the class smart board and have the students write their answers on notebook paper. What is a test administration concern for this assessment?

Accessibility

A learning experience designer meets with a school principal who seeks data for improvement. The principal wants to know if a proposed new e-reading curriculum for the fourth grade will improve reading comprehension measures.

Arrange for each fourth-grade classroom to complete one lesson from the proposed curriculum and compare the scores to student scores on the same lesson from the current curriculum.

A high school art teacher teaches students to create portfolios to demonstrate their skills. Students are required to submit monthly artifacts to include in their final portfolio. The artifacts represent students' monthly grade. Students' artifacts must be accepted to receive a passing grade for each assignment. Which monthly process would provide meaningful formative guidance for the students?

At the end of each month, students are provided with a detailed explanation of why their artifacts were accepted or not accepted.

The following is a question on a middle school American history test: Who won the American Civil War? A. The Union (The North) B. The Confederacy (The South) C. Americans D. No one won About a third of the students chose "C" as the answer and got upset when told their answer was incorrect. Why did students choose "C" instead of the right answer?

Because it's too close to the right answer.

A prospective pilot who is seeking licensure is required to complete a final assessment where they are required to fly a plane with an instructor. The assessment is measured on concrete learning objectives and the student will need to perform a variety of skills to determine whether or not they are able to move on to the next phase of the training program. This assessment can best be described as this type of assessment:

Criterion-Referenced Assessment

How can prescriptive learning analytics data from course evaluations in higher education be used for the purpose of making improvements to a course?

Critical comments can be analyzed to determine areas in the course that may need revision.

A learning experience designer has changed the learning resource for this semester's Chapter 2 test, but the test remains the same. The learning experience designer wants to know if students scored better on this semester's test than previous semesters to determine if the new learning resource should be kept. What data strategy is being implemented?

Data for improvement

A lesson objective for a twelfth-grade physics class is "Students will be able to explain how the laws of gravity apply to the tides and currents of the ocean." Which assessment would align with this objective and the Understand level of the revised Bloom's taxonomy?

Describe how the tides and currents of the ocean are effected by gravity.

The student decides what he or she learned and how well it was learned. Examples: surveys, teaching evaluations.

Indirect Assessment

A teacher assigns a 5th grade student an online reading software that provides different e-books with built in assessments throughout. As the student progresses through a book, a learning quiz will pop up to check for understanding. If the student gets the question correct, the student will continue to progress on through the e-book. However, if the student answers incorrectly, the software will point the student directly back to the previous chapter where the content from the learning check was covered. Students get a gold star for each learning check that they master, but will lose that star if they answer the question incorrectly and are forced to go back. Which type of assessment is built into this reading software?

Ipsative

A teacher assigns an essay during the first week of school to assess students' writing abilities. The teacher provides the following prompt: "Share what you did during summer vacation by describing one of the following: (1) a long-distance trip you took, (2) a fancy restaurant you visited, or (3) a major league sporting event you attended." How is this assessment biased?

It does not consider socioeconomic differences among the students.

Students in a physical education class must write an essay about their favorite athlete. A teacher for the class reviews the drafts and finds that one student did not include all of the required information. The teacher writes a comment on the draft that says, "You did a great job in discussing who your athlete is and what sport they play. However, please add four more details about the athletes life and provide a picture from the internet." Why would this comment be considered quality feedback?

It includes specific detail about changes that are needed.

Students in a poetry course complete a series of writing practice quizzes using a self-paced approach. Quizzes are automatically scored in a learning management system (LMS) with feedback provided by the instructor. The following feedback is provided to the first student who completes a quiz: "You got 8 out of 10 questions correct--great job!." How should this feedback be modified in order to be more effective for students?

It should discuss how the student's performance can be improved.

A high school music teacher administered a summative exam that required students to answer short-response items focused on the impact of technology in music. The responses were graded on a rubric scale that measured accuracy of information, discussion of impact, and a minimum word count of 500 words. One student received the following written feedback from the teacher: "The information you provided barely met the standards. You also had some grammar issues. For example, while you included some great information on the impact of technology in music, you incorrectly used the words their/there/they're and had multiple run-on sentences. Please revise and resubmit." Is this feedback effective?

No, because grammar was not a category in the rubric

What is a sign that an assessment includes barriers to fairness?

The scores of a certain ethnic or cultural group are lower.

_________________ assessment offers recommendations based on possible outcomes and helps identify the best options. What should be done? Examples: automate decisions, recommend actions, automate notifications, determine actions, outcome.

A Prescriptive

A graduate student is conducting research for a professor who wants to write an article about the positive impact of mentorship on scholarship recipients. The professor has advised the graduate student to only collect and record data that shows a positive relationship. The graduate student is worried that the research is not ethical and approaches the professor to explain that there is an issue with ___________.

confirmation bias

As part of a computer coding course for high school students, all learners are required to provide a discussion board post about their daily activity using coding software. The post must include at least three sentences and use correct spelling, grammar, and punctuation. One student writes, "I wrote sum code last nite". Initially, the teacher provides this feedback: "You did not complete the assignment correctly." Which additional statement would make this feedback more effective?

"Please include at least 3 sentences. Rewrite your response with the correct spelling of "sum" and nite"

Students in a yearbook club are assigned to develop a custom cover for the cover of the yearbook. One student includes an offensive image within their design. Which instructor feedback would help motivate this learner to recognize the importance of inclusivity in design?

"We want to insure that we are being inclusive to all students at the school. Consider changing the image on your cover so that you do not offend potential readers."

A foreign language instructor designs an assessment activity where pairs of students meet on a video conferencing platform to participate in a friendly conversation using basic vocabulary. The instructor will use a checklist to assess students' use of conversational dialog. Which feedback meets the goal of motivating learners?

"Your skillful use of vocabulary will continue to serve you well in effectively expressing what you would like to say to your partner."

An assessment question for an introduction to communications class requires learners to watch a video of a speech and critique the speaker's tone, eye contact, and use of idioms. The video has closed captions, and students have time in the class computer lab to complete the assessment. Why would this assessment pose a barrier for some students? Choose 3 answers.

- Closed captions do not provide information about speakers' eye contact. - Tone is not conveyed in closed captions. - Idioms are challenging for English learners to identify.

Bryan leads an HVAC company and wants to select a learning analytics type that will maximize the company's goals. What are two actions that Bryan must incorporate from the company's training practices to ensure the correct learning analytics type is chosen?

- Pick a strategy and invest in resources and instruments. - Create a program that analyzes data solicited from trainees who completed the training program.

Which two goals align with prescriptive analytics?

- Plan how to achieve an 80% passing rate on chapter tests. - Determine how to increase the number of students who will graduate next year.

_________________ assessment is used to inform; based on data from gathered information. What happened? Examples: business metrics, graphs, storytelling, specify how data is collected and used to inform business decisions. specify how data is collected and used to inform business decisions.

A Descriptive

_________________ assessment will inform teachers of student achievement gaps and other academic abilities. Why did it happen? Examples are finding anomalies, finding patterns, data discovery, time-series data.

A Diagnostic

_________________ assessment uses data from the past to predict what might happen in the future. What is likely to happen? Examples: detect future trends, predict future outcomes, forecast events that will occur at a specific time.

A Predictive

At the end of a grammar unit for second-grade students, a teacher administers an assessment to determine the students' ability to write complete sentences. Learning analytics show that 25% of all students do not demonstrate competency. How could this instructor use prescriptive learning analytics in order to customize instruction and address this problem?

By determining individual skills that need reteaching by categorizing common errors

A trainer is responsible for teaching the features and capabilities of a proprietary software system to large groups of new employees with widely varying technology skill levels. How can this trainer use prescriptive learning analytics to enhance individual employees' learning experiences?

By developing personalized learning at scale

Middle school students watch a video with embedded quizzes for a social studies assignment. How could the students' teacher use descriptive learning analytics with the purpose of determining individual student progress with the presented content?

By examining student engagement with the video and quiz question responses

Students in an introduction to marketing course took an online assessment requiring them to discuss the marketing techniques demonstrated in a set of statements. The assessment had to be completed within a specific period of time and without access to resources. More than half of the students inaccurately discussed the technique associated with the following statement: "You must rely on the esoteric interests of a group to place your product." How could this assessment be modified to reduce bias?

By using language suited to the students' level of knowledge and experience

A trainer in a large organization teaches evidence-based selling techniques to sales agents after they have completed their first year of employment. Following the first unit of training, participants access an online module that reviews the content presented and captures participant responses to related questions. After reviewing the participants' responses, the trainer determines that most participants have not correctly answered the questions associated with the unit's learning objective. The trainer is required to show that 80% of the participants correctly respond to 80% of each unit's questions. How could the trainer use prescriptive learning analytics with the purpose of presenting a modified version of the unit that will have a greater impact on this same group of participants?

By using the identified weaknesses in the participants' collective performance to adapt the first unit of training

The learning goal of a seventh-grade science unit is "Students will be able to calculate the density of a given object." The students' teacher is planning an assessment that aligns with the Apply cognitive level and the Procedural knowledge dimension. Which assessment would align with this learning goal, the Apply cognitive level, and the Procedural knowledge dimension?

Calculating the densities of several objects from the student's home.

Robust -

Can I access the content with my technology? Technology capabilities

To see if a learner can perform the kind of skill needed for the workplace. Usually considered an authentic assessment. Example: Creating a presentation Goal: Learners will be able to deliver a training talk on student-centered learning for first- year teachers.

Competency-based Assessment

Grades are due at the end of the week, and a teacher realizes that the students in their class still need to take an end-of-chapter quiz, a unit test, and a midterm exam. The teacher explains to the students that they should study hard each night because they will be taking these three assessments over the next three days. How could this assessment plan be modified to improve the students' experience?

Consolidate the content to lessen the cognitive load and alleviate student stress

_________________ bias happens when tests are more difficult for certain groups of students because of their culture or experiences.

Content-validity

In a college biology course, students construct a model of the nervous system in the human body. The learning objective is "Investigate the functions of the nervous system." Which cognitive level of the revised Bloom's taxonomy does the students' construction of the model reflect?

Create

(a) A university publishes its graduation rate on its website. The graduation rate is 5% lower than it was last year. (b) Data indicate that the graduation rate dropped mainly because of the pandemic. (c) Data indicate that, due to the continuing strains and challenges from the pandemic, the university anticipates lower graduation rates for the next two years. (d) To support students toward their goal to graduation, data suggest students need more faculty support than they did prior to the pandemic so the university updates their faculty support policies. The statement labeled (a) is which type of data analytics?

Descriptive

____________ ____________ answers the question: "What happened?

Descriptive analytics

(a) A university publishes its graduation rate on its website. The graduation rate is 5% lower than it was last year. (b) Data indicate that the graduation rate dropped mainly because of the pandemic. (c) Data indicate that, due to the continuing strains and challenges from the pandemic, the university anticipates lower graduation rates for the next two years. (d) To support students toward their goal to graduation, data suggest students need more faculty support than they did prior to the pandemic so the university updates their faculty support policies. The statement labeled (b) is which type of data analytics?

Diagnostic

A company has concerns about employee turnover. The learning and development team decides to analyze employee morale information from the past to find out why employees are leaving the company at such a rapid rate. Which type of learning analytics should the learning and development team use?

Diagnostic

An instructor is developing a new set of training modules for a 3-day professional development seminar for employees but is unsure which topics to cover. Which of the following assessments would give the instructor the most effective information on which topics to include for the training?

Diagnostic

____________ ____________ answers the question: "Why did it happen?

Diagnostic analytics

A professional makes a decision regarding what a student learned and how well it was learned. Examples: faculty evaluated papers, tests, or performances.

Direct Assessment

Administrative support staff in an organization are learning how to prepare digital presentations through an online course. The course is mostly self-paced; content is delivered by a learning management system (LMS) but facilitated by a trainer. New to the training curriculum is an enhanced coaching component. Coaching consists of regularly scheduled one-on-one sessions where course participants meet with the trainer for an hour each week. While the enhanced coaching experience increases the course length, the trainer notes the productive working relationship between coaches and course participants and believes it may positively influence participants' application of digital presentation skills after the training. Two months after the course ends, the organization's human resources (HR) department asks participants to describe how their participation has enhanced their ability to prepare digital presentations. What should the trainer do to determine whether the longer training period and enhanced coaching are valuable to employees and the organization?

Examine total course hours per participant and associated participant feedback collected by HR

A learning experience designer is developing an online assessment that checks employees preparedness for a a competency-based human resources assessment that is now required by the state.

For each incorrect answer, learners are provided with an instructional video that provides an explanation of the correct answer.

A math teacher is instructing students on a 5-day unit on the Pythagorean theorem. The teacher wants to know how each student is progressing at the end of each day and use that data to adjust instruction for the following day. Which type of assessment should the teacher use to obtain this information?

Formative

An eighth-grade teacher notices that students are struggling to submit the homework. The teacher wants to use direct data collection methods to determine whether students are still retaining the content even though assignments are not being turned in. What could the teacher do to get the necessary data?

Give the students a pop quiz that covers the objectives of the test.

A corporate office is providing sales training to its employees. The two-hour long training features a guest speaker. The office has two employees who are Deaf, so a transcript of the speaker's speech was provided. After the training, each employee will be asked to participate in a mock sales call to demonstrate the new knowledge. What can be changed to make the training accessible?

Have an American Sign Language interpreter at the training.

A teacher uses a rubric to assess students' political cartoon assignments in three areas: message, content, and visual presentation. One student scored four points for the message, four points for the content, and three points for the visual presentation. In addition to providing a score of 11 out of 12 points, the teacher included the following feedback for the student in a comment box: "Please review your classmates' political cartoons." Is this an example of quality feedback?

No, the feedback does not include specific, actionable statements relative to the learning goal.

A college Philosophy professor received feedback on his most recent assessment from students that the test was too hard and there was not enough study material provided. The data from this assessment does show an irregularity, as the scores from this exam overall are much lower than previous ones that the professor has given. To provide some flexibility and support his students, the professor has decided to grade this assessment on a curve to help booster the students' struggling GPA's. This assessment can now best be described as a...

Norm-Referenced Assessment

After reading books about different animals and viewing paintings of animals in an online art class, second grade students are asked to paint a picture of another animal for a homework assignment. Why is this assessment inaccessible?

Not all students will have access to painting supplies.

Students are assigned to write a paper or create a presentation on their family's history. How can barriers to fairness be mitigated in this assessment?

Offer students the option to research another family's history.

Can I use the interface or will I have trouble operating. Use of hands

Operable

Barton is a first-grade student who has repeated struggles in mathematics. Barton seems to have a good understanding of basic mathematics facts, but has difficulty with subtracting multidigit numbers. Which type of data can be collected to identify how to help Barton?

Patterns of incorrect answers

A chart describes the change in a population over time. If a user cannot see the chart, how can she understand the data? Which principle is considered in this example?

Perceivable

Can I perceive the information? Eyes/Ears

Perceivable

On an online test, students are given a picture of a partially completed chart of the periodic table and asked to "fill in all of the missing element names." The blank spaces are numbered, and students are asked to type the name of the element next to the corresponding number on the chart. Students who cannot use the mouse are able to use keyboard controls to complete the test. Which POUR principle needs to be addressed in the above assessment scenario?

Perceivable

(a) A university publishes its graduation rate on its website. The graduation rate is 5% lower than it was last year. (b) Data indicate that the graduation rate dropped mainly because of the pandemic. (c) Data indicate that, due to the continuing strains and challenges from the pandemic, the university anticipates lower graduation rates for the next two years. (d) To support students toward their goal to graduation, data suggest students need more faculty support than they did prior to the pandemic so the university updates their faculty support policies. The statement labeled (c) is which type of data analytics?

Predictive

____________ ____________ answers the question: "What is likely to happen?"

Predictive analytics

(a) A university publishes its graduation rate on its website. The graduation rate is 5% lower than it was last year. (b) Data indicate that the graduation rate dropped mainly because of the pandemic. (c) Data indicate that, due to the continuing strains and challenges from the pandemic, the university anticipates lower graduation rates for the next two years. (d) To support students toward their goal to graduation, data suggest students need more faculty support than they did prior to the pandemic so the university updates their faculty support policies. The statement labeled (d) is which type of data analytics?

Prescriptive

____________ ____________ answers the question: "What should be done?"

Prescriptive analytics

Each week in an organizational management course, students are required to view recorded simulations of conflict resolution scenarios and evaluate the manager's performance in each scenario. The recorded simulations offer transcripts of the dialogue. Which modification would improve accessibility?

Providing audio descriptions that present the visual-only content in the videos

Which analysis is used to determine why something happened based on non-numerical information, such as observations, reflections, and interviews.

Qualitative analysis

When using learning analytics to assess student learning in online courses, which learning analytics technique is often used with descriptive analytics to conduct item analysis of multiple-choice formative assessments?

Quantitative analysis

Which analysis is based on objective, numerical data, and statistics. Uses Descriptive analytics to determine what happened.

Quantitative analysis

A teacher notices that a former straight-A student has started to fail tests. The teacher wants to personalize the lesson to help the student succeed. What assessment data should the teacher use?

Review and compare cumulative test scores.

A website requires a specific version of a web browser to make use of its features. If a user doesn't or can't use that browser, how can that user experience the features of the site? This example presents a problem with which principle?

Robust

Which analysis is used to predict future interactions, the study of patterns or trends in relationships among groups of learners or between learners and instructors.

Social network analysis

Prospective teachers in a new, accelerated-licensure program have the opportunity to enter the classroom early as a full-time teacher if they are able to demonstrate their mastery of certain pedagogical skills. Many students in this new program have experience in schools as either substitute teachers or in after-school programs. To obtain full-licensure, students must pass 6 classroom observations by a clinical supervisor who will assess them on a variety of skills including classroom management, questioning techniques, and social-emotional learning. These classroom observation assessments can best be described as a...

Standards-Based Assessment

How do students benefit when Universal Design for Learning (UDL) guidelines are applied to assessment design?

Students face fewer barriers.

A lesson objective for a 5th grade social studies class is "Students will be able to recite the capitals of all 50 states." Which assessment would align with this objective and the Remember level of the revised Bloom's taxonomy?

Students will label each state's capital on a map.

After all teachers in a school completed a 5 hour professional development training on classroom management, a high school principal wants to understand how much information his teachers have retained. This will help him better understand how effective the training was and what topics still need further instruction. Which of the following types of assessment would be best for the principal to determine how well his teachers have understood the content from the training program?

Summative

An instructor wants to determine whether to continue using a new textbook in a course or return to last year's textbook. What kind of assessment data should the instructor use to inform this decision?

Test performance compared to last year

What type of learning analytics data could this professor use for the purpose of identifying areas that need improvement for students to be more successful?

The mean score for each assessment in the course

A high school history teacher has taught the same group of students all year. The teacher is familiar with the work habits of each student and, at times, has been able to guess the score that each student will receive on an assignment based on its level of difficulty. For their final project, students are required to submit a research paper that highlights a historical event or figure. Students have two weeks to submit the paper. Which scoring bias issue may be a concern in this situation?

The teacher may score the assignment based on prior knowledge of the students.

Norm-referenced

To see how a learner's work compares to the average work completed by a similar group of learners. Example: Test graded on a curve Goal: Learners will be able to identify and solve engineering problems that may occur in the field.

Standards-based

To see if a learner can meet requirements and have a mastery of knowledge based on a predetermined standard. Example: State Standardized Test Goal: Learners will master grade-level mathematics.

Traditional

To see if a learner can meet the requirements based on memorization of data and facts. Example: Multiple Choice, True/False Goal: Learners will be able to describe and identify parts of the human nervous system.

Ipsative -

To see if a learner has improved based on previous knowledge. Sometimes AWARD based. Example: Compare previous work and progress to current work Goal: Learners will increase reading and comprehension skills.

Criterion-referenced -

To see if a learner has met predetermined milestones and requirements. Example: Advanced Placement Test Goal: Learners will be able to apply the basics of calculus.

Can I use the content explained at a level I understand? Think, Cognitive level

Understand

A learning experience designer is analyzing data from a study about the effectiveness of server training in a 24-hour restaurant that focuses on upselling desserts. Servers attended training and consented to have their sales data analyzed. The data being analyzed will include the average number of dessert items sold per customer. The study is not taking the time of the order into consideration. Which ethical issue is present in the study?

Validity and reliability of data

The following prompts are designed for students in an online writing course. Which assessment prompt is biased?

Write about your favorite experience in the snow.

In a business management course, students must research and write a paper analyzing the founders of modern management principles. To ensure students approach the assignment as intended, the course instructor requires students to submit an outline of their papers. The instructor reviews each outline and provides feedback to help students develop a complete, clear analysis. Which feedback provides students with formative guidance?

Written comments in each student's outline identifying the student's progress and opportunities for enhancement

A basketball coach challenges students to improve their accuracy when shooting free-throws. Students use a shooting machine that automatically tracks their accuracy in an online database where the students can review their progress. At the end of each training session, the coach sends an email to each student. One email reads, "You have been committed to the process of improving your skills. This week's data show that you have greatly improved your accuracy by 10 percent. For next week, try to work on improving your shooting form by keeping your elbow tucked. Then make sure you are following through after each shot you attempt." Is this feedback specific and actionable?

Yes, it mentions the learning goal, the progress made, and a plan for moving forward.

Item-selection bias -

a subcategory of content-validity bias, refers to the use of individual test items that are more suited to one group's language and cultural experiences.

Content-validity bias -

occurs when the content of a test is comparatively more difficult for one group of students than for others