Search Quality Rater

LOWEST

4. Untrustworthy Webpages 4.4 Deliberately Obstructed or Obscured MC Pages are untrustworthy if the MC is deliberately obstructed or obscured due to Ads, SC, interstitial pages, download links or other content that is beneficial to the website owner but not necessarily the website visitor. Attempts to manipulate or coerce users away from the MC is evidence of untrustworthiness. Pages with deliberately obstructed or obscured MC should be rated Lowest.

Side-by-Side Rating: Categories & Examples 4. CLEAR INTENT

4.1. Know simple 4.2 Website 4.3 Device Action

Rating LOWEST

4.2 Lowest E-E-A-T and Lowest Reputation of the Website or Content Creator Explanation The level of expertise of the author of this content is not clearly communicated. Providing this background information is particularly important for medical, financial, or other YMYL topics for which expertise is needed.

Rating LOWEST

4.4 Deliberately Obstructed or Obscured MC Explanation The main content on the landing page is inaccesible due to the Ad and it's not possible to close it.

Rating LOWEST

4.5 Suspected Malicious Behavior Explanation This is a shopping checkout page, but it asks for the user's government ID number, driver's license number, ATM pin number, bank account information, etc.

Rating FailsM

5. Lowest quality pages Results that are extremely low quality should be rated Fails to Meet. Some common low quality results are: • Pages that harm users. . Pages that misrepresent the website's owner or purpose, for example imitating a different website. • Pages that appear to be deliberate attempts to misinform or deceive users by presenting factually inaccurate content.

Rating LOWEST

5.1 Cannot Determine a Purpose, No MC, Little MC, or Lowest Quality MC Explanation It is not possible to understand the purpose of the page truly because the main content is gibberish. Also, the word "imodium" is used repeatedly in hopes that the page will show up for the search "imodium". Attempting to spam search engines and trick people into clicking on links is highly untrustworthy behavior. This page has no helpful MC and no helpful purpose.

ModeratelyM

A very helpful result for a common interpretation or common intent may be Highly Meets or Moderately Meets depending on how likely the interpretation is, in the context of the query.

Copied MC with no Added Value

Another way to create an abundance of content with little effort or originality is to copy content from another source. The word "copied" refers to the practice of "scraping" content or copying content from other non- affiliated websites without adding any original content or value to users. Any type of content may be copied: text, images, video, etc. Images may be slightly cropped or edited to avoid detection. People may alter videos as well, or even make a video of a screen playing video! Copied content with no value added is Lowest quality. Sometimes content is "embedded" or included on the page from another source. There is nothing wrong with embedding content, but the Lowest rating applies if there is no significant original content or value added on the page. Likewise, social media posts may be reposted by an unrelated account. There is nothing wrong with reposting content, but the Lowest rating applies if there is no value added in the form of additional content, context, or discussion.

LOWEST

Category 4.2 Lowest E-E-A-T and Lowest Reputation of the Website or Content Creator Explanation There are many claims of fraud and financial problems about this organization on reputable news sites and charity watchdog sites.

LOWEST

Category 5.1 Cannot Determine a Purpose, No MC, Little MC, or Lowest Quality MC Rating LOWEST Explanation It is not possible to understand the purpose of the page truly because the main content is gibberish. This page has no helpful MC and no helpful purpose.

Query [angela merkel height]

Category Know simple queries Explanation The user is trying to find Angela Merkel's height. User query is under the category of know-simple query and the user is looking for quite specific information, so this query can have a FullyM result. The SCRB shows the correct answer directly, meeting the user intent perfectly. Therefore, this result should be rated as Fully Meets.

2. Device action queries

Device Action queries are a special kind of Do query. Users are asking their phone to do something for them. Users giving Device Action queries may be using phones in the hands-free mode, for example, while in a car. It's very important for mobile phones to accommodate Device Action queries, and we have a high standard for rating these results. A Device Action query usually has a clear action word and intent. The verb or action word is often at the beginning of the query, but a query might start with "OK Google" or "Google" or "Siri" or "I want to." Use your judgment.

LOW 5. Unsatisfying Amount of Information about the Website or Content Creator

Expect to find some form of website information for many or most websites. We expect clear information about who (e.g., what individual, company, business, foundation, etc.) created the MC, unless there is good reason for anonymity. A long-standing Internet alias or username can identify the content creator. For personal websites or forum discussions on non-YMYL topics, an email address or social media link alone may be sufficient. Many people are reasonably hesitant to share personally identifiable information online! Stores and websites that process financial transactions require a high level of user trust. If a store or financial transaction website has just an email address and physical address, it may be difficult to get help if there are issues with the transaction. Likewise, many other types of websites on clear YMYL topics also require a high level of user trust. Important: Pages that offer payment functionality or process other types of financial transactions should receive a Low rating if there is an unsatisfying amount of customer service information or contact information. Pages on YMYL topics and other pages that require an important level of user trust should receive a Low rating if there is an unsatisfying amount of information about who is responsible for the website or who created the content. For forums, social media pages, and other websites where people commonly exchange opinions, a username is an acceptable identifier.

2. Low Quality Main Content Rating LOW

Explanation Advice on stock investment could significantly impact a person's financial decisions and security. This is a YMYL topic. The MC is superficial and includes mostly commonly known information for this topic however there is little to no information about the content creator and there is no evidence that the author has financial expertise. Because this article gives advice on a YMYL topic, lacking expertise is a reason for a Low rating.

rated as Highly Meets+.

Explanation Different users may be looking for different things and information with (U2) query. There is no dominant interpreation for this broad query. Therefore, any result related to this query cannot be rated as Fully Meets. However, the result shows the official website of U2, which has unique authority on the topic. Therefore, this result may be rated as Highly Meets+.

Rating LOWEST

Explanation The page repeats the error code "P2570" beyond what is helpful for users- a technique known as irrelevant keywords. The first sentence is incomplete and does not explain the code. This should be the most important content on the page! Looking at other pages on the website, it is clear each page is created automatically by filling in a template from a list of error code information. There appears to be little to no manual editing or checking to make sure the error code content is correctly pulled in, as evident by the missing reason on this page. As with many spammy websites, there is no information about who is responsible for this website and no way to report problems with the page.

Category 5. Unsatisfying Amount of Information about the Website or Content Creator Rating LOW

Explanation There is no evidence that the author has financial expertise. Because this is a YMYL financial article, lacking expertise is a reason for a Low rating.

Category 2. Mixed, but with some redeeming qualities Rating MEDIUM

Explanation This forum page is about dance topic. On this particular page, participants have everyday experience on the topic and make recommendations based on their own experiences. This page is "mixed" because there is some distracting content and layout that make it hard to read the MC. However, this is not a beauty contest! Even though the page may be cluttered, there is some valuable everyday expertise and helpful MC, making Medium a good rating for this page.

Rating HIGHEST

Explanation This is a series of investigative stories about the conflict in Yemen on a newspaper website that has won a variety of awards including the Pulitzer Prize. The series has a comprehensive amount of very high-quality MC that is unique and original, including in-depth investigative reporting by the author with extensive knowledge and experience in investigative journalism.

Rating LOWEST

Explanation This is an example of a spammed forum page, where the goal is to try and get users to watch a movie online, which has nothing to do with the purpose of the site. For a forum page, the user comments are the MC, and much of the content is not helpful for users.

Rating HIGHEST

Explanation This is an seasonal flu reference page on a trustworthy and authoritative medical website. This website has a reputation of being one of the best web resources for medical information of this type.

2. Positive Reputation Rating HIGH

Explanation This is the homepage of a newspaper that has won several Pulitzer Prizes and Overseas Press Club awards. This newspaper has a positive reputation for its objective reporting.

Category 1. Nothing wrong, but nothing special Rating MEDIUM

Explanation This page is on a website dedicated to entertainment news. This page is OK for its purpose, but it doesn't display characteristics associated with a High rating.

5. Comprehensive information

Factually accurate, entertaining, representative, recent, in-depth, clearly written, and comprehensive high-quality pages can receive a Highly Meets rating, depending on how likely the interpretation is in the context of the query.

1. SCRB VS SCRB 1.1. Single entity seeking query

For a single entity seeking query, if the difference between the two sides depends only on the SCRBs at L1 and R1: The side that contains a SCRB providing information about the entity requested by the user should generally be rated better than the side that contains a SCRB returning a list with additional unhelpful entities.

7. BROAD INFORMATION QUERY

For broad queries, different users may be looking for different things. For this reason, different users may prefer different types of results at the top - no exact ordering would be perfect for all users. In such cases, the result set should offer different results.

3. Know simple queries

For know simple queries, if the landing page of a result block clearly displays the requested answer, it should be rated as Highly Meets.

3. DIVERSITY 3.3. Diversity of result types

For some type of queries such as video seeking queries, there can only be one type of helpful result. If the query is looking for an image of a celebrity, the result set should show the images of a relevant celebrity. Also, for example, if the user wants to find a specific fact or piece of information, special content result blocks may be appropriate and helpful.

2. Authoritative results for broad queries

Highly Meets is the most ideal and appropriate rating for authoritative results in the case that when the user intent is too broad to have a Fully Meets result as different users may be looking for different things. Also, there shouldn't be any other uniquely authoritative result in this case. If there is any unique authority for the topic (e.g. (U2) query), please use Highly Meets+ for the relevant result.

4. CLEAR INTENT 4.3. Device action

If the user query is looking for a device action, the result set should offer the relevant device action.

4. CLEAR INTENT 4.1. Know simple

If the user query is looking for a specific fact or information, the result set should show the relevant answer prominently.

3. DIVERSITY 3.1. Multiple intents

In case the user query has more than one intent or interpretation, the results on both sides should reflect this.

Rating LOWEST

LOWEST 5. Spammy Webpages 5.3 Auto-generated MC Creating an abundance of content with little effort or originality with no editing or manual curation is often the defining attribute of spammy websites. One way to do this is to use "auto-generated" content. It is possible to create many pages or even entire websites by designing a basic template from which hundreds or thousands of pages are created, sometimes using content from freely available sources (such as an RSS feed or API). These pages are created with truly little effort and have little to no editing or manual curation. There is little value when pages repackage freely available content with little attention to no attention to the experience of people visiting the page. Pages and websites made up of auto-generated content with no editing or manual curation, and no original content or value added for users, should be rated Lowest.

Type of Page and Purpose

News website homepage: the purpose is to inform users about recent or important events. MC - News Homepage

FailsM

No chance interpretation A No Chance Interpretation is an interpretation so unlikely that almost no user would have this in mind. Results for no chance interpretations feel off-topic even if they match the query words. For example, almost no one wants a result about an overheated pet for the English US query [hot dog]. If a result targets an unlikely interpretation of the user intent, it should be rated as FailsM.

Rating ModeratelyM+

Query [Seattle washington] Category Average to Good Quality Explanation: The user wants to find information, news, maps, etc. related to Seattle, Washington. This is a broad query for a city. Different users may be looking for different things.

Rating ModeratelyM

Query [where is virginia in the us] Category Average to Good Quality Rating ModeratelyM Explanation The user is trying to find where the state of Virginia is located in the U.S. The description in this result block may be helpful for users who are already familiar with this area in the United States. Many or most users would probably want to see a map or a clearer description. Therefore, Moderately Meets to Moderately Meets+ is an appropriate rating for this result.

Rating SlightlyM

Query [wikipedia] Category Reasonable Minor Interpretation Explanation This result from Similarweb website provides internet traffic statistical information about the Wikipedia website. By [Wikipedia] query, most users would want to navigate to the Wikipedia website. This result still fits the query, however it serves a minor interpretation for the query. Therefore, it should be rated SlightlyM.

Rating FailsM

Query [aapleprice] Category Very stale or outdated information Rating FailsM Explanation The user wants to find the current price of Apple stocks. The result provides information regarding the apple stocks in 2017. Since the result is very stale compared to user intent, it should be rated Fails to Meet.

Rating ModeratelyM

Query [amazon] Category Helpful result for any Dominant, /user intent Common or Reasonable Minor query interpretation/user intent Explanation The dominant interpretation of this query is to go to the Amazon website. Some users might also be looking for other things such as information about the company, specific subpages, recent news etc. The SCRB offers several general and helpful elements about the Amazon corporate such as official website URL, Wikipedia link, customer service number, etc. While the majority of the users would like to go to the official Amazon website, this SCRB can be helpful for some users. Therefore, it should be rated as ModeratelyM.

Rating ModeratelyM

Query [apple] Category Helpful result for any Dominant, Common or Reasonable Minor query interpretation/user intent Explanation By issuing this query, most of the users would like to see Apple website or information on Apple products while some users may look for information about the fruit apple. While the dominant interpretation of the query is Apple Inc, this SCRB shows information about apple as a fruit, including general information, images, nutritional values, etc. Since the result satisfies the common interpretation of the query, it should be rated as Moderately Meets.

FailsM

Query [batman] User Location San Francisco, USA Category No chance interpretation Rating X Go ge Batman/Batman Province Turkey FailsM Explanation The user is trying to find information about the fictional superhero. Result shows a city with the same name in Turkey on the map. It is extremely unlikely that this query is looking for information on a city in Turkey called Batman since the user location is San Francisco, USA. Therefore, it should be rated Fails to Meet.

Rating FailsM

Query [betty white] Category Lowest quality pages Explanation X FullyM At the time this example was written, Betty White was alive. This profile page on a social media website leads users to believe she is dead. Comments on the page indicate that some users have realized that it is inaccurate. If the page were truly created in error, these comments should have led the creator to realize the mistake and delete the page. While it is not clear why this type of page was created in the first place, it appears to be a deliberate intent to misinform users, which is a harmful purpose. rate content.

FailsM

Query [call mom] Category Unrelated Results Rating FailsM Explanation The user clearly wants to call his/her mom, which is a device action query. The device action result attempts to set an alarm. In this case, this result is unrelated to the user query and user intent. The user won't be satisfied with this result at all. Therefore, it should be rated Fails to Meet.

Side by side ratings

Query [snakes] Category Broad Information Query Rating LS is much better Explanation The user query is [snakes] and users could be looking for several different things, including information, images, videos, etc. X Images and videos are helpful results for this query. Even though all results on both sides are Highly Meets, the left side is much better than the right side because it includes helpful image and video snake results among the result set. Also, for a broad query like this one, diversity is important both in the result set and type of content, which can be found on the left side.

HIGHEST 2. Very Positive Reputation

Reputation research is important when giving Highest ratings. For YMYL topics, positive reputation is often based on recommendations from known experts or professional societies appropriate to the page's topic. Prestigious awards may be evidence as well, depending on the topic and type of content. For non-YMYL topics, reputation information may be less formal. Popularity, user engagement, and user reviews can be considered evidence of reputation for non-YMYL websites: websites and content creators can be considered to have an incredibly positive reputation if they are seen as one of the best sources available online for a topic or type of content. Remember: Many smaller websites and ordinary people have little reputation information. A page can still receive a Highest rating without reputation information.

Side-by-Side Rating: Categories & Examples

Side-by-Side Rating: Categories & Examples 1. SCRB VS SCRB 2. MINOR DIFFERENCES 3. DIVERSITY 4. CLEAR INTENT 5. ORDER OF THE RESULTS 6. NO RESULT ON ONE SIDE 7. BROAD INFORMATION QUERY

5. ORDER OF THE RESULTS

The order of the results is extremely important, especially for mobile users with smaller devices. Most users look at the top results, so the best and most helpful results should appear at the top of the page. Results in lower positions get less attention from users, even if they are helpful. Useless results may discourage users from looking further down the page. Therefore, results should appear in order of helpfulness to users.

MEDIUM 2. Mixed, but with some redeeming qualities о

The page or website has some signs of High quality (E-E-A-T, quality of the MC, positive reputation), but also has one sign or mild signs of Low quality.

Needs Met Rating: Categories & Examples Rating FailsM

The user is trying to find out calorific value of an avocado. However, the result provides information for apples. Since the result misses an important aspect of the user intent, it should be rated Fails to Meet.

LOWEST RATING 1. harmful to Self or Other Individuals

This landing page provides step- by-step information on how to commit suicide. Such kind of pages that encourage harm should be rated as Lowest.

Low Quality Main Content Rating LOW Explanation

This page and website have many of characteristics of Low quality pages. The MC is low quality and lacks important information. For example, it gives no indication of how to make a crust and doesn't list premade crust as an ingredient. It would be very difficult to actually make an apple pie using this recipe. This page is lacking the kind of helpful content we expect in recipe pages, such as user reviews or comments. Without such information, it's hard to tell if the recipe is any good. The small pink text at the top is not helpful for users

SEARCH QUALITY RATING QUALIFICATION PROGRAM Side-by-Side Rating: Categories & Examples

This tutorial provides information about the characteristics of different factors that play an important role in Side-by-Side rating. You will see examples and explanations of these factors and considerations that need to be made that will help you determine which side is better and how much better or if the sides are about the same. You will experience common Side-by-Side Rating scenarios and examples. Please check all examples to fully understand the differences between ratings.

2. MINOR DIFFERENCES

While rating a SXS task, sometimes you may encounter cases where the difference between the two sides is minimal. In such cases, before quickly rating the task as "About the same", you need to carefully examine the difference between the two sides to see if there is any additional benefit to the user.

"user" refers to a person trying to find information or accomplish a task on the Internet.

user

Side-by-Side Rating: Categories & Examples 2. MINOR DIFFERENCES

.

Side-by-Side Rating: Categories & Examples 5. ORDER OF THE RESULTS

.

Side-by-Side Rating: Categories & Examples 6. NO RESULT ON ONE SIDE

.

ModeratelyM

1. Helpful result for any Dominant, Common or reasonable Minor Query interpretation/user intent 2. Average to good quality

HighlyM

1. Highly satisying SCRB's for broad queries 2. Authoritative results for broad queries 3. Know simple queries 4. Very helpful results for any Dominant, Common or Reasonable Minor query Interpretation/user intent 5. Comprehensive information

FailsM Unrelated results

1. Know simple queries 2. Device action queries 3. Website-Webpage queries

SlightlyM

1. Minor Interpretation 2. Stale or outdated information 3. Low quality pages 4. Results that do not address an important part of the query

SlightlyM

1. Minor Interpretation Results that satisfy a minor interpretation of a user query should be rated as SlightlyM. A very helpful result for a very minor interpretation may be Slightly Meets or lower because few users may be interested in that interpretation. Minor Interpretation: Sometimes you will find less common interpretations. These are interpretations that fewer users have in mind. We will call these minor interpretations. • Reasonable minor interpretations: Reasonable minor interpretations may help fewer users, but they still are helpful to have in a set of results because different users may want different things. Click to see example • Unlikely minor interpretations: An unlikely minor interpretation is an interpretation that is in theory possible but is very unlikely - very few users are likely to have this in mind.

FailsM

1. Unrelated results Results unrelated to the user query and the user intent should be rated as FailsM. Some rating scenarios can be tricky or deceptive. Although the result is useless to the user, it may seem that there is a connection between the user query and the result. Put yourself in the user's shoes and carefully consider whether the result corresponding to the relevant query will actually satisfy the user in any way. Make sure that you fully understand the user intent

FailsM

1. Unrelated results 2. Lack of attention 3. No chance interpretation 4. Very stale or outdated information 5. Lowest quality pages 6. Very poor and upsetting results

HighlyM+

1. Visit in-person & website intent 2. Uniquely authoritative results 3. Visit-in-person SCRBs for list seeking queries

rated as Highly Meets

1. Visit-in-person & Website intent Some queries have two possible strong intents: 1. Go to the website intent: in order to, for example, find out information, buy something online, make a reservation, schedule an appointment, interact with customer support, or fulfill some other need that can be satisfied online. 2. Visit-in-person intent: user wants to visit the store, business, etc. in person. We know the user intent is to accomplish one or the other, but it is unclear which one the user wants. For these queries, result blocks that only satisfy one intent should be +

Side-by-Side Rating: Categories & Examples 1. SCRB VS SCRB

1.1. Single entity seeking query 1.2. List seeking query

LOWEST

3. Harmfully Misleading Information 3.3 Harmful unsubstantiated theories/claims Pages with content that is not grounded in any reasonable facts or evidence, especially those that could erode confidence in public institutions should be rated Lowest. This includes unsubstantiated theories that have been thoroughly debunked or are too outlandish to be given credence. Examples: ⚫ Claims that the 9-11 attacks were planned by the U.S. government • Claims that world leaders are lizard people

SlightlyM

3. Low Quality Fri Aug 23 5:26 PM If a result fits the query, but the landing page features low quality characteristics, the relevant result should be rated as SlightlyM. Please note: A page can have such low Page Quality that it is unhelpful or even harmful (malicious, misleading, deceptive, hateful, etc.), such as gibberish pages or scams. These types of extremely low quality pages should be rated FailsM because the Page Quality issues interfere with helpfulness.

3. Visit-in-person SCRBS for list seeking queries

3. Visit-in-person SCRBS for list seeking queries Some queries may be clearly looking for a list of entities such as (parks with swings), (hotels in California), etc. Since these types of queries are somewhat broad, any result related to such queries cannot get Fully Meets rating. However, a dedicated Visit-in-Person SCRB displaying a very complete list can be rated as Highly Meets+ for such queries.

Side-by-Side Rating: Categories & Examples 3. DIVERSITY

3.1 Multiple Intents 3.2 Diversity of sources 3.3 Diversity of results types

LOWEST RATING

3.2 Harmful claims that contradict well-established expert consensus Rating LOWEST Explanation Claims about cancer cures could significantly impact people's decisions to seek medical treatment. This page is on a YMYL topic. The claims made about the healing power of crystals contradict well-established expert scientific and medical consensus.

LOWEST

4. Untrustworthy Webpages 4.1 Inadequate Information about the Website or Content Creator for the Purpose of the Page 4.2 Lowest E-E-A-T and Lowest Reputation of the Website or Content Creator 4.3 Deceptive Page Purpose and Deceptive MC Design 4.4 Deliberately Obstructed or Obscured MC 4.5 Suspected Malicious Behavior

LOWEST

4. Untrustworthy Webpages 4.2 Lowest E-E-A-T and Lowest Reputation of the Website or Content Creator If the E-E-A-T of a page is low enough, people cannot or should not use the MC of the page. If a page on YMYL topics is highly inexpert, it should be considered Untrustworthy and rated Lowest. Use the Lowest rating if the website and content creator have an extremely negative reputation, to the extent that many people would consider the webpage or website untrustworthy.

Side-by-Side Rating: Categories & Examples

Query [angelina jolie] Category Minor Differences Rating LS is slightly better - About the same - RS is slightly better Explanation X For very broad know queries like this one, different users may be looking for different things: biographical information, news, images, etc. For this reason, different users may prefer different types of results at the top-no exact ordering would be perfect for all users. All of the results on both sides are highly satisfying. The only difference between the two sides is in the order of the interactive SCRBS in the top two positions. The SCRB in L1/R2 shows a selection of recent news stories focused on Angelina Jolie. The SCRB in L2/R1 shows a variety of helpful content about her directly on the page with an option to see more. While no single particular result will be the most helpful for all users, most users could easily find a helpful result for their needs in either set of results. Therefore, since either side would offer a highly satisfying user experience and the overall differences in user experience are small, the most appropriate rating is about the same. However, there is some room for judgement in situations like this. If the recent news stories shown are particularly interesting, you might choose slightly better for the left side where the news appears at the top. Alternatively, if the recent news seems less interesting, you might decide the right side is slightly better. Yet, please note that better or much better for either side would be too extreme for this particular set of results.

Side-by-Side Rating

Query [barack obama birth date] Category No Result on One Side Rating About the same - RS is slightly better Explanation The query is a know-simple and the user wants to know the birth date of Barack Obama. The left side shows an unhelpful SCRB that provides information about the birth date of Brad Pitt. The right side did not generate any results. When both sides are unhelpful, you should usually use the about the same rating. However, if the results on one side are clearly FailsM and time wasting, the side with no result may also be rated as slightly better.

Rating LS is much better

Query [chicago weather] Category Diversity of Result Types Rating LS is much better Explanation X The top result on the left side is an interactive block that displays the complete and correct answer (i.e., current weather and forecast for Chicago) prominently in the result block, in a way that is easy for users on mobile devices to read. The top result on the right side has similar details on Chicago weather though users would need to open the LP to see the information. Both sides would satisfy users, but the L1 result summarizes the information in a way that's easy to digest immediately, and saves the user from a cost-of-a-click. The interactive block also adds diversity to the result set, making the left side much better than the right side.

Rating HighlyM

Query [heart attack] Category Authoritative results for broad queries Explanation The users is looking for information about heart attacks. The user query and intent is not specific. Also, since there is no unique authority regarding the topic, any result related to this query can get Highly Meets rating at most. This result is from a very authoritative website, which is www.mayoclinic.org. Also, the landing page provides a lot of helpful information about heart attacks. Therefore, this result should be rated as Highly Meets since it would be very helpful for many or most users.

Rating Highly Meets.

Query [high definition tv] Category Comprehensive information The user intent is somewhat broad. Since the landing page provides comprehensive information about different types of high definition TVs, it covers different aspects of the query. Also, the landing page provides reviews about many specific models. Therefore, it should be rated as Highly Meets.

Rating FailsM

Query [how to quit smoking] Category Very poor and upsetting user experiences Explanation The user intent is find information on ways to quit smoking. This LP has gibberish text. Read this sentence: "How do you make a cigarette symbol on the keyboard? In." The quality of the landing page is so low that the page Fails to Meet the user intent.

Rating ModeratelyM

Query [kristen wiig] Category Average to Good Quality Explanation The user is looking for information about the actress, comedian, writer, and producer. The result is Kristen Wiig's official website, but the only content is linked to her talent and PR agencies, along with a note that says she is not on any social networking sites. While this information is helpful, many users would want to see more. Therefore, it should be rated as ModeratelyM.

Low Quality Rating SlightlyM

Query [lack of sex and problems with my marriage] Category Low Quality Rating SlightlyM Explanation The user is trying to find help for marital issues. The quality of writing in this article, which was created by a person without expertise in marriage or relationship counseling, is poor. Even though the article is about the query, the page is low quality and untrustworthy. It would be helpful for few users.

Low Quality Rating SlightlyM

Query [marion cotillard net worth] Category Low Quality Rating SlightlyM Explanation The user is trying to find the queried actress' net worth. Although the result fits the query, it features several low quality characteristics such as poorly written content, grammatical mistakes and poor user experience. Therefore, it should be rated SlightlyM.

Rating LS is much better

Query [open instagram] Category Device Action Rating LS is much better Explanation Users are asking their device to open Instagram app. L1 shows the phone in the process of opening the Instagram app and it's Fully Meets according to the user intent. The remaining results on both sides are HM. Therefore, LS is much better than the RS.

Rating HighlyM+

Query [playstation] Category Uniquely authoritative results Different users may be looking for different things and information with (playstation) query. There is no dominant interpreation for this broad query. Therefore, any result related to this query cannot be rated as Fully Meets. However, the result shows the official website of Playstation, which has unique authority on the topic. Therefore, this result may be rated as Highly Meets+.

Rating FullyM

Query [set alarm for 5 o'clock am] Category Device action queries Explanation This is a Device Action query. The user wants to set the device's alarm to go off at 5:00 AM. Since the result completely fulfills the user's request, it should be rated as Fully Meets.

LOW 2. Low Quality Main Content MC is Low quality if

it is created without adequate effort, originality, talent, or skill necessary to achieve the purpose of the page in a satisfying way. Mild inaccuracies on informational pages are evidence of Low quality. However, if the page has harmful inaccuracies or so many inaccuracies that it feels untrustworthy, the Lowest rating is justified. There are many possible ways for MC to be Low quality, and Low effort is important to recognize in all forms. Some examples: • • Producing content quickly with little attention to important aspects such as clarity or organization. Saving time by summarizing or paraphrasing from other readily available sources. Filling a report with large pictures or other distracting content rather than clear, helpful information on the topic. Using images from other sources (magazines, stock image websites, etc.) to avoid having to create pictures or diagrams to support the material in the report. Using commonly known facts that require little research, for example: "Argentina is a country. People live in Argentina. Argentina has borders." Using "filler" content, for example: "Pandas eat bamboo. Pandas eat a lot of bamboo. Bamboo is the best food for a Panda bear."