Leapforce Rater Quality Exam Study Guide

Advertisements/Monetization (Ads)

(Ads) is content and/or links that are displayed for the purpose of monetizing (making money from) the page. -The presence or absence of Ads is NOT by itself a reason for a High or Low quality rating.

Characteristics of high quality pages

-High quality pages are satisfying and achieve their purpose well. High quality pages exist for almost any purpose, from giving information to making you laugh. -High level of Expertise, Authoritativeness, and Trustworthiness (E-A-T), including the EAT of the publisher and/or individual author for news articles and information pages on YMYL topics. A satisfying amount of high quality MC. Satisfying website information and/or information about who is responsible for the website or satisfying customer service information, if the page is primarily for shopping or includes financial transactions. Positive website reputation for a website that is responsible for the MC on the page.

Highest Quality Pages Charcteristics

-Highest pages are very satisfying pages which achieve their purpose very well. The distinction between High and Highest is based on the quality of MC as well as the level of E-A-T and reputation of the website. -Very high level of Expertise, highly Authoritative, and highly Trustworthy for the purpose of the page (E-A-T), including the EAT of the publisher and/or individual author for news articles and information pages on YMYL topics. -A satisfying amount of high quality MC. -Highly satisfying website information and/or information about who is responsible for the website or for stores and pages involving financial transactions, highly satisfying customer service reputation is very important -Very positive website reputation for a website that is responsible for the MC on the page.

Low Quality Pages Characteristics

-Low quality pages are unsatisfying or lacking in some element that prevents them from achieving their purpose well. These pages lack expertise or are not very trustworthy/authoritative for the purpose of the page. The author of the page or website does not have enough expertise for the topic of the page and/or the website is not trustworthy or authoritative for the topic. In other words, the page/website is lacking E-A-T. The quality of the MC is low. There is an unsatisfying amount of MC for the purpose of the page. MC is present, but difficult to use due to distracting/disruptive/misleading Ads, other content/features, etc. There is an unsatisfying amount of website information for the purpose of the website (no good reason for anonymity). The website has a negative reputation.

Medium Quality Pages

-Medium pages achieve their purpose and have neither high nor low expertise, authoritativeness, and trustworthiness. However, Medium pages lack the characteristics which would support a higher quality rating. -

Slightly Meets

A rating of Slightly Meets is assigned to results that are helpful and satisfying for some or few users. Slightly Meets results may serve a minor interpretation, be low quality, have stale or outdated information, have some minor inaccuracies, be too specific, too broad, too niche for the general user population considering the query, etc. to receive a higher rating. Important note: On mobile phones, the "cost of a click" is high. Many users decide which result to click or tap on based on the title of the web result on the search results page. (Note that these titles usually come from webpages.) A result with a very misleading or exaggerated title should be rated Slightly Meets or lower, due to the poor user experience that occurs when the landing page does not match the expectation of the user when clicking or tapping on the result.

Rating Queries with Multiple Interpretations and Intents

A very helpful result for a dominant interpretation should be rated Highly Meets, because it is very helpful for many or most users. Some queries with a dominant interpretation have a FullyM result. A very helpful result for a common interpretation may be Highly Meets or Moderately Meets, depending on how likely the interpretation is. A very helpful result for a very minor interpretation may be Slightly Meets or lower because few users may be interested in that interpretation. There are some interpretations which are so unlikely that results should be rated FailsM. We call these "no chance" interpretations.

Website E-A-T is important in the following situations:

All content on the website is produced by the same person or organization. An example is a medical website which is produced by a reputable physician group. The content of the website is produced by different authors or organizations, but the website has very active editorial standards. An example of this is a science journal with very high standards for publication. The website has an extremely positive reputation from experts in the topic of the website, i.e., the website is acknowledged to be one of the most expert, authoritative, or trustworthy sources on the topic.

Locale

All queries have a locale, which is the language and location for the task. Locales are represented by a two-letter country code.

Broad Queries

Broad queries: If the query is broad, then returning the same piece of content is not what the user is looking for, and hence those results should be flagged as dupes. Results may be considered dupes even if they have different minor content on the page (such as different ads, images, or related links).

How to find the homepage of a website

Examine the landing page of the URL in your PQ rating task. Find and click on the link labeled "home" or "main page." Having trouble finding it? Try using "Ctrl-F" ("command-F" on a Mac) to search the page for the text "home" or "main." You may also try clicking on the website logo, which is usually at the top of the page. -Even some High or Highest quality pages lack a way to navigate to the homepage.

Fully meets rating

Fully Meets is a special rating category, which can be used in the following situations: The query and user need must be specific, clear, and unambiguous. The result must be fully satisfying for mobile users, requiring minimal effort for users to immediately get or use what they are looking for. All or almost all users would be completely satisfied by the result—users issuing that query would not need additional results to fully satisfy the user intent. In other words, the Fully Meets rating should be reserved for results that are the "complete and perfect response or answer" so that no other results are necessary for all or almost all users to be fully satisfied.

Rating Queries with Both Website and Visit-in-Person Intent

Go to the website intent: in order to, for example, find out information, buy something online, make a reservation, schedule an appointment, interact with customer support, or fulfill some other need that can be satisfied online. Visit-in-person intent: user wants to visit the store, business, etc. in person

Lowest Quality Pages Characteristics

Harmful or malicious pages should be rated Lowest Here are some types of pages or websites that are highly untrustworthy, unreliable, unauthoritative, inaccurate or misleading—these pages should always receive the Lowest rating: -deceptive page purpose -deceptive page design -lowest quality main content -No website Info -Highly untrustworthy or unreliable content -hacked, defaced or spammed content -extremely negative reputation -promotion of hate or violence

Ratings for foreign language results

In most cases, pages you flag as Foreign Language should be rated FailsM, because they cannot be understood by most users in your locale and are therefore useless. If the search intent is for a foreign language, then assign a fullyM rating.

Ratings for Encyclopedia Pages

Lowest rating should be used if facts are NOT accurate. There are many encyclopedia-type websites. Some are highly-respected publications which are standard references, while some are websites with content created and edited by anonymous users, with no editorial oversight or fact checking. We may not always know the author of the specific encyclopedia article, and therefore must rely on website reputation research to determine the E-A-T of the article. High and Highest quality ratings should only be used for encyclopedias with very good reputations for accuracy and expertise, where the article itself is well-researched with appropriate references cited.

All of the content on a webpage can be classified as one of the following:

Main Content (MC), Supplementary Content (SC), or Advertisements/Monetization (Ads)

Main Content (MC)

Main Content is any part of the page that directly helps the page achieve its purpose. Webmasters directly control the MC of the page (except for usergenerated content). MC can be text, images, videos, page features (e.g., calculators, games), or it can be usergenerated content such as videos, reviews, articles, etc. which users have added or uploaded to the page. Note that tabs on some pages lead to even more information (e.g., customer reviews) and can sometimes be considered part of the MC of the page.

Device Actions:

Mobile phones and other devices can respond to voice commands to perform many actions, such as setting an alarm or opening an app. This is a specific type of query which we'll refer to as a Device Action query.

interstitial pages

On the web, interstitials are web pages displayed before or after an expected content page, often to display advertisements or confirm the user's age (prior to showing age-restricted material). Most interstitial advertisements are delivered by an ad server. -make it extremely difficult to use the MC. Pages which disrupt the use of the MC should be given a Low rating -Examples: Ads that actively float over the MC as you scroll down the page and are difficult to close. It can be very hard to use MC when it is actively covered by moving, difficult-to-close Ads An interstitial page which redirects the user away from the MC without offering a path back to the MC.

Upsetting-Offensive Flag

Please assign the Upsetting-Offensive flag to all web results that contain upsetting or offensive content from the perspective of users in your locale, even if the result satisfies the user intent. Upsetting-Offensive content typically includes the following: Content that promotes hate or violence against a group of people based on criteria including (but not limited to) race or ethnicity, religion, gender, nationality or citizenship, disability, age, sexual orientation, or veteran status. Content with racial slurs or extremely offensive terminology. Graphic violence, including animal cruelty or child abuse. Explicit how-to information about harmful activities (e.g., how-tos on human trafficking or violent assault). Other types of content which users in your locale would find extremely upsetting or offensive. In particular, to receive a Highly Meets rating, informational results about Upsetting-Offensive topics must: Be found on highly trustworthy, factually accurate, and credible sources, unless the query clearly indicates the user is seeking an alternative viewpoint. Address the specific topic of the query so that users can understand why it is upsetting or offensive and what the sensitivities involved are.

Misspelled and Mistyped queries

Some misspelled or mistyped queries are more difficult to interpret. Use your judgment and do query research. For obviously misspelled or mistyped queries, you should base your rating on user intent, not necessarily on exactly how the query has been spelled or typed by the user. For queries that are not obviously misspelled or mistyped, you should respect the query as written, and assume users are looking for results for the query as it is spelled.

Queries with an Explicit Location

Sometimes users tell search engines exactly what kinds of results they are looking for by adding the desired location in the query, regardless of their user location. We'll call this location inside the query the "explicit location." The explicit location makes queries much easier to understand and interpret.

Minor Interpretations

Sometimes you will find less common interpretations. These are interpretations that few users have in mind. We will call these minor interpretations.

Special Content Result Blocks

Special Content Result Blocks (SCRBs) appear in the search results page, along with Web Search Result Blocks. They are frequently, but not always, the first result on the search results page. Special Content Result Blocks are designed to show content directly to users on the search results page. From working calculators to playable videos to interactive weather information, these results help users immediately get information or content, and may not require additional "clicks" or page loads.

Specific Queries

Specific queries: For queries where the user is looking for a specific piece of content (such as queries looking for song lyrics, queries looking for a specific article, etc.), obtaining that piece of content from different sites could be helpful for users to verify the information, so they should not be rated as dupes.

Device Action query

Specific type of query where users ask their phone to perform an action. These are frequently spoken commands used to complete actions which would normally require interaction with the screen or the device's controls.

Supplementary Content (SC)

Supplementary Content contributes to a good user experience on the page, but does not directly help the page achieve its purpose. SC is created by webmasters and is an important part of the user experience. One common type of SC is navigation links which allow users to visit other parts of the website. Note that in some cases, content behind tabs may be considered part of the SC of the page.

Didnt Load Flag

The MC of the landing page is a web server or web application error message and there is no other content on the page (no navigation links, no home link, no SC, no Ads). See here for a Wikipedia page on different types of error messages. The landing page is completely blank (no MC, no SC, no Ads). Assign the Didn't Load flag based on the landing page, not the result block. ASSIGN A FAILSM RATING

Dominant Interpretation

The dominant interpretation. of a query is what most users mean when they type the query. Not all queries have a dominant interpretation. The dominant interpretation should be clear to you, especially after doing a little web research.

Do and Device Action Queries

The intent of a Do query is to accomplish a goal or engage in an activity on a phone. The goal or activity may be to download, to buy, to obtain, to be entertained by, or to interact with a website or app. Users want to Do something. A Device Action query usually has a clear action word and intent. The verb or action word is often at the beginning of the query, but a query might start with "OK Google" or "Google" or "Siri" or "I want to." Use your judgment.

Know Simple Query

The intent of a Know query is to find information on a topic. Users want to Know more about something. Know Simple queries are a special type of Know query. Know Simple queries seek a very specific answer, like a fact, diagram, etc. This answer has to be correct and complete, and can be displayed in a relatively small amount of space: the size of a mobile phone screen. As a rule of thumb, if most people would agree on a correct answer, and it would fit in 1-2 sentences or a short list of items, the query can be called a Know Simple query

Website Queries

The intent of a Website query is to locate a specific website or webpage which users have requested. This single webpage is called the target of the query.

Search Engine Results Page (SERP)

The page a search engine shows after a user enters a query in the search box. The SERP is made up of result blocks.

Device Action result:

The phone may respond to a Device Action query by performing an action, such as calling a phone number, etc.

User

The user is the person trying to accomplish something by typing or speaking into a mobile phone with a small screen (i.e., size of a smartphone, not a tablet).

Result Block

This is an individual "block" which appears on the user's phone in response to the query. The result block may display information in the block itself or contain links, or may do both. The Landing Page (LP) is the page you see after you click a link in the result block.

Query

This refers to the word(s) and/or number(s) that a user types or speaks into a mobile phone. In these guidelines, queries have square brackets around them. If a user says "navigate home," we display: [navigate home]. If a user types "iPhone" in the search box, we display: [iphone].

Common helpful purposes of a webpage

To share information about a topic. To share personal or social information. To share pictures, videos, or other forms of media. To express an opinion or point of view. To entertain. To sell products or services. To allow users to post questions for other users to answer. To allow users to share files or to download software.

Low quality Main Content (MC) charateristics

Unsatisfying amount of main content DIstracting or disrupting or misleading ads and supplementary content Negative reputation Lacking of E-A-T

Visit-In-Person Queries and user location

Users carry mobile phones with them throughout the day, for example, at work, to school, to restaurants, or running errands. One reason to carry a mobile phone is to have help with Visit-in-Person queries, such as finding coffee shops, gas stations, ATMs, restaurants, etc. Because mobile phones are often used for Visit-in-Person queries, make sure to consider Visit-in-Person intent as a possibility for mobile phone users. Some queries clearly "ask" for nearby information or nearby results (e.g., businesses, organizations, other nearby places). Some queries are not asking for nearby information or nearby results.

Result:

We will use the word result to refer to the result block and the landing page.

User Intent

When a user types or speaks a query, he or she is trying to accomplish something. We refer to this goal as the user intent.

URL

a character string which your web browser uses to "find" and display a webpage.

Webpage

a document which is part of the World Wide Web. Webpages can be viewed or "visited" with a web browser such as Firefox. In the 1990s, webpage content was mostly text and links. Today, webpage content includes many forms of media (such as images, videos, etc.) and functionality (such as online shopping features, email, calculator functionality, online games, etc.).

Website or Site

a group of World Wide Web pages usually containing hyperlinks to each other and made available online by an individual, company, educational institution, government, or organization. Popular websites include Facebook, Wikipedia, Yahoo, YouTube, etc.

Subpage of a website

any page on the site other than the homepage

Homepage of a website

the main page of the site. It is usually the first page that users see when the site loads.

Webmaster

the person who designs, develops, and/or maintains a website.

Your Money Your Life (YMYL) Pages

-Shopping or financial transaction pages: webpages which allow users to make purchases, transfer money, pay bills, etc. online (such as online stores and online banking pages). -Financial information pages: webpages which provide advice or information about investments, taxes, retirement planning, home purchase, paying for college, buying insurance, etc. -Medical information pages: webpages which provide advice or information about health, drugs, specific diseases or conditions, mental health, nutrition, etc. -Legal information pages: webpages which provide legal advice or information on topics such as divorce, child custody, creating a will, becoming a citizen, etc. -News articles or public/official information pages important for having an informed citizenry: webpages which are important for maintaining an informed citizenry, including information about local/state/national government processes, people, and laws; disaster response services; government programs and social services; news about important topics such as international events, business, politics, science, and technology. Please use your judgment and knowledge of your locale. -Other: there are many other topics which you may consider YMYL, such as child adoption, car safety information, etc. Please use your judgment. Keep in mind that not all news articles are necessarily considered YMYL.

Common Interpretation

A common interpretation of a query is what many or some users mean when they type a query. A query can have multiple common interpretations.

Fails to Meet

A rating of Fails to Meet should be assigned to results that are helpful and satisfying for no or very few users. Fails to Meet results are unrelated to the query, factually incorrect (please check for factual accuracy of answers), and/or all or almost all users would want to see additional results. These results completely fail to meet the user intent, such as a lack of attention to an aspect of the query (or user location) that is important for satisfying user intent. Fails to Meet may also be used for results which are extremely low quality, have very stale or outdated information, be nearly impossible to use on a mobile device, etc. Fails to Meet should be used for the following types of Lowest Page Quality results: Results which harm users (e.g., malicious download pages, "phishing" pages or websites, etc.). Pages or websites which misrepresent the website's owner or purpose, such as by impersonating a different site (e.g., copied logo or branding of an unaffiliated site, URL that mimics another site's name). Pages or websites which appear to be deliberate attempts to misinform or deceive users by presenting factually inaccurate content (e.g., fake product reviews, demonstrably inaccurate news, etc.). Pages or websites with factually inaccurate content which can cause harm to users regardless of their purpose or intent (e.g., inaccurate medical information; inaccurate information about how to vote, obtain a visa or green card, pay government taxes, etc.).

Highly Meets Rating

A rating of Highly Meets is assigned to results that meet the needs of many or most users. Highly Meets results are highly satisfying and a good "fit" for the query. In addition, they often have some or all of the following characteristics: high quality, authoritative, entertaining, and/or recent (e.g., breaking news on a topic). In order to receive a Highly Meets rating, information pages such as encyclopedia articles and news articles must be accurate and highly credible. Highly Meets medical and scientific information pages must represent well-established scientific/medical consensus unless the user is clearly seeking an alternative viewpoint. For all information pages, you must check for accuracy and confirm that the information is supported by expert consensus where such consensus exists. A query can have many Highly Meets results. Have high standards for the Highly Meets rating. This is especially important for queries with many on-topic results.

Moderately Meets

A rating of Moderately Meets is assigned to results that are helpful and satisfying for many users or very satisfying for some users. Moderately Meets results have fewer valuable attributes than Highly Meets results. Moderately Meets results should still "fit" the query, but they might be less comprehensive, less up-to-date, come from a less authoritative source, work less well on a mobile device, etc. Moderately Meets results are not low quality, out-of-date, or inaccurate. Moderately Meets results are generally average to good.