IMS 414

Exploratory vs Reporting metrics

exploratory: speculative, trying to find unexpected, unknown or interesting insights that can give you the upper hand, focus on interesting / unexpected and on trying new things reporting: predictable, keeping you abreast of normal, managerial operations, day-to-day operations

followers/friends/likers time on site, or pages/visit emails collected number of downloads

followers/friends/likers: counting followers rather than actions is a bad idea. Count actions instead. find out how many followers will do an action valuable to you. time on site, or pages/visit: poor substitute for engagement / activity unless your business is tied to this behavior. If customers spend a lot of time on your support or complaints pages, that could be a bad thing. emails collected: A big mailing list of people excited about your new startup is nice, but until you know how many will open your mails (and act on what's inside them) this isn't useful. number of downloads: While it sometimes affects your place in app stores and rankings, downloads alone don't lead to lifetime value. Measure activations / active accounts

hits page views visits unique visitors

hits: metric from the early, foolish days of the Web. If you have a site with many objects on it, this will be a big number. Count people instead. page views: counts the number of times someone requests a page. Unless your business model depends on page views (i.e. display advertising inventory) you should count people instead. visits: Is this one person visiting a hundred times, or are a hundred people visiting once? Fail. unique visitors: the only thing this shows you is how many people saw your home page. It tells you nothing about what they did, why they stuck around, or if they left.

____ metrics are leading indicators and ____ metrics are lagging indicators. By definition, it can take time for the output to reflect positive or negative changes in the inputs.

input, output

Analytics is ?

utilizing data to make business decisions.

vanity vs actionable metrics

vanity: makes you (& your investors) feel good, but doesnt change how youll act actionable: change your behaviors by helping you pick a direction / course of action

Tableau is a data [_____] platform.

visualization

what is share of voice?

your brand advertising / total market advertising = share of voice Share of voice is basically determining how much impact and spread your advertising and social media voice has in the market compared to everyone else.

to find a real need we use ____ to find a right solution we use _____ to find a good product we use _____ to find a sustainable biz we use _____ to find a healthy market we use _____ to find a successful exit we use _____ in between which stages do we use growth rate?

to find a real need we use empathy to find a right solution we use stickiness to find a good product we use virality to find a sustainable biz we use virality/rev to find a healthy market we use revenue to find a successful exit we use scale between solution to successful exit

5 media metrics

1) Audience and churn: How many people visit the site, and do they keep coming back? 2) Ad inventory: How many impressions do we have that we can monetize? 3) Ad rates: What might we be paid for those impressions, based on the content we cover the people who visit us? 4) Click-through rates: How many of those impressions actually turn into money? 5) The content/advertising balance: Are we properly balancing money, ad inventory, and content to maximize overall performance? - ads is where fb / twitter/ google get their money - any place you can put an ad, is inventory - click through rates are paid based on how many clicks there are

7 e-commerce metrics

1) Conversion rate (the number of visitors who buy* something.) 2) Purchases/year (the number of purchases made by each customer per year) 3) Average shopping cart size (the amount of money spent in a purchase.) 4) Abandonment (the percentage of people that begin to make a purchase, and then don't.) 5) Cost of customer acquisition (the money spent to get someone to buy something.) 6) Top keywords driving traffic to the site. 7) Top search terms that lead to revenue; top search terms that don't have any results. *buy is the best conversion but in some cases it may be conversion to something else you want them to do

3 things to ask yourself to find useful metrics and avoid false metrics

1) Will this metric predict business outcome we are trying to achieve / is the metric is aligned with the goals? 2) Do colleagues understand the metric? Does it resonate with their role and responsibilites? 3) Does the metric enable action?

dave mcclure's pirate metrics (customer lifestyle / conversion behavior)

1) acquisition (SEO, PR, widgets) - how do your users become aware of you? users come to the site from various channels 2) activation (product features) - do drive by visitors subscribe, use, etc? users enjoy 1st visit "happy" user experience 3) retention (emails and alerts, blog content) - does a one time user become engaged? users come back, visit site multiple times 4) revenue (ads, lead gen subscriptions) - do you make money from user activity? users like product enough to refer others 5) referral (emails & widgets, campaigns / contests) - do users promote your product? users conduct some monetization behavior dave founded 500 Startups and coined the term "Pirate Metrics" or "AARRR" which helps startups think through the five most important metrics of their business. Each one of these five metrics is critical for startups, and for businesses in general. Without tracking these metrics, you won't have a sustainable, scalable business nor will you know if you are succeeding. 5 steps users must progress to get value from them

4 social media content performance measures

1) converSATION rate 2) amplification rate 3) applause rate 4) economic value

8 examples of vanity metrics

1) hits 2) page views 3) visits 4) unique visitors 5) follows/friends/likes 6) time on site or pages/visit 7) emails collected 8) number of downloads Generally these are considered vanity metrics but that depends on the context of your business and sometimes what you have is better than nothing like 'uniques' or 'likes' and that's a starting point.

if you want to choose the right metric keep 5 things in mind

1) qual vs quan metrics qual metrics are unstructured, anecdotal, revealing and hard to aggregate quan metrics involve numbers and stats, and provide hard numbers but less insight 2) vanity vs actionable metrics vanity metrics make u feel good but dont change the way you act. actionable metrics change ur behavior by helping u pick a course of action 3) exploratory vs reporting metrics exploratory are speculative and try to find unknown insights to give u the upper hand, while reporting metrics keep u abreast of normal day to day operations 4) leading vs lagging metrics leading metrics give u a predictive understanding of future, lagging metrics explain the past. leading are better bc you still have time to act on them 5) correlated vs causal metrics if 2 metrics change together they're correlated. but if one metrics causes another to change, theyre causal. if you find a causal relationship b/w something you want (rev) and something you can control (which ad u show) then you can change the future

4 types of add revenue (4 ways visitors generate value)

1) subscriptions (subscriptions rate, RSS/follow/email, click through rate) 2) on-site engagement (engagement metrics: time since visits, time per visit, visits per day, pages per visit, time on page) 3) ad revenue (display ads, affiliate links, pay per click) 4) sharing (on-site tools, off-site copy/paste) other: sponsorship - this is more direct

how to choose a OMTM

1. understand the purpose of Product/Page 2. get as specific as possible  3. use one you can track  4. understand the Business Value (conversions, customer lifetime value)

what makes a good metric?

A RATE or a RATIO (rather than an absolute or cumulative value. New users per day is better than total users.) (Focus on rates and ratios and year over year comparative sorts of things. They are good bc they take out the effects of seasonality) COMPARATIVE to other time periods / segments. No more complicated than A GOLD HANDICAP. Otherwise people won't remember and discuss it. something which MAKES YOUR PREDICTIONS MORE ACCURATE. something which WILL SIGNIFICANTLY CHANGE YOUR BEHAVIOR

Standard Ad Unit

A set of ad specifications for standard image or animated in-page ad units that establish a framework for advertising inventory and webpage design. Inclusive of 728x90, 300x250, and 160x600 ads

Which is the order for Startup Pirate Metrics?

Acquisition, Activation, Retention, Revenue, Referral

how do affiliates get paid

Affiliates work as a relationship with someone online. These affiliates get paid when someone purchases whatever item they are promoting through that relationship. For example, if a website sells comic books through an affiliate relationship with an online bookstore that is featuring them on their homepage that week, the affiliate will get paid once someone buys that book.

amplification rate

Amplification, the rate at which your followers take your content and share it through their network. = # of re-tweets/shares / # of posts

e-commerce

Are you focused on loyalty or acquisition? Pricing matters more than you think Don't overlook logistics, delays, and ratings Old "average conversion rates"

What is audience churn? If you had data with last months visitors, last months unique visitors, this months visitors and this months unique visitors, could you figure it out and roughly how would you do it?

Audience churn is the number of people who visit your site, and a measure of how loyal they are. To calculate this you look at the change in unique visitors in one month by the number of new visitors that month. To calculate the numbers in this specific example you would subtract the number of unique first-time visitors this month from the difference in unique visitors you had this month over last.

CPM vs CPC

CPM stands for cost per thousand, and CPC stands for Cost Per Click. CPC is the price you pay per clicks you want to receive from online viewers, where as CMP is impressions and not clicks.

sales funnel

Concept: easy to understand where we are going in terms of traffic and importance conversion funnel / sales funnel can lead to a purchase, but also to something else - can be dif in terms of optimizing between dif steps strangers become visitors and visitors become leads and you have to close on the leads for them to become customers and if they like your product/service, they become promoters

conversation rate

Conversation Rate = # of Audience Comments (or Replies) / # of Posts A high conversation rate requires a deeper understanding of who your audience is, what your brand attributes are, what you are good at, what value you can add to your followers and the ecosystem you participate in.

What is a conversation rate?

Conversation rate is the number of comments or replies from the audience per post. This helps us understand if we are interacting well with our audience; if what we are showing them and saying to them is engaging and they want to engage with us. It helps us do things the right way by showing us if it is what our audience wants to see.

2 product growth metrics

DAU (daily avg users) / MAU (monthly avg users) measure of How many people use your app on a daily or monthly basis the ratio of your daily active users over your monthly active users, expressed as a percentage. Usually apps over 20% are said to be good, and 50%+ is world class. if your product is a high-frequency, high-retention product that's ultimately going to be ads supported, DAU/MAU should be your guiding light. But if you can monetize well, develop network effects, or quite frankly, your natural cadence isn't going to be high - then just measure something else!

dimensions vs measures

Dimensions: Are qualitative and do not total a sum. For example, sales region, employee, location, or date are dimensions (date, measure names) Measures: numerical values that mathematical functions work on. For example, a sales revenue column is a measure because you can find out a total or average the data (numbers)

restaurant example with empathy, stickiness, virality, revenue, and scale

Empathy: Before opening, the owner first learns about the diners in its area, their desires, what foods aren't available, and trends in eating. Stickiness: Then he develops a menu and tests it out with consumers, making frequent adjustments until tables are full and patrons return regularly. He's giving things away, testing things, asking diners what they think. Costs are high because of variance and uncertain inventory. Virality: He starts loyalty programs to bring frequent diners back, or to encourage people to share with their friends. He engages on Yelp and Foursquare. Revenue: With virality kicked off, he works on margins—fewer free meals, tighter controls on costs, more standardization. Scale: Finally, knowing he can run a profitable business, he pours some of the revenues into marketing and promotion. He reaches out to food reviewers, travel magazines, and radio stations. He launches a second restaurant, or a franchise based on the initial one.

software company example with empathy, stickiness, virality, revenue, and scale

Empathy: The founder finds an unmet need, often because she has a background in a particular industry or has worked with existing solutions that are being disrupted. Stickiness: She meets with an initial group of prospects, and signs contracts that look more like consulting agreements, which she uses to build an initial product. She's careful not to commit to exclusivity, and tries to steer customers towards standardized solutions, charging heavily for custom features. She supports the customers directly from the engineering team until the product is stable and usable. Virality: Product in hand, she asks for references from satisfied customers, and uses them as testimonials. She starts direct sales, and grows the customer base. She launches a user group, and starts to automate support. She releases an API, encouraging third-party development and scaling potential market size without direct development. Revenue: She focuses on growing the pipeline, sales margins, and revenues while controlling costs. Tasks are automated, outsourced, or offshored. Feature enhancements are scored based on anticipated payoff and development cost. Recurring license and support revenue becomes an increasingly large component of overall revenues. Scale: She signs deals with large distributors, and works with global consulting firms to have them deploy and integrate her tool. She attends trade shows to collect leads, carefully measuring cost of acquisition against close rate and lead value.

delivery model business model

How the product gets to the customer. usually through: Hosted service Digital delivery Physical delivery dropbox ex: Cloud storage, web interface. Desktop client software. e-com ex: Ground shipping of purchases via courier service. media ex: Media is online but also available through downloadable app.

revenue model business model

How the startup extracts money from its visitors, users, or customers. usually through: One-time transaction Recurring subscription Consumption charges Advertising clicks Re-sale of user data Donation drop box ex: $99/year, monthly fees, enterprise tiers. e-com ex: Price of purchase. media ex: Banner, PPC, and affiliate links generate revenue.

acquisition channel business model flipbook page

How the visitor, customer, or user finds out about the startup. usually through: Paid advertising (banner on an info website) Search Engine Mgmt. (High pagerank for ELC in kid's toys) Social media outreach (active on accounts) Inherent virality Artificial virality Affiliate marketing (Sharing a % of sales with a referring blogger) Public relations App/ecosystem mkt. drop box ex: Sharing files with others. Free storage when others sign up e-com ex: Word of mouth, sharing purchases, affiliates. Less emphasis on loyalty programs or return purchases. media ex: Publishing to social and RSS feeds; promoting tablet/mobile app.

analysts look at metrics that drive the bus called

KPI's

benefits of outliers

Outliers can not only help you identify if something in particular isn't working, but it can help you think through UX designs of your page and the way people go through the site. When looking at engagement outliers can tell you important things such as: a lot of people come to your site but leave immediately and don't engage - this is a problem, something is throwing them off from the beginning. It can also tell you what pages they are most engaged on, and which they are not by outliers showing a very small amount of time on some verses the others.

input metric vs output metric

Output metrics help you set long term goals for the sustainable growth of your business - ex: $6 million in revenue, 100k weekly active users, 1 million monthly active users Input metrics represent the actions that influence the output metric - ex: 10,000 pageviews, 1,200 registrations, 700 upgrades. You can't focus exclusively on output metrics because they're too big, too broad, and not actionable - they are a scoreboard. To win the game you need to focus on the individual plays that drive the score. Monitor output metrics to know how you're doing, but build experiments around the input metrics you can directly influence. actions are input metrics and there are several of them to reach the result of the output metric

ppc

Pay Per Click: The advertiser pays a certain amount for each click through to their website. The amount paid per click through is arranged at the time of the insertion order and varies considerably. Higher pay per click rates recogniz

Split testing (A/B testing)

Split testing applies to social media posting, ad campaigns and product features. It is simply comparing two versions of something versus each other to see which one performs better. Offering 2 variations is A/B testing need to have statistical significance bc a win by a small amount may not predict with enough confidence if adopted it will have the same effect.

marketing cycle

Want to acquire people, then activate them / sign them up, then you want to keep them, then you want to sell them again and then have them share and refer to others

Cost Per Action (CPA):

What an advertiser pays for each visitor that takes some specifically defined action in response to an ad beyond simply clicking on it. For example, a visitor might visit an advertiser's site and request to subscribe to their newsletter.

product type business model

What the startup does in return. May be a product or service; may be hardware or software; may be a mixture. usually through: Software Platform Merchandising User-generated content Marketplace Media/content Service dropbox ex: Storage-as-a-service with APIs, collaboration, synchronization tools. e-com ex: Bulk-breaking, assortment, and distribution of physical goods. media ex: News site with editorial content.

selling tactic business model

What the startup does to convince the visitor or user to become a paying customer. usually through: Simple purchase Discounts & incentives Free trial Freemium Pay for privacy Free-to-play drop box ex: Limited-capacity accounts are free; subscribe when you need more. e-com ex: Seasonal specials, free shipping. media ex: Have good content; make it relevant to desirable ad placement.

Earned Media:

When people speak about and share your brand and your product, either in response to content you've shared or via voluntary mentions. It's free publicity generated by fans.

Do you metrics like amplication differ based on varying social media platforms like Twitter vs Instagram? Why?

Yes metrics such as amplification differ based on the social media platform we are looking at, this is because every social media channel has different rules, limitations and guidelines when it comes to sharing content. For example, the changes can be different if one social media platform allows you to buy shares, ads, or reposts more than others.

A/B tests tell you what happened (if setup well and you have enough visitors and are measuring the variables correctly). How can you tell why it happened? What methods could you use?

You should use qualitative methods to figure out why things are happening the way they are happening in the results of your A/B test. Surveys are a quick, simple, and effective way to gather the answers to your why questions on user engagement and interaction. Another way to get answers to why things are happening is focus groups. You could do A/B testing in a controlled area with a random group of users, and then after the tests, ask them why they made the decisions they made.

5 aspects of business that make a business model flipbooks

acquisition channel selling tactic revenue model product type delivery model

How people find out about the business is through the _______ ____. How you convince visitors to become customers is the ______ ____ . How you make money is the ____ _____ . What your business offers in return for the money is the ___ ____ . How you get it to customers is the ___ ____ .

acquisition channel selling tactic revenue source product type delivery model

the five stages of lean analytics

empathy stickiness virality revenue scale

to avoid false metrics, we look for useful metrics, which are both?

both accurate (in that it measures what it says it measures) and aligned with your goals.

Metrics that make another metric happen are called ____ metrics

causal

____ is a superpower, bc it lets you change the future

causality

Each group of users is a WHAT? participants in an experiment across their lifecycle. You can compare them against one another to see if, on the whole, key metrics are getting better over time.

cohort

lean analytics: what makes a good metric

comparative, understandable, ratio or rate (ratios are easy to act on, comparative), changes the way you behave

Correlated versus causal metrics

correlated: 2 variables/ metrics that change in similar ways together, perhaps bc they're linked to something else causal: an independent factor that directly impacts a dependent one. indicates that one event is the result of the occurrence of the other event

____ lets you predict the future ____ lets you change the future

correlation lets you predict/report the future causality lets you change the future

ex of vanity metrics

data u cannot act on that strokes ur ego should be able to answer the question "what will i do differently based on this info?" - users, followers/fans, DAU/MAU, total signups, total active users, number of hits, number of page views, number of visits, number of unique visitors, number of followers/friends/likes, time on site/number of pages, emails collected, number of downloads not: - percent of users who are active, number of users acquired over period of time

Tableau separates data in to dimensions and measures . _____ can't be aggregated._____ are fields you can use for calculations. ____fields are often used for row or column headers; ____ are usually used for plotting values.

dimensions, measures, dimensions, measures

economic value

economic value=conversion of some kind = value per visitor Economic Value = Sum of Short and Long Term Revenue and Cost Savings

leading vs lagging metrics

leading: number today that shows metric tmrw - predicts the news. By that I mean that you find a number that's correlated (or causal) to something else that matters to your business. lagging: historical metric that show how you're doing - reports the news - Leading metrics give you a predictive understanding of the future; lagging metrics explain the past. Leading metrics are better because you still have time to act on them - you move from tracking lagging indicators to identifying, tracking and experimenting with leading ones

the __ ___ cycle is a constant cycle of hypothesis and experimentation

lean analytics

a ___ ____ is someone that is a unique visitor who purchased something and shared something over the past 30 days.

loyal user

applause rate

measures vary = # of favorite clicks/likes/+1 / # of posts what audience likes and doesnt like

Good North star metrics are an ____ metric ____ metrics represent results and _____ metrics represent actions.

output When choosing which metrics to focus on, you must differentiate between output and input metrics. output, input

_____ metrics help you set long term goals for the sustainable growth of your business - $6 million in revenue, 100k weekly active users, 1 million monthly active users, $10 million in MRR are all great examples. _____ metrics represent the actions that influence the output metric - 10,000 pageviews, 1,200 registrations, 700 upgrades from free to paid, for example.

output input You can't focus exclusively on output metrics because they're too big, too broad, and not actionable - they are a scoreboard. To win the game you need to focus on the individual plays that drive the score. Monitor output metrics to know how you're doing, but build experiments around the input metrics you can directly influence.

prioritization frameworks we've used in split testing

pixel, pxl, vice this is how we do prioritization - which is a defensible process organizationally and helps you choose most impactful order (given constraints)

how to choose OMTM

purpose, specific, trackable, business value

5 things you need to know about metrics

qual or quan exploratory or reporting vanity (impressions, views, etc) or actionable correlated or causal (not related) leading or lagging

Qualitative vs. Quantitative metrics

qual: unstructured, anecdotal (based on personal exp - not necessary reliable), revealing, hard to aggregate (warm and fuzzy) quan: numbers and stats, hard facts but less insight (cold and hard)

A WHAT is simply a group that shares some common characteristic.

segment

Exploratory metrics are ____ If two metrics change together they are ____ , if one metric "makes" another happen they are ___

speculative correlated, causal

The four main advertising revenue models are

sponsorship, display advertising, click advertising and affiliate.

combine ____ and ___ to find your OMTM

stage and model (one metric that matters)

eric rie's three engines

stickiness approach: keep people coming back math that matters: get customers faster than you lose them virality approach: make people invite friends math that matters: how many they tell, how fast they tell them price approach: spend revenue getting customers math that matters: customers are worth more than they cost to get