Marketing Analytics Exam 2
A/B tests are...
controlled experiment
A between-subject
design is an experiment that has two or more groups of subject search being tested by a different testing factor simultaneously.
A reduction in the price of a good
does not shift the good's demand curve leftward but does increase the quantity demanded.
What can you conclude about the price elasticity of demand in the following statement? "The pizza delivery business in this town is very competitive. My company will lose half my customers if I raised the price by as little as 10%."
elastic
James pays overhead each month, including his company's bills for rent, heat, interest, and salaries, which are examples of ________ costs.
fixed
A manufacturer has invested $750,000 in a new product and wants to set a price to earn a 15 percent ROI. The cost per unit is $18 and the company expects to sell 50,000 units in the first year. Calculate the company's target-return price for this product.
$20.25
Price elasticity:
--the change in demand in a market in response to a product's change in price --how sensitive consumers are to a change in price of a product.
General pricing model
-Price at the profit maximizing point Solve the profit maximization problem.
Target-return pricing
-Price that yields its target rate of return on investment =unit cost + {(desired returnxinvested capital)/(unit sales)}
Monopoly pricing
-Price your product without worrying about your competitor -Price = 1/(1+own elasticity)*MC
What is the most Common form of AI?
Applied AI
Threats to Internal Validity
1.History effect: Events external to the experiment that affect the responses of the people involved in the experiment 2.Maturation effect: Changes in the respondents that are a consequence of time, such as aging, getting hungry, or getting tired. 3.Pre-test effect: The fact that someone has been measured previously might effect their future behavior 4.Instrument variation: The method used to collect data changes within the experiment (e.g., questionnaire, interviewer, etc.) 5.Selection effect: If units self-select themselves into the treatment and control groups then this is of serious concern if the selection reason is related to the outcome of interest. 6.Mortality (or drop out):The sample becomes unrepresentative
What is the break-even quantity (in units)?
14,000
Consider the following regression: log(Yi) = α + βlog(Xi) + εi, where Yi is sales in region i, Xi is the average price of product A in region i, and εi is the error term. The estimated value of β is -2.37 and the estimated value of α is 1350. The marginal cost of producing product A is 100. Based on the above information, what is the optimal price based on monopoly pricing rule?
29.67
The table below lists the cross-price elasticities of demand for several goods, where the percent quantity change is measured for the first good of the pair, and the percentage price change is measured for the second good.
3
The table below lists the cross-price elasticities of demand for several goods, where the percent quantity change is measured for the first good of the pair, and the percentage price change is measured for the second good.
3.15%
A/B Testing = A/B/N Testing
=Split testing
Markup pricing
=unit cost/(1-desired return on sales)
Before and after
A before-after design involves the researcher in measuring the dependent variable both before and after the participants have been exposed to the independent variables. The before-after design is based upon the after-only design, in that the effect of the independent variable, if any, is established by observing differences between the value of the dependent variable before and after the experiment.
●Propensity Model:
A statistical algorithm used to predict customer behavior ●Customer Behavior created by feeding large quantities of data to machine learning algorithm
Which is not true about a natural experiment?
A.synonymous with a "lab experiment"
What is a market?
An exchange between two partners (frequently buyer and seller, but also apply to non-profit or non-monetary exchange)
Experimentation -Definition
An experiment is a procedure in which one or more causal variables -for example, a marketing message about a product -are systematically manipulated and data on the effect variable -for example, choice to buy the product or not -are gathered while controlling for other variables that may influence the effect variable (e.g., Christmas season, weather, etc.)
After only
As the name suggests, with after-only experimental designs measures of the independent variable are only taken after the experimental subjects have been exposed to the independent variable. This is a common approach in advertising research where a sample of target customers are interviewed following exposure to an advertisement and their recall of the product, brand, or sales features is measured.
Field experiments are...
B.conducted in a natural setting.
Which is false about the requirements for causality?
C.(3) must be done by randomizing participants to condition
Which is true?
D.An experiment has 1 type of unit to which its treatment is assigned only
Which is false about causal inference of X on Y?
D.X must be manipulated
Other things being equal, if the price of coffee increases significantly, then:
Demand for coffee substitutes will increase.
Elasticity
Elasticity = (ΔQ/Q) / (ΔP/P)
The follow output models (Coefficients image) the final exam score prediction based on homework, clicker, exam 1 and exam 2 scores. What variable is the most significant predictor of final exam score?
Exam 1
Number of Factors
Factors can be any number of things; these are typical: •Product •Promotion •Place •Price •Segment
AI MARKETING: Data Collection
Organisms amplifying group intelligence by forming flocks, schools, shoals, colonies, and swarms(eg. bees, birds, fish, ants) ●Research shows →groups outperform most individuals in decision making & prediction regardless of expertise ●Most surveys, focus groups, and polls fail to recognize this social aspect of decision making
Elasticity interpretation
PE = 0: Perfectly inelastic demand -meaning that price has no impact on quantity. PE between -1 and 1: Relatively inelastic demandPE > 1: Changes in price impact demand in opposite direction. PE < -1: Changes in price impact demand in the same direction.
Own and Cross Price Elasticity
PEcookies= (%ΔQcookies/ %ΔPcookies) (-)CPEmilk,cookies= (%ΔQmilk/ %ΔPcookies) (-) CPEmuffin,cookies= (%ΔQmuffin/ %ΔPcookies) (+)
Design the Experiment
Potential experimental designs •After only •Before and after •Between subject vs. within Subjects
Sales = 37.69 + 1.23 * TVads. If you spend $2387 on TV advertising, your best guess at sales is (plug number back into equation)
closer to $3,000
Which of these is an example of Applied AI?
Spotify
MACHINE LEARNING:
Systems that automatically learn and improve without being programmed to do so using large amounts of data and algorithms 2 Categories: ○Supervised ○Unsupervised
Unsupervised Machine Learning
Unsupervised learning is where you only have input data (X) and no corresponding output variables. •The goal for unsupervised learning is to model the underlying structure or distribution in the data in order to learn more about the data.
●Artificial Design Intelligence:
Use of AI for ○Design of websites ○Written content and graphics ○Writing Reports ○Creating website dashboards ○Email communications ●If a company has numerically based content and data-driven info, the company could benefit
Supervised Machine Learning
You have input variables (x) and an output variable (Y) and you use an algorithm to learn the mapping function from the input to the output. Y = f(X)
What is the most used form of artificial intelligence?
applied AI
Which of the following is a graph that displays information as a scatter plot with an additional dimension of quantitative data represented by changing the size of the dot?
bubble chart
Suppose people buy more of good 1 when the price of good 2 falls. These goods are
complements
If demand (i.e., consumer purchase decisions) hardly changes with a small change in price, the demand is said to be
inelastic
AI
is an area of computer science that "emphasizes the creation of intelligent machinesthat work and react like humans"
A within-subjects design
is an experiment in which the same group of subjects serves in more than one treatment. Note that I'm using the word "treatment" to refer to levels of the independent variable, rather than "group".
relationship visualization
relationship charts are used to see the relationship between the data and can be used to find correlations, outliers, and clusters of data. "is there correlation between advertising spend and sales for our products:"
The demand for a goods increases when the price of a substitute ________ and also increases when the price of a complement ________. (price elasticity formula)
rises;falls
A drop in the price of a compact disc shifts the demand curve for prerecorded tapes leftward (in other words, the demand for prerecorded tapes drop, all else equal). From that you know compact discs and prerecorded tapes are
substitutes
General (Less common)
that possessed the same characteristics of human intelligence. You've seen these machines endlessly in movies as friend —C-3PO —and foe —The Terminator. General AI machines have remained in the movies and science fiction novels for good reason; we can't pull it off, at least not yet.
•Applied (More common)
trading stocks or controlling an autonomous vehicle, Image classification on a service like Pinterest and face recognition on Facebook.
comparison visualizations
used to compare magnitude of values to each other and can be used to easily find the lowest and highest values in the data
composition visualizations
used to see how a part compares to the whole and how a total value can be divided into shares
distribution visualization
used to see how quantitative values are distributed along an axis from lowest to highest
relationship visualization
used to see the relationship between the data and can be used to find correlations, outliers, and clusters of data
Cost-based Pricing
• A method in which companies use the cost of a good or service as the basis for determining its final selling price • Cost-based pricing is calculated by taking the product cost and adding a markup to determine the final price
Break Even Point
• Breakeven Quantity The quantity a company needs to sell at a certain price in order to cover fixed costs • Breakeven Price The amount a marketer needs to price a product in order to cover expenses at a certain quantity sold
Know your cost
• Fixed Cost The costs associated with the operating and marketing expenses of a company • Variable Cost The per-unit costs associated with the product
A/B Tests
•A/B Tests are field experiments in the digital marketing context making use of big data •A/B Tests are often run as part of a live, real marketing effort. •A/B are often automated to be adaptive.
Experiments -Procedure
•Decide on your treatments •Design the experiment design •Recruit participants and run the experiment •Analyze the experiment data
Definitions
•Dependent variable -the outcome variable •Independent variable -manipulated variable for which the marketing analyst is interested in seeing an effect on the dependent (outcome) variable •Moderator variable -it changes the way the independent variable affects the dependent variable
Validity
•Internal validity is the extent to which the observed results are due to the experimental manipulation •External validity is the degree to which the experimental results are likely to hold beyond the experimental setting •Usually there is a tradeoff between the two •Without internal externality, external validity means nothing ==> Fix the level of internal validity and strive for the highest possible external validity.
What are the tools to deliver those three marketing principles?
•Marketing mix (4 Ps ): product, place, promotion, price
Price Elasticity
•Own price elasticity: changes in demand of a single product due to its price •Cross price elasticity: with changes in demand of one product due to changes in price of another.
Laboratory Versus Field Experiment (and AB Test)
•Validity: A field experiment tends to have a higher external validity but a smaller internal validity •Exposure: By doing a field study, you may provide information to competitors or adversely affect the marketplace
Validity and Reliability
•Validity: I am measuring accurately •Reliability: I can depend on the measure
Marketing Channels for A/B Testing
•Website promotion (copy, images, video) •Email campaign (messaging, subject line) •Social media ads (messaging, images, video) •Digital retailing (messaging, pricing) •Text coupons (messaging, price promotion) •Mobile ad (messaging, images)
○Artificial Applied Intelligence
•refers to systems that are designed to work on specific tasks, such as trading stocks or controlling an autonomous vehicle. •is most common
○Artificial General Intelligence
•refers to systems that can handle any task •is more complicated since the systems are dealing with an infinite number of tasks rather than focusing on just one •is less common
A/B Tests are rooted in experimental design ●However, often:
○A/B Tests often test two or more competing messages without necessarily seeking to understand why. ○Experiments test hypotheses, seek to understand outcomes and reasons
●Propensity model consumer metrics:
○Likelihood of conversion ○Price Point of conversion ○Which consumers to target to become repeat customers ○Predict lead effectivity ●Only as good as the data you put into them
AI MARKETING: Content Personalization
●(Artificial) Intelligent Personalization: Algorithms that can personalize websites to individual site users ○Based on: ■Geographical Location ■Demographics ■Types of Device being Used ■On-Site Interaction ○Companies like WSJ, Pandora, and TopFan use AI to improve conversion rates
Market-based Pricing
●A company looks to its competitors as the main pricing factor to determine the price of its product or service ●The company can price similarly or strategically price higher or lower than competitors
OVERVIEW: Marketing Application
●AI Marketing Tool Platforms ●AI used for Content Personalization ●AI used for Data Flow Monitoring ●Propensity Model (Predicting Customer Behavior) ●AI used forContent Creation ●AI used for Data Collection
AI MARKETING: Data Flow
●AI detects when unexpected traffic is visiting a site ●Analytics is checked autonomously by AI platforms
Marketing Examples & Applications
●According to a recent survey, one third of advertisers are using A.I. to deliver personalized web experiences ●Applications include image recognition, video analytics, language processing and much more ●Personalization is based on factors such as geographic location, demographics, types of device being used, & on-site interaction ●This info is used to display offers and content that fits each user ●One of the main applications of this can be seen from music streaming services such as Spotify when it curates recommended playlists and suggests songs based on previous listens
AI MARKETING: Tool Platforms
●Albert ○Used for autonomous media buying ■Buying digital media on behalf of clients ○Analyzes, manages, and optimizes paid advertising campaigns
The Econometrics theory behind AB Testing: Causal Inference
●Causal inference is a field for understanding the causal relationships between different events. ●Let's formalize the notations used in causal inference. ○Let 𝑥𝑖be the data units, e.g., consumer. ○𝑥𝑖: Pretreatment covariates. These can be segments, age, gender, registration rate, past purchase history of the users. It is often a 𝑚dimensional vector where 𝑚is the number of features about each user. ○𝑑𝑖: Treatment. is often binary (e.g., Facebook case), but can also take multiple, or even continuous values (e.g., Google case). ○𝑦𝑖: Observed outcome. In the above examples, this can be an indicator of whether user stay with Facebook or click or the weblinkprovided by Google search engine.
Examples of Machine Learning
●Computational finance, for credit scoring and algorithmic trading ●Image processing and computer vision, for face recognition, motion detection, and object detection ●Production, for price forecasting ●Automotive, aerospace, and manufacturing, for predictive maintenance ●Natural language processing, for voice interaction
Experiments -Procedure
●Decide on your treatments○What are you trying to test? ○How many factors? ○What will you do if you have the information?
open source tools
●Designed to deliver easy-to-make charts, plots and maps ●Trulia uses R to form map overlays for crime, traffic, amenities, etc
4 cornerstones of data visualization
●Distribution of a single variable ●Relationship between two variables ●Composition of a single or multiple variables ●Comparison between different categories/individuals
AI MARKETING: Tool Platforms
●Frank ○Pay-Per-Click tool ■Uses Machine learning to find best paid advertising channels depending on specific audience
Defining The Fixed Unit Effect
●In the presence of others, people consume fixed number of food units based on what they perceive to be an appropriate quantity of units of a given food. ●The result ○Unit size is held constant ○Total caloric intake varies with unit size (i.e., smaller or larger units), and consumers will consume more calories with larger unit sizes.
Cost-based Pricing
●Markup pricing ○Add a standard markup to the product's cost ●Target-return pricing ○Price that yields its target rate of return on investment
Applied (Most common)
●Perform single task extremely well ●Run automated and repetitive tasks ●Does NOT involve decision making ●Ex) Netflix, Spotify, self-driving cars
The marketing mix
●Price refers to the total cost to customer to acquire the product. ●Product refers to what the business offers for sale and may include products or services. ●Place refers either to the physical location where a business carries out business or the distribution channels used to reach markets. ●Promotion refers to the marketing communication used to make the offer known to potential customers, such as advertising, public relations, direct selling and sales promotions.
Causal Inference in Simple Terms
●Requirements for Causal Statement X →Y ○(1) X must occur before Y ○(2) There must be evidence of association between X and Y ○(3) Control of other causal factors ○(1) is typically manipulated. (2) can be assessed by the data. (3) is typically done by randomizing, matching, or blocking ●Example: ○product advertisement on TV →Choice of product purchased
an example of AI at Uber
●Uber used K-Means Clustering within Machine Learning to minimize wait times -utilizes centroids
