Facebook Product Growth Analyst

"Say you notice a surge in average activity among users but the number of users is down, how would you explain this? "

feature is best used by smaller, more targeted segment I explained that users that may not have been very active decided to deactivate their facebook, causing the average activity to go up, but the amount of users to decrease.

"Explain how you would diagnose a 10% YoY ""like"" improvement"

59

"How would you compare the relative performance of two different back-end engines for automated generation of Facebook ""Friend"" suggestions? "

A/B test, comparing percent clicks and average number of friend adds of the two samples

You are the PM for a B2C product that has an advertisement-based monetization model with significant and steady daily revenues. One day, there are no ads served and the revenues plummet to zero. What would be your strategy, as a Product Manager, to deal with this crisis?

Clarification No ads served - was there an abrupt drop (sharp cut off in a few hours) or did it taper off over a period of time (say 1 month) and finally there were no ads that were served one fine day Was there any geographic focus of the product? Did this drop happen specific to a geographic region? Is it a web based product? Or Mobile product? Or both? Assumptions This was a sharp drop (as otherwise as PM i should have noticed the steady decline) This was an overall drop This is both web and mobile environment. Advertising ecosystems are 2 sided Client traffic (demand) Advertisers (supply) System Root cause analysis Client Was there any sharp drop in client traffic on one fine day? If yes, need to explore why Was there any significant event externally for e.g. US presidential elections which tuned users away from the app or blacklisting the product by Gov etc. Was there a recent product release for e.g. login changes which locked out a large no of users Did the product depend on traffic from external partners significantly say FB and there was a drop due to algo change or some other issue on their part. Advertisers Analyze if the campaigns inadvertently ended on the one fine day This could be similar to Y2K problem If bulk of the ads are from Third party Supply Side platforms, check if there are issues from their end (blacklisted , contract changes, technical changes etc) Analyze if bulk of the revenue came from 1 or 2 specific segments of advertisers which were affected by environmental changes, Analyze if there are external factors which lead to advertiser cancellation Pricing increases Policy changes Key features inadvertently dropped System Issues Release which caused ad systems to be unavailable Releases which caused key DBs to be unavailable Release/ Software changes which caused ads to no longer called from client Release/ software changes which caused key attributes to not send (which causes ads to be not matched) Considering the above scenarios, based on the root cause analysis the following actions can be taken Client Traffic drop due to login or release issues Hot release to fix it (considering mobile environment, cannot roll back if there have already been installs) External issues: If the product is left redundant due to external factors, check what other services/ content/ features which can be provided If it is a temporary drop,then wait it out for a couple of hours to see if traffic comes back Advertisers Check for campaign statuses with last updates made (to see if advertisers ended the campaigns) Work with BD teams to resolve issue with ad networks/ products Work with Sales to reach out to the big customers to see who & why cancelled? System Work with engineering teams to identify any technical issues

"If you were to pick a sample for an experiment, how would you choose the size of the sample. "

how fast you can collect data but also considering how much of a sample size to get accurate data

"We at Facebook would like to develop a way to estimate the month and day of people's birthdays, regardless of whether people give us that information directly. What methods would you propose, and data would you use, to help with that task? "

map usage habits of people with usage habits of users who have given their demographic. This way you can predict their ages. what day of the year there are the highest number of posts from others. Perform text analysis and search for "happy birthday" in post comments, then use that to find the date on which the post occurred. Zodiac

How do you decide between displaying Facebook's 'People You May Know' feature or an Advertisement?

Define the product goal and metric of success, North Star, for X. Define the product goal and metric of success, North Star, for Y. Convert both metrics into a single common metric that measures a common business impact. we need to run A/B tests across different segments, from new to older users.: "new users with few friends," "semi-new users with more friends," and "existing users with more friends." I would suggest two attributes to define these segments: "the number of days passed since they registered" and "the number of friends." Feature -> goal -> metric -> common metric -> A/B segmented testing

How to measure success of facebook likes:

Describe Like button offers low friction way to signal interest Inside FB, provides validation back to content creator Across web allows user to bookmark content they are interested in Allows FB to build expanding interest graph for each user Creates virtuous engagement cycle, both liker and like-ee Feature goals EngagementMore consumption and production of content Richer user insightsSurface more personalized contentOffer advertisers better targeting Journey I'm scrolling through newsfeed I see friend's video No time to comment Click Like I feel good about sending positive signal My friend gets endorphin hit More like-able content shows up in my feed My friend is motivated to post more content Success metrics Is the feature discoverable?Focus groupsLikes per session Are users using the feature as intended?% Un-likes% of Likes followed by a comment Is usage of the feature growing?average likes per userLikes per session What is driving usage of the feature?Likes by content type (video, photos, articles, webpages, groups, etc)Likes by user segment (age, geography, time on platform etc) Does the feature increase engagement?Average session length by Like volume by userPost frequency by like volumeContent interactions by like volumeSession frequency by like volume Does the feature increase user value to advertiser?Ad clicks by like volume by userLength of ad video viewed by like volume

"How do you map nicknames (Pete, Andy, Nick, Rob, etc) to real names? "

Detect how his friends address him in comments posts and birthday greetings posts. From comments in his Facebook network, detect from comments again how his friends address him. From external sources (other social media network or websites) Using FB first name, last name and profile pic, search for external sources for possible real names Create a mapping table based on user feed where both real names and nick names are available. Develop a probability of nick names for real names. Select the nick names with higher probability. Pilot Testing: Run a pilot testing program to test out the correctness of mapping and user's interaction. provide suggestion of nick name for real name. for example suggest Pete for Peter. record the feed back: 1. correctness of suggested nick name 2. user interaction

measuring success of a feature

total number of times people are using the feature. the number of unique users who are using the feature. the percentage of your total active users who are using the feature. the average number of times per day users are using the feature.

How would you measure the health of facebook group? What metrics and counter metrics would you use?

Engagement Metrics : Number of members within the FB Group, Number of posts per day, Number of likes per post, Number of comments per post, number of events posted within the group, Number of active people within the group ie. Number of people that post/like/comment/share posts , Number of new members per week Retention : Divide the member groups into cohort based on when they joined the group (Month 1 Cohort, Month 2 cohort and so on). Calculate the retention rate with time across these cohort groups, Engagement across the retained members Get these metrics and validate how these metrics look to assess how the health of the FB group is.

"What is a Facebook product that you'd be interested in working on, and how would you use analytics?"

Facebook integration of TikTok reals. Because Facebook is more targeted now towards older people, would be interesting to see the demographic of people who engage.

Structure of answering a trade off question:

Feature -> goal -> metric -> common metric -> A/B segmented testing

"How many high schools that people have listed on their profiles are real? How do we find out, and deploy at scale, a way of finding invalid schools? How can Facebook figure out when users falsify their attended schools? "

Find similar users based on places born, grow up, street names, tags, also their friends. And if in a circle no one or very few people have listed that high school name, it is likely a fake high school. If you need to use an algorithm, us K Nearest Neighbors, find all the similar users with the person with potentially fake high school, then list out and rank of similar users' high school. Find from available public data:Get a list of high school names (from government or trust worthy organizations) and match high school names. Find from users voting or set up reporting mechanism:People can report scam or down vote fake high school names

How can we improve (grow) the no. Of connections? Feature suggestions:

For this we have to check the ctr for 'people suggestions' and 'invite' features. If CTR for 'people suggestions' is low it could be A. Not enough folks to recommend B. Feature not visible Therefore, improve 'people suggestions' to not only display people to add but to follow (e.g. influencers). Second, Show people suggestions at the top of my wall Third, display as suggestions new connections added by anyone in my circle. If invites sent out by the user is less, then improve invite feature. Integrate with phone contact and allow users to send out invite using sms or whatsapp. Improve the invite message by showcasing what FB has to offer. For anyone joining using invite link add them to invitee network.

Structure of answering a growth question:

Goal (What facebook wants to deliver) --> hypothesis (WHY it is not growing) --> stats to prove this (data, HOW) -> possible suggestions to grow (WHAT to deliver now) -> segmentation -> test A/B suggestions on segmentation -> look at stats again What facebook does -> why this feature exists -> what metrics we care about -> segmentation of users -> pain points for segmented users -> solutions for pain points -> evaluate impact to cost ratio -> conduct A/B testing -> Evaluate change in important metrics

Estimate why would the ratio of like/daily active users would increase suddenly?

Having daily active users in the denominator is misleading because there are different user groups that will be responding differently to your product update.

"Facebook has personal information such as gender and height. If I were to make a claim that men were taller than women, how would you go about proving/disproving this claim? "

Height_male > Height_female as the null hypothesis, taking a few random samples and perform a Z test. I messed up on the details of the Z test though.

How to choose emotions in like/dislike systems and any problems in the A/B test.

I believe that we should think about the average of engagement of a content (test a variety of contents - sad, happy, impressing, etc) in both A and B groups. Let's say that if the group that is seeing the reaction button engages more with content, then the tool is successful.

What experiment would you run to implement new features on Facebook?

Use Facebook experiment feature on Facebook's own new features. Calc CPM, Engagement, Retention

How would you improve Instagram?

What is Instagram Instagram is a mobile application that helps people quickly edit and share photos / videos publicly or to a pre-approved audience. They can also share images, videos, and other people's posts privately via direct messages. People can also share disappearing stories via Stories feature, watch long-form videos on IGTV. User Categories Instagram has three broad categories of users: Creators who share photos Consumers who like and comment on posts, or share them privately Advertisers that want to reach relevant audiences Business Objective Is the goal of our improvement exercise to acquire new users, drive more engagement among existing users, increasing revenue, or something else (like saying limiting Instagram use)? Let's say that our objective is to drive more engagement among existing users. Specific User Segment Let's assume we are focused on improving the experience for Creators because more photos by creators would allow consumers to consume more, which will drive other metrics of engagement. Creators come in various shapes and forms. They can be segmented by kind (celebrities, regular creators, brands and organizations), by frequency of use (high frequency posters and low frequency posters), by nature of users (new users, old users, etc.), by posting history (used to be frequent but not anymore, never posted, posts but has a lot of gaps between posts), etc. Let's pick a regular user who is low frequency poster, an old user, has created some posts in the past but doesn't post too frequently. For this user, our goal is to increase the number of posts they create. User Problems To drive that, we need to understand why people post on Instagram and why they don't. Why people post → why they don't: Show off creativity → I don't feel I'm creative enough or click good enough pictures. Share about life with their friends → My life isn't interesting enough. To get likes and comments → I won't or don't get enough likes and comments. It's a part of my daily life → I forget to post on Instagram even when I click a good picture. User Journey Clicking a good picture → Applying filters and editing the pic right → Adding captions and hashtags → Post → Get likes and comments Possible Touchpoints and Solutions At the creator end Clicking good picture: When a user clicks a good picture, Instagram in the background can run some screening to pick really good pics and tell users to share them (possibly with an indication of number of likes or comments they'll get for such a picture). Editing the picture: If users are confounded by which filter to apply, how to adjust - give an auto suggested set of edits (using AI) that the user can apply in one tap. Adding captions and hashtags: Auto append relevant hashtags to a post based on whatever the post contains (#dogstagram for pics of dogs, etc.) and potentially give caption suggestions (Quotes, perhaps). At the consumer end One of the biggest drivers of posting on Instagram for creators is to get more likes. If that can happen, they will naturally be more incentivized to share on Instagram more frequently. So, we can add a couple of levers to make this more probable as well. Setup a Rising Star leaderboard that shows best pics from new posters (in our cohort) so that they can get more likes from Encouraging Users. Push out a notification to people who have liked or commented on this person's post or interacted with them a lot in the past so that they can like the post. Prioritizing Solutions via RICE InstaWorthy Pic Selector: 9 | 10 | 90% | 7 Filter Suggest: 7 | 8 | 75% | 3 Autocaptions and Hashtags: 8 | 6 | 70% | 4 Rising Star leaderboard: 7 | 8 | 80% | 4 Notification to drive engagement: I think it already exists Metrics to focus on Number of new posts per user per week in this cohort (North Star) Clickthrough rates on InstaWorthy notification Number of posts created with AutoSuggest Filters Number of posts created with AutoSuggest Captions and Hashtags Number of views per post on Rising Star leaderboards (compared to posts by same cohort not on the leaderboard) Number of likes per post on Rising Star leaderboards (compared to posts by same cohort not on the leaderboard)

Describe the personality of a manager who you'd never want to work for.

"Generally, I'm pretty adaptable. However, I think I would struggle if I were working for a manager that didn't provide constructive feedback. I believe that feedback is an opportunity to learn and grow, but it can only serve that function if it is actionable and can help me enhance my capabilities. As a result, a manager that prefers to berate over being supportive is not ideal for me."

Facebook Algorithm Values

- Friends and family come first. - A platform for all ideas - Authentic communications - You control your experience -Constant iteration

Was also asked to create or take an existing product on Facebook and come up with ways to drive engagement

- more videos - more live since people are notified - filter your own feed

LIST OF METRICS

-Number of Fans. -Follower Demographics. -Page Views by Sources. -Actions on Page. Reach by Post Type. -Post Engagement Rate. -Click-Through-Rate (CTR) -Ad Impressions & Frequency.

What metric would you show small businesses if you were trying to have them sign up for Facebook Ads

-The average multiple return on initial investment into the ad. -The highest multiple return on initial investment. -The % of people using ads that have had a positive ROI. -An ability to track ad performance across both Insta and Facebook. - Stats on your best customer. - % increase sales - Conversations from clicks - average increase in following

Top strategies for managing cross-functional teams

1. Build your best team 2. Clearly define your goals Without clear alignment on shared goals, team members may find it difficult to cooperate and collaborate. 3. Use the right communication channels and tools 4. Build trust When building cross-functional teams, encourage team members to reframe their ideas in more simple, general terms that are understandable to anyone in the organization. By developing a cross-functional shared language, you can foster improved collaboration, trust, and creativity. 5. Encourage failure permitting teams to fail results in more breakthroughs. Create a risk-free culture that encourages failure, iterative processes, and growth. 6. Be ready for conflict resolution and difficult situations 7. Differences in opinion are often nothing more than masked opportunities for growth and new ideas. 7. Develop a plan and timeline for the project Even the best-laid plans can go awry without a clear picture of where you're going. Visualize your project from start to finish with the right project management templates customized for the needs of your project and your project management methodology. 8. Continually reevaluate processes

"Facebook sees that likes are up 10% year over year, why could this be? "

1. Check if it is the number of users or likes per users that are increased year over year. If it is the total number of users that increased by 10% or so, then that could be the reason you have the increase in the likes. If it's not the number of users (or if the growth in the user is not quite close to 10%) then check the following two things.1-1. Check if there has been a feature update that made it easier and more convenient for the users to click "like" (e.g., change in the size of font/icon as well as its location and etc). If so there could be another feature that could have been de-prioritized due to this change. Check what other impacts it could have caused due to the increase in the likes.1-2. Check if there has been events that might be the drivers (e.g., number of events, news, posts/shares associated with the news)

How to increase your number of likes

1. Develop a smart Facebook marketing strategy 2. Craft a great Page 3. Make your Facebook Page easy to find 4. Post relevant, high-quality content 5. Engage consistently and at the right times 6. Host a Facebook contest 7. Engage with other brands and communities on Facebook 8. Use the Facebook algorithm to your advantage 9. Run Facebook ads to expand your reach 10. Learn from Facebook Insights

Tell me about when you influenced

1. Think about your persuasion skills 2. Research the company culture 3. Choose a specific example 4. Describe the outcome of your actions. If possible, share tangible or data-driven changes that resulted from your help or abilities. Convincing Michelle of my ideas

What is your favorite product? Why?

A good product is: 1. Useful - Is the product useful and solving a key pain point? 2. Efficient - Is the product solving the issue quicker with less effort from the user 3. Innovation - Is the product solving the user problem in a new way that makes the user smarter 1. Useful - grammarly saves time by providing accurate suggestions. It does it by using latest machine learning algorithm and natural language processing to find the best match/ correction. It also enhanced user vocabulary by recommending more nuanced words that I can use in my writing. It also teaches me how to write more grammatically correct sentences. 2. Efficient - making correction using grammarly is very easy. As the user of the application, I just have to single click on the suggestion and it replaces the incorrect word with the new one. I don't have to open up a new window to do so (in MS word, user has to click multiple times for correction). Correcting an entire para takes as many clicks as there are grammatical errors. 3. Innovation - grammarly uses latest machine learning algorithm and predictive technologies to improve accuracy of detecting wrong word and suggesting the best word to replace. Also grammarly work cross platform.

"Imagine we see a lot of users filling up a form but not submitting it, why would it be the case and how would you use data to finding it out? "

A question made mandatory that they don't wanna answer. WHICH QUESTIONS Form is too long. How far did they fill 1) What kind of form? 2) How many questions in there? 3) What type of questions - drop down selection, radio button selection, fill in type etc.? 4) What percentage is not filling in? If it is >50% not finishing the form, then there is a big issue with the type of form questions structure, most likely it is too verbose and need lot of typing in answers 5) If none of the above applies, I would look at the data and see if any blackout reporting on those areas during the same time when users filled in forms 6) Any geographical localization pattern

"How can we tell if two users on Instagram are best friends?(answer question) Ok, how can we use this algorithm in the product? "

As a proxy, we could measure # of likes+shares+comments between accounts, and pick the top interacting ones every day for past 30 days.

There is a feature for showing friend requests. How do you measure its impact and how do you decide whether to keep it.

Awareness (feature) % of total facebook users who have atleast sent or recieved one friend request ( how many people know about this feature) Activation (feature) % of total facebook users who have atleast sent and recieved one facebook friend ( how many people are activated) Engagement (feature) Average # of freind request sent per user in 7 days period Average # of freind request sent per user organically Average # of freind request sent per user via suggestion Average # of friend request accepted per user Average # of freind request not accepted per user Engagament of (Facebook) Average # of total friends per user Avegare total engagemet of user (sum of all activity by user i.e post, comment, chats, like etc) by All friends % direct engagement and % indirect engagement Average engagement of user by New Friends % direct engagement and % indirect engagement Retention (feature) 30-day retension of users using freind featue ( how many people keep on using the freind feature) Retention (facebook) (retention of user on facebook due to friends) 30-day retension of user due to all friends 30-day retension of user due to new freinds

How would you conduct an experiment to test if a change in facebook app is effective and what metrics will you look at?

Before applying the change to the all users, divide them into 2 groups (A/B test), apply the change to one group and control the metrics below of both. Check the time spent on facebook, check the conversion rate of ads, check the overall engagement of posts, check the app rating on Google Play or Apple Store, with NLP check the content of the posts related to "Facebook" to make sentiment analysis

How to evaluate the success of a product?

Breadth of Use, Efficiency, User Satisfaction, Usability and NPS

Describe a time you had to deal with a conflict?

Briefly describe the conflict that occurred. The situation portion of the STAR method involves explaining the workplace issue briefly, but with enough context that the interviewer can more easily picture the situation. Mention your role in the situation. Use the task section of the STAR method to describe your role in the situation and what you were trying to accomplish. Explain how you approached the problem and any actions you took. During the action step, you should be finished describing the situation and issue. You can now move on to the actions you took to resolve it. Use this section to explain the steps you took to resolve the workplace issue. Share results that prove how the outcome was positive. The final portion of the STAR method is the positive results you achieved from the actions you took to solve the problem. Mention the exact positive outcome you gained from the situation, like a client leaving you a positive review, or share statistical proof with numbers, like an 20% increase in sales, for example. Communication problems with Michelle - open dialogue.

What should be the goal for Facebook Live?

Clarify product: Facebook live to my knowledge is basically a video streaming feature where it's real time and participants and react and comment, and share at the same time correct? Yes OK. Let me ask a couple of clarifying questions: is this for a specific platform like mobile or desktop? No. All platforms included. How about regions? You can think it's US only and limited to English users for now, but it really doesn't matter as Live is used the same everywhere. Goal of Facebook live is to allow people to express themselves through a live broadcast, so it aligns with the oveall mission of facebook to connect, allow people to build a community and express themselves. User Groups: Live broadcasters - they produce the content and therefore provides the supply. What they would do is initiate a live session on their phone or a webcam through their desktop/laptop, and Live watchers - they are the audience and therefore provides the eyeballs. They can also react to a live session with providing reactions, comments, and shares. Latter of which can get more viewers. Typically they would see a live video on their newsfeed being broadcasted by a source based on they liked, or one of their friends. So they would initiate a watch session that way. The alternative is they can go to the Live tab and just scroll and see which one they want to view List of events based on their user journey above: Broadcasters: # of broadcasters, # of live sessions, and length of each live session Watchers: # of live videos watched, time of live videos watched overall, reactions to each live session, comments to each live session, and also shares to each live session I would prioritize some specific metrics: # of live sessions per 10,000 users in a given amount of time - this would be able to tell me how many live broadcasts are initiated, and of course we want this to go up over time # of live videos created per 10,000 users over a given amount of time - while not every facebook user will create a live broadcast this would give me a sense of the avg use. Obviously we would like this number go up over time. # of meaningful reaction per 10,000 users over a given amount of time - same here where I would aggregate reaction, comments, and shares . Since this encompasses views - meaning you have to view a live session to offer a reaction this would be a more meaningful metric All of these metrics however all contribute to how much overall Live content is viewed which can be measured in terms of avg time watched. This is the north star So to summarize, the goal for Facebook Live is to create an additional way for users to express themselves to drive engagement, where we should measure by actions of the live watchers. They can be measured by: % of live sessions by users, % of live sessions created by users, and % of meaningful reactions by users. All of these all contribute to the north star metric of avg time watched per user.

New users signing up for FB are not returning after 30 days. How will you identify the problem? Discuss solutions to that. You are a Growth Marketing Analyst at FB and you have been tasked to identify reasons as to why users signing up from a country are not returning to FB after 30 days. What will be your hypothesis? What data/metrics will you look at prove or disapprove each of the hypothesis How will you prioritize your problem to solve? How will you measure success of your change?

Clarifying questions is this a new country for FB? if not, has this started happening recently? how long has it been happening? Assumption: FB has been in this country for a long time. This started happening recently 3-4 months [repeat question now] Ok, so the change is recent and the new users who are signing up are not returning after 30 days Let me think about what a new user needs on FB- New user is looking to socialize with their friends and engage with meaningful content on the platform >> Hypothesis #1: they may not be finding enough friends to connect and thus the platform may not be engaging to them >> Hypothesis #2: New user is looking for content to engage with and there not finding enough engaging content Data that I will look for: # of connections made per new user per month in that country: I will compare it within the country before this drop started occuring and also to other similar countries I will also look at % of times People You may know (PYMK) feature was shown to the user in their newsfeed and % of connections user made from the feature. This will help me verify if FB showed enough friends and whther the firends were relevant or not. Based on % times this feature was shown, I can devise an A/B test to show this feature more and validate further if it helps improve % of non-friend content shown to new user in the newsfeed:I will again compare it within the country before this drop started occuring and also to other similar countries I will like to conduct an A/B test by showing relevant pages or groups to them in the newsfeed (based on their demographic, FB can gues their interest; since they are new users FB may not have enough info about their interests) and by showing the pages, we can guide them to find new content. We see if the test group stays with FB compared to control to validate the hypothesis. I will measure the success of the tests in terms of improvement in 30 day retention. Since this is metric we were concerned about, we will measure the improvement in the metric

How do you solve conflict?

Conflict often arises when cross-functional teams fail to recognize and understand the trade-offs necessary to achieve outcomes. * In order to help team members be more self-aware and empathetic, you can walk through the process from the perspective of their own team and draw corollaries to other teams. One thing some managers miss is providing opportunities for stakeholders to revisit and revise their agreements as the reality of implementation sets in.

How do you say no to feature ideas or requests?

In my experience, this is how I say no to ideas. Whenever someone suggests an idea to me First and foremost, I would ask them, what the user problem or gain is. If there is none and I am not able to see it, I will explain to the one who suggests that and so the idea to implement would take so much of time and if it doesn't solve a user problem, then we should not waste so many resources on it. If there is a user problem, but the user group is not the current focus of the product, I will tell them that some other user group is in focus currently as decided by the organization If this user group is the focus right now, I will go ahead and ask them what metric is this idea going to improve and if the metric is not the most important metric, I will suggest that this is a good user problem, but we are satisfied with this metric as of now (for example onboarding is in a good shape right now) If the metric is also a focus metric, I will compare the user problem with other opportunities that the product team is facing right now, if this opportunity does not seem promising, I will politely show them the prioritization of opportunities or user problems and say that this is a low priority If this opportunity ranks very high in objective prioritization, I will thank them for the idea and say that we will add the user problem to the backlog and note the idea as well. But, there might be other ways as well to solve the same problem and give them some examples If the idea seems promising, I will test out the idea using an MVP before building it into the product. Irrespective of what happens, I will thank the person for the idea and tell them the importance of providing the feedback. But, also make sure that they understand that each idea takes a long time to implement and multiple iterations, so as a product we really need to focus and only build when we are really confident about it.

"The company developed a new feature and perform A/B test. Here is the result,Comments +5%,Likes -10%,Timespent +1%, All else neutral, How would you decide to whether putting into product based on the A/B test result? Any ideas?"

Is discussion more important or engagement ? Lets examine what does increase in Comments mean for the business. It has increased user engagement in its most richest form. Probably more than likes. This would help Facebook profile users better by understanding users meaningfully from their comments. This opens up the possibility of better targeting ad campaigns (impressions) in the future. The increase in comments could potentially offset the loss in information from the reduction in likes. Assuming that the 1% increase in time spent is statistically significant, and the effect translates to roughly 50% of the users (2 billion total users) and an average user spent 1/2 hour on their app. It would mean half a billion hours more. Thats significant eyeballs and opportunity for improving revenue through impressions and click rate.

You launched a new signup flow to encourage new users to add more profile information. A/B test results indicate that % of people that add addtl. profile information increased by 8%. However, 7 day retention decreased by 2%. What do you do?

Let's start from the WHY behind the change- likely that you were implementing this change to improve retention ; let's proceed with the assumption that this is a social app where profiles play a critical role. An increase of 8% for sign up flows is a significant increase in number of people completing profile information - I would ask how was this implemented because from experience I know that every time we add a step it introduces a 3-4% drop off. My assumption here is that this is an added pop up screen during sign up which is causing a drop off in total number of users completing sign up successfully since they drop off on the profile info screen. *Interviewers nods yes* Also I would recommend changing the way we measure success of the A/B test , let me tell you why - consider the following scenarios In case A for every 100 people signing up 50 people ended up signing up of which 20 completed profile information In case B ( winning variant ) for 100 people signing up 48 people ended up signing up of which 22 complete their profile So the blocking screen is causing an overall drop off which reduces D7 while increase number of people completing their profile I would re-configure the experiment hypothesis to " users with better profile information have higher retention than users who don't - how can i increase in the number of users filling their profile information on D0 ( primary metric) while increasing/not affecting the successful sign up rate ( secondary metric that doubles up as a kill metric) " if my goal is to increase profile information of new sign up I would focus on passive methods ( push notif, in app pop ups, incentives) POST users successfully signing up so as to negate this drop off if my larger business goal is to increase retention I would reduce the steps of sign to increase successful signup and focus on passive methods ( push notif, in app pop ups, incentives) POST users successfully signing up so as to increase total user successfully signing up and help improve D7

"How would you measure the health of Mentions, Facebook's app for celebrities? How can FB determine if it's worth it to keep using it?"

Look at what happens when a celebrity starts using Mentions. Does it drive increases in engagement (celebrities' use of FB)?

How do you measure newsfeed health?

Monitor the active users, number of engagements (likes, comments, shares, views), time spent online, number of content created, ad CTR, revenue from ads. Another way is asking directly to users what they think of the newsfeed shape/organization/sorting of contents.

How can you improve Facebook Stories?

My take on product improvement for Facebook Stories 1. Describe the productFacebook's mission is to connect people, establish and foster connections between them. Stories are a fun and spontaneous way of keeping your friends updated with what's happening at your end. For Facebook, user generated content is extremely important. Considering the decline in sharing of status messages and the clutter in an individual's news feed, Stories is imperative for Facebook in order to create high quality user generated content and increase engagement among users. 2. Scope of the questionAsk the interviewer if there is anything in particular we should be focusing on, otherwise carry on. 3. MetricsSome of the metrics I feel are important are:a) % of users creating stories and new users creating storiesb) % of followers viewing one or more storiesc) % of followers responding / reacting to a storyd) Retention among story creators (Story creators creating more stories)e) Average session time The metric I want to focus on is the % of people creating stories. If we can increase this, it will lead to increased adoption of stories, increased engagement, more connections and will drive Facebook's core mission. 4. List down user groupsa) Power users - these people are already well aware of how stories work. They create stories on an ongoing basis and engage frequently with their followers through this mediumb) People who view stories and want to get started with creating stories. They have been viewing stories of other people in their network and are encouraged to try doing so themselvesc) People who view stories, but haven't thought of creating a storyd) People who don't view stories but spend time on the feed I would like to focus on the 2nd category of users, i.e. those who view stories but haven't created a story themselves 5. Some of their use cases and pain points are the following:a) They may not now how to create a storyb) They may think it's cumbersome to create a storyc) They may not feel comfortable sharing stories with everyoned) They may be short of ideas for content in the storye) They may not be aware of the benefits of creating stories I will prioritize these use cases in the following manner (from highest to lowest)c, d, e, a, b 6. My solutionsa. Allow a user to select whom should be able to view your story. While creating a story, I should be able to select which group of users this should go out to. If I want, I should be able to remove particular users from viewing the story. Maybe, I only want to share this story with my family or with my friends or with my work colleaguesb. I will use data to predict when one is likely to create a story. If someone has been tagged in specific types of events, ex. are on vacation / playing a sport / eating out. Based on this, I will prompt a user to create a story around that particular situation through notifications scheduled dynamically (some sort of processing will be required to not send too many)c. I will create collateral highlighting some of the benefits of creating stories- ex. reach out to people in a personal and fun way, undivided & uncluttered attention and the number of people within my network who are sharing stories.d. I will use data to predict who are the kind of users most likely to create stories. Some of the parameters to be considered could include age, number of followers/connections, engagement on their FB posts (likes, comments), whether the user is creating stories in other social networks owned by Facebook. To these people, I will send out periodic emails or push notifications which will also highlight the benefits of creating stories along with call to actionse. I will create an interactive tutorial which shows how one can create a story in a stepwise manner- adding a picture, recording a video, adding titles, adding symbols or colors 7. Evaluation of solutionsa) Already is there in Facebook posts, should be easy to implement for Stories toob) Requires analysis of past data to see if stories created were preceded by any event, such as a vacation/trip, eating out, going out for drinks, playing a sport. Based on this, we will need to model an AI solution. There will be significant user research, planning, A/B testing, UI/UX, engineering required for this. Also, too many notifications will annoy the user so we may need background jobs to send these dynamicallyc) Easy to do as it involves creating of material and distributing them at the right placesd) Requires analysis of past data to see which kind of users are creating stories the most. There will be significant user research, planning, A/B testing, UI/UX, engineering required for this. Also, too many notifications will annoy the user so we may need background jobs to send these dynamicallye) UI/UX and design intensive, how it seamlessly fits in with stories, should not affect experience of others who are well versed with storiesMy prioritization for these solutions from highest to lowest is a, c, e, d, b 8. Measuring performance of solutionsa) % of people using this feature out of all people creating storiesb) Conversion rate: % of people engaging (click through) with the notification, % of people creating stories after receiving this notificationc) % of people who have never created a story who viewed this collaterald) Conversion rate: % of people engaging with the notification, % of people creating stories after receiving this notificatione) % of first time story creators who viewed the entire tutorial, % of first time users who created a story with the assistance of the tutorial 9. To summarize, we discussed Facebook stories and various segments of users and their engagement with Facebook stories. We focused on users who view stories and know about it, however have not yet created any stories. We looked into the pain points of these users such and thought about solutions which could initiate them into creating stories

We've introduced this particular new feature (it was something about friend recommendations). How should we determine whether to keep it or not?

Number of friend requests sent through friend recommendations. Percentage of friend recommendations that had mutual friends

How would you test the success of a new feature added to Messenger?

Number of messages. Conversation views Performance metrics. Response rates.

We have a product that is getting used differently by two different groups. What is your hypothesis about why and how would you go about testing it?

Segment the users to understand what exactly is the difference between the two groups: □ Device (mobile/tablet/laptop) □ Platform (iOS/Android/others) □ Demographics (age/gender/kids or not/marital status/relationship status/income level/education level/job title/workplace etc.) □ Location □ Interests (hobbies, fav. Entertainment, organic food, action movies etc.) □ Behaviors (purchase behavior (ex- recently purchased shoes), device usage etc.) Once we have a clear understanding of the differences between the two groups, then we can come up with an explanation for why they use the product differently.

What to use for prioritization?

RICE Framework: To structure your answers, we recommend using the RICE framework. Reach : how many customers this product or feature will affect in a given time period Impact : the degree to which this product or feature will contribute to your goal Confidence : how sure you are about the values you've chosen in this calculation Effort : the overall time your team will invest in the project

What would you add to Facebook and how would you pitch it and measure its success.

To help Facebook retain and engage more users of a younger age, they can: - allow users to customize their feed based on their content preferences (like TikTok) , less family related things. Measure Success: number of users<18 who make an account - Build a better engagement model to allow users to interact with public figures. Mini Consultations. Measure success: Visitation no. Engagement with feed

How would you grow Facebook Birthdays?

So that we're on the same page, when you say Birthdays I think of it in two ways. The first is how a user adds and manages their birthday on their profile and the platform at large. The second is how a user can see their friends birthdays and interact with them by writing on their wall or sending them a message on their special day. I'll assume we want to focus on the ladder part. In terms of 'improvement', do we want to look at a specific platform? Do we want to increase the number of users? Or would you like me to define my own measure of improvement. Ok, sure. When I think about what we might want to improve about the birthdays feature, I first think back to why the Birthday's feature exists. I know Facebook's mission is to connect the world and bring it closer together. Every human on earth has a birthday, and I see that as something that inherently connects us and makes us relatable to one another. The feature fits neatly with Facebook's mission statement and since birthdays happen each year, for every user, I imagine Facebook originally launched this feature to grow engagement. The more users it can get to come back onto the platform, the more opportunity it has to drive advertising revenue maximizing Facebook's value as an advertising platform. That being said, I'd like to improve engagement with the Birthday's feature. I think it's best if we define engagement more specifically so we can measure the success of our improvements. If we think of the birthday feature as an engagement loop we can begin to identify what metrics we care about.1. User makes their birthday visible to friend2. User's birthday arrives3. Friend made aware of user's birthday4. Friend engages with the birthday user5. The birthday user engages with the friend6. Step 2 happens again For step one, I think most users birthday audiences are set to friends by default so we shouldn't focus our time there, and unless time stops we're also ok with not focusing on step 2. I think the feature today already does a decent job of making users aware of users birthdays with notifications etc. I think where we can have the most impact is improving engagement for 4 because many users that are aware of a friends birthday may not engage. A good engagement metric could be to look at the % of users that post on at least one friends wall each day. When I think about who uses this feature I think about a user's relationships to the birthday user. This may be a friend or acquaintance.Friends-These are family members, best friends or people that a user may interact with day to day in real life.-User may know more about this persons non internet life-Users that may be interacted with more on facebook than other users Acquaintances-User doesn't know much about this person outside of what they share on facebook-User may have forgotten how they met-User is not familiar with this person I'd like to focus on the second group because I believe these are the users that 1) use the feature the least, and 2) represent a larger amount of users thus improving our chances to make an impact. Some common pain points for acquaintances include:a) having too many users to wish happy birthday tob) not remembering who the users arec) not knowing what to say to usersd) not wanting to come across as weirde) not caring enough to wish happy birthdayf) not seeing any value from wishing the user happy birthday I think if we prioritize these by most important to the user we get a list of pain points in this order: b, c, d, a. I'd like to come up with some solutions for the top use cases above.b)1. Show connection info: surface the date, location, and the connection type (ie college) the connection happened. This will allow users to get a quick refresh of how the connection happened and what it means.2. Show mutual friends: surface a few names of mutual friends so the user can again get a quick refresh on what the connection means. c)1. Show last communication: the last interaction you made with the user. This gives the user an opportunity to think back to what they might've talked about with this person in the past2. Show mutual interests: this gives the user topics of interests to feed off of when creating wish3. Prefill post: prefill the post with common saying like 'Happy Birthday'4. Smart prefills: Look across mutual interest categories to add context to the wish. Example, "{The Lakers lost last night} but I hope you have a great birthday!" d)1. Show total birthday wishes count so far: show the number of other people who have already wished this person happy birthday. This will help to show that's its a normal thing activity to partake in. a)1. Recommended birthday list: show a subset of users in a recommended birthday area apart from the other users. This list would be comprised of people that you've engaged with in the past year but don't fall in the 'Friends' category.2. Wish all friends a happy birthday at once with one click. I'd like to evaluate these solutions in terms of impact to cost ratio and pick the ones that will help us achieve our objective to increase engagement. b1: Low hanging fruit: Low Cost, Low Impact. While easy to do, I think surfacing this much info may distract the user and isn't the quickest way for the user to process the connection.b2: Quick win: Low Cost, High Impact. I think showing other faces is easier to process for the user and similar in cost to implement as the other data points. c1: Money Pit: Low Impact, High Cost. I think some interactions will be so old that showing them wouldn't really help. I think how this is presented as well may be a challenge and time consuming addition.c2: Quick Win: Low cost, high impact. This would go a long way for sparking the conversation.c3: Quick Win: Low cost, high impact. These phrases would be generic and easy to prefil, but take the time and thinking out of the process., but the more there used the more value that is lost for each post.c4: Must have: High Impact, High cost. This would provide the type of personalization for each connection that could motivate the birthday user to respond. d1: Quick Win: High impact, low cost. This total would provide the social proof and make it 'ok' for users to post and not feel weird. a1: Must Have: High impact, high cost. This would really help users to focus on what's important but if the algorithm to create this list isn't right, it may be excluding important users.a2: Low hanging fruit: Low impact, low cost. Easy to post to multiple walls at once, but this would again reduce the value for each happy birthday post because users receiving the wishes will know it's been automated to some extent. Moving forward, I build solutions b2, c2, c4, d1, and a1.

How would you explain a confidence interval to a non-technical audience?

Take yourself back to high school. You were out sick one day. That day you missed a massive food fight. You show up to school the next day and you want to know who or what caused the fight. Now imagine two scenarios: A) you talk to 100 people and their stories are pretty much aligned. It was Bill who started it B) you talk to 10 people and each one identifies a different person There is less certainty in situation B. This is clear. This is the same thing with confidence intervals. When we estimate the features of a population (a person's entire bloodstream, the nation's voting habits, e.g.) by using a sample (a blood test, a survey, e.g.) our confidence in that estimate is a function of sample size and standard deviation (i.e. how much the information changes/varies)

how to grow the number of people who complete their profile on facebook?

To drive completion, we can i) show stats about how completing their profile wiill increase response from other users b) let them know the privacy policies, etc.

"How do you calculate monthly active users, churned users and resurrected users from a user activity log with userID and DateTime "

To get the number of churned users, we group by 'userID', then sort by time_elapsed, then apply .diff() with a period of 1 to get differences between rows. Finally, we take .max() over this new column to get the maximum time a user hasn't used FB. If that time is greater than a month, the user is churned. We filter over rows with time_elapsed greater than a month and count these users. To get resurrected users, we use our table of churned users and find users who have been active in the last month.

What is your favorite Facebook product?

Whatsapp - As an international student, Egypt heavily rely on Whatsapp. Text costs a lot of money. Photos, background of messages, etc etc My Favorite product is : WhatsApp. WhatsApp is a messaging app available freely for android and ios users. It allows user to send text and voice messages or video calls . It's one of the early products invented in 2009 which solved the problem of communication via internet. Before that people used to communicate via sms. Why I like : Useful : It's very useful , user can communicate irrespective of region or country . One doesn't need a ISD connection to be able to connect to friends or family in other countries. Innovative : WhatsApp has continuously came up with innovative features to engage and acquire new users. Some of them are Enabling users to record a voice message and share. This has impacted huge customer base who are not very good in typing like users in age group of 45 to 60 and belonging to tier 3 cities.WhatsApp web: Connecting your laptop with mobile phone via scanning a QR code is very innovative. IT gives the end user the flexibility to use WhatsApp through laptop/desktop. It comes with some limitations though.Fingerprint Authentication : Adds another level of security to the app. With this feature enabled, even if you leave your phone unlocked, your chats cannot be accessed by anyone else, until it is verified by your fingerprint. Efficient : The best thing I like about WhatsApp is the ease of usage. Users need not learn new representations for each task - setting up a group chat in WhatsApp is aligned with adding new members to the existing chat . In application tutorial is not required to educate users.It comes natural to them. Things to improve Target Metrics to improve : Engagement I am considering the engagement as metrics to improve and below mentioned features are suggested taking this metric in to consideration Profile Photo Views : If user can see how many people viewed their profile pic , it gives appreciation to users. Currently one can comment on other's DP but cannot really know how many people viewed it. Bookmark Conversations : As a user I can bookmark the conversation in individual or group chat and can reference later. Improving Whats App groups : There are few improvements specifically for groups , Firstly while forwarding a message to several groups, WhatsApp should automatically disable those groups that already have that message .Secondly providing group based password protection

How to migrate Facebook users to Instagram

You can encourage Facebook Fans to follow you on Instagram through an incentive You can lead Facebook Fans to follow you on Instagram through a link You can engage Facebook Fans to follow you on Instagram through a contest You can also attract Facebook Fans to follow you on Instagram by giving them a sneak peak