A/B Testing

Note: We calculate your A/B test results using the _______________________ A window you've specified in your account. If you later change your attribution window, you may see a message in the Experiments tool explaining any possible discrepancies, but your previous test results will not change.

Attribution

Additionally, you can create an A/B Test when: Duplicating an existing _________________ C Duplicating an existing ___________ A __________ S or _________ A When creating a ______________ N ad campaign. Directly from the __________________________ E tool. From a common scenarios ________________ P.

Campaign Ad Set Ad New Experiments Prompt

A/B Test examples with a placements variable If you're not already using aut________________________ placements, you can run an A/B test to see how aut________________________ placements performs with your current business-as-usual strategy. Compare Fee________________________-only placements with Stor________________________-only placements to see how both perform when reaching your audience with a creative that works for both.

If you're not already using automatic placements, you can run an A/B test to see how automatic placements performs with your current business-as-usual strategy. Compare Feed-only placements with Stories-only placements to see how both perform when reaching your audience with a creative that works for both.

A/B Test examples with a delivery optimization variable CbovsBau

If you're not already using campaign budget optimization, you can run a campaign level optimization test if you're trying to understand how it affects delivery and performance for your ads compared to business-as-usual.

Your test can start showing results once there's at least _____________ events observed for each strategy you're testing. However, we recommend waiting until your test is completed and results are finalized to judge your test findings and act on them.

100

After choosing the variable you want to test, we'll divide your _______________ B to equally and ________________________ R split exposure between each version of your creative, audience, or placement.

Budget Randomly

The winning campaign is determined by comparing the ___________________________ CP of each campaign, where the result is the ___________________E you selected during test creation, such as cost per click, cost per purchase, or cost per incremental purchase.

CPR Event

Here are a few ways you can continue testing to improve your advertising strategy. It's important to note that these are only recommendations and testing strategies will vary by advertiser and vertical. After testing audience, creatives, placements, or delivery optimizations: Run the same test with a different ____________________ C objective. Test different ad _______________ C with the winning audience. Test different ________________ P with the winning audience. Test different creatives in the same ______________ F against each other. Test different ______________ A of ad creative. Test different ____________________ A on your winning placement. Test with the same conversion _______________ W. Test with the same optimization _______________ E.

Campaign Creative Placements Format Aspects Audiences Windows Event

A/B Test examples with an audience variable are: Compare how a cust_____________________audience of recent website visitors and looka_____________________audience of likely visitors perform when trying to generate leads or purchases. Setting up audi_____________________with similar inter_____________________, but different ag_____________________ groups to compare performance by demogr_____________________

Compare how a custom audience of recent website visitors and lookalike audience of likely visitors perform when trying to generate leads or purchases. Setting up audiences with similar interests, but different age groups to compare performance by demographic.

A/B Test examples with a creative variable are: Comparing a slid_______________ video with a car_______________ of imag_______________ promoting your products or services. Choosing the same ass_______________ (such as a video or carousel of images) but trying different ad te_______________in each version. Using different ima_______________ of different prod_______________ or serv_______________in each ad ver_______________ to see which products perform best when shown in your ads.

Comparing a slideshow video with a carousel of images promoting your products or services. Choosing the same asset (such as a video or carousel of images) but trying different ad text in each version. Using different images of different products or services in each ad version to see which products perform best when shown in your ads.

The kinds of variables available to change in an A/B test include: Crea__________ Aud__________ Deliv__________ Opti__________ Plac__________ Pro__________ S__________ M__________ th__________on__________

Creative. Comparing different ad images, text, or creative types. For example, you might have a video with an image carousel, or different ad text with the same image or video. Creative variables are managed at the ad level in Ads Manager. Audience. Comparing how effective your ads are at reaching different audiences or demographics. For example, you might add or remove certain regions for your audience, or compare a custom audience with an interest-based audience. Audience variables are managed at the ad set level in Ads Manager. Delivery Optimization. Comparing campaigns with and without Campaign Budget Optimization enabled to measure its impact on performance Placements. Comparing placement types when reaching your audience. For example, comparing automatic placements with specifically chosen ad placements. Product Set. Comparing different product sets as configured in your Ads Manager assets. For example, choosing two different product sets to use when running product set ads on Facebook. Product set variables are managed at the ad set level in Ads Manager. More than one. Some A/B tests let you change multiple variables if you want to compare two complex strategies against each other on the same cost per result or cost per conversion lift basis. You can make changes to the campaign, ad set or ad when changing multiple variables.

Begin by reviewing the A/B testing best practices and guide for troubleshooting ad performance. If you've implemented the tips outlined in those articles and are still having problems: Ensure your audiences are _______________ L enough to avoid under delivery. Since we divide your audience for an A/B test, ad sets that are part of A/B tests may be more vulnerable to under-delivery caused by small audiences. You may need to broaden your audiences even more than usual to avoid under-delivery when running an A/B test. Increase your___________________. B Make sure your ad sets are __________________ D enough when testing audience, placement or delivery selection. If your ad sets are too similar, we may not be able to confidently declare a winner.

Large Budget Different

Here are 2 scenarios in which testing more than one variable may be useful: To compare the effectiveness of 2 different _______________ Pro in 2 different ___________________ Loc . In this case, you'll need different variables for your audience and ad creative. To compare 2 ________________ Aud and __________________ Plat simultaneously.

Products Locations Audiences Platforms

If you're comfortable using ____________________ Q creation in Ads Manager, you can use it to set up an A/B test that has more than one variable. This may be useful if you just want to know which of ___________ T specific ad sets is performing better. It may not be useful for helping to guide your overall advertising strategy, since you won't be able to isolate which variable is _____________ C improved performance.

Quick Two Causing

A/B testing can measure the performance of each strategy on a Cost Per __________________ R basis or Cost per ______________ Con ______________ Lif basis with a holdout.

Result Conversion Lift

We recommend A/B testing when you're trying to measure changes to your advertising or quickly compare _______________ T strategies. You should use A/B testing to learn new strategies rather than testing ____________________ I, such as by turning on and off ad sets or campaigns manually, since this can lead to inefficient delivery and unreliable results. For example, A/B testing helps ensure your audiences will be evenly split and statistically comparable, while informal testing can lead to _________________________ O Audiences

Two Informally overlapping

A/B Test examples with a product set variable $vs$

Using product set tests to fine tune your product set to best fit your audience Compare a product set with items that are less than $50 with a product set with items that are more than $50

A/B testing lets you change ______________________V, such as your ad creative, audience or placement, to determine which strategy performs best and improve future campaigns. For example, you might hypothesize that a custom audience strategy will outperform an interest-based audience strategy for your business. An A/B test lets you quickly compare both strategies to see which one performs best.

Variables

Once you've created an A/B test, you might want to make changes to it. If your A/B test is still In Review or Scheduled, then you can still make any changes you'd like to the campaign, ad set or ads in the test. To do this, edit your ad set or ad in Ads Manager just as you would in any other situation. However, once your A/B test is Running, you ________________ W be able to make any changes to the ________________ V you've chosen for your test. For example, if your A/B test uses a Creative , you won't be able to make any changes to the headline, description or other content in any of your test's ads, but you can still change the ad sets' audience and placements from Ads Manager.

Won't Variable