Skip to main content

Incrementality Testing in Triple Whale

Find out whether your marketing created additional sales, then use that evidence to make better budget decisions.

Y
Written by Yirmi Rubin

Plan requirement: Incrementality is available on Triple Whale’s Enterprise plan.

What incrementality tells you

Incrementality helps you answer a simple question:

Did your marketing create additional sales, or would those sales have happened anyway?

For example, imagine that a campaign generated 100 sales. That number alone does not tell you whether the campaign caused all 100. Some customers may have purchased without seeing the campaign.

An incrementality test compares a group that received the advertising with a similar group that did not. If the advertised group generated 100 sales and the comparison suggests that 80 would have happened anyway, the campaign created an estimated 20 additional sales.

Those 20 sales are the campaign’s incremental impact.

Incrementality can help you understand:

  • Whether a campaign created new demand or captured demand that already existed

  • How many additional sales or acquisitions your marketing generated

  • Whether the incremental return justified the investment

  • Whether you should increase, maintain, reduce, or rethink the activity

How incrementality works with attribution and MMM

Incrementality is one part of your measurement strategy. Attribution and marketing mix modeling provide other perspectives.

Method

The question it answers

When to use it

Attribution

Which marketing touchpoints received credit for a conversion?

Use it for daily campaign analysis and understanding customer journeys.

Marketing mix modeling (MMM)

How did marketing and other business factors contribute to performance over time?

Use it for broader planning and budget decisions across channels.

Incrementality testing

What happened because the tested marketing activity ran?

Use it when you need to measure the additional impact of a campaign, channel, or tactic.

Attribution helps you understand where conversions received credit. Incrementality helps you determine whether the marketing caused additional conversions.

These methods work best together. Attribution supports day-to-day optimization, MMM supports broader planning, and incrementality gives you a controlled way to validate impact.

Experiment types in Triple Whale

Triple Whale offers two ways to run an incrementality test. The right option depends on where you advertise and the question you want to answer.

GeoLift

GeoLift measures the impact of advertising across different geographic areas.

Some locations receive the advertising change, while similar locations provide a comparison. Triple Whale uses the comparison to estimate what would likely have happened without the change.

Use GeoLift when you want to measure the impact of a campaign or channel across geographic markets.

During setup, you can group locations by:

  • Geo Pods

  • State

Geo Pods is selected by default. Geo Pods are groups of locations designed to create a more balanced comparison.

Learn more about Geo Pods and how GeoLift tests work.

Meta Conversion Lift

Meta Conversion Lift measures the additional impact of your Meta ads.

Meta separates eligible people into two groups. One group can receive the ads included in the experiment, while the other does not. The difference between the groups helps estimate how many additional sales or conversions the advertising created.

Use Meta Conversion Lift when you want to test the impact of Meta advertising directly.

Your Meta ad account and selected campaigns must meet Meta’s eligibility requirements before an experiment can run.

Before you create an experiment

A useful experiment begins with a clear question. Decide what you want to test and what you may do differently once you have the answer.

Before setup:

  • Decide which campaigns you want to test.

  • Choose whether you want to measure revenue or acquisitions.

  • Confirm that the campaigns are eligible for the selected experiment type.

  • Make sure the experiment has enough budget and time to run as planned.

  • Avoid scheduling other major campaign or business changes that could interfere with the result.

  • Plan to keep the experiment setup in place for the full test.

For a GeoLift experiment, choose the primary metric that matches your question:

  • Select Revenue to measure the additional revenue created by the tested marketing activity.

  • Select Acquisitions to measure additional conversions.

Create a GeoLift experiment

From the Incrementality page, start a new experiment and choose GeoLift. The setup flow contains six steps.

To create a GeoLift experiment:

  1. From the left-hand menu, select Marketing Acquisition.

  2. Select Incrementality.

  3. Select Create New Test.

  4. On the Test Design screen, select GeoLift.

  5. Choose whether you want to test active campaigns or campaigns you plan to launch or reactivate.

  6. Select Next.

GeoLift is selected by default when the Test Design screen opens. Active Campaigns is also selected by default.

The GeoLift setup contains six steps:

  1. Test Design

  2. Campaigns

  3. Holdout Setup

  4. Budget

  5. Scheduling

  6. Review & Launch

You can select Save Draft during setup if you are not ready to complete and launch the test.

1. Test Design

Choose the testing method and the type of campaigns you want to test.

Select GeoLift to measure marketing impact by comparing geographic regions that receive the advertising with similar regions that do not.

Then choose one of the following campaign options:

  • Active Campaigns: Test campaigns that are currently running.

  • New or Paused Campaigns: Test campaigns you plan to launch or reactivate.

Select Next to continue.

2. Campaigns

Configure the primary metric, regional granularity, and advertising activity included in the experiment.

Choose a primary metric

Select the result you want to measure:

  • Revenue

  • Acquisitions

Revenue is selected by default.

Choose the regional granularity

Select how locations should be grouped:

  • Geo Pods

  • State

Geo Pods is selected by default.

Select the test scope

Choose the channels, tactics, campaigns, or ad sets you want to include in the experiment.

You can browse the available advertising activity using:

  • Custom Categories

  • Source

Use Custom Categories to select from your organized campaign groups, or use Source to browse activity by advertising platform.

Review the Total Daily Spend shown for your selections. When you are ready, select Next.

3. Holdout Setup

Review the geographic design recommended for the experiment.

One group will receive the advertising change. The other group provides the comparison used to estimate incremental impact.

Review the proposed regions and holdout percentage before continuing.

Changing the geographic groups while the experiment is running can make the result harder to trust.

4. Budget

Enter the planned duration and budget for the experiment.

Review the experiment guidance shown during setup. If Triple Whale warns that the planned experiment may not be sufficiently powered, adjust the budget, duration, or geographic design before launching.

5. Scheduling

Add a name and set the timeline for your GeoLift experiment.

Name the experiment

Enter a clear Experiment Name that will help your team identify the test later.

Review the start date

Confirm the Start Date shown for the experiment.

Set the duration

Enter the number of days the experiment should run. Triple Whale recommends a minimum duration of 21 days.

The calendar displays the experiment’s start and end dates based on the selected duration.

Select the cooldown period

The Cooldown is the period after the experiment ends when results are allowed to stabilize before they are analyzed.

Review the recommended cooldown period and use the calendar to confirm the complete timeline. The calendar displays the active experiment period and cooldown period in different colors.

When the schedule is correct, select Next. If you are not ready to continue, select Save Draft.

6. Review & Launch

Before launching, review:

  • Selected campaigns

  • Geographic groups

  • Primary metric

  • Budget

  • Experiment dates

  • Estimated cost

If something is incorrect, return to the relevant step and update it before launch.

For supported GeoLift experiments, Triple Whale applies the geographic targeting changes needed to create the test and comparison groups.

Create a Meta Conversion Lift experiment

To create a Meta Conversion Lift experiment:

  1. From the left-hand menu, select Marketing Acquisition.

  2. Select Incrementality.

  3. Select Create New Test.

  4. On the Test Design screen, select Meta Conversion Lift.

  5. Select Next.

Complete the following fields:

  • Experiment Name

  • Meta Ad Account

  • Campaign(s)

  • Meta Pixel

Review & Launch

Select the experiment configuration that best matches your testing goals.

Configuration

Holdout

Duration

Best for

Optimize for Speed

30%

14 days

Reaching a result sooner with a larger holdout group

Balanced Approach

20%

21 days

Balancing test speed with the size of the holdout group

Minimize Revenue Impact

10%

28 days

Limiting the holdout group while running a longer test

Optimize for Speed is selected by default.

The holdout group is the percentage of the audience that will not see the campaigns included in the experiment.

A larger holdout can help the experiment reach a result sooner, but it also prevents more eligible people from seeing the tested campaigns. A smaller holdout limits that impact but requires a longer test.

Review the experiment parameters

When you select a recommended configuration, Triple Whale automatically updates:

  • Holdout Group (%): The percentage of the audience that will not see the tested campaigns.

  • Test Duration (days): The number of days the experiment will run with reduced campaign reach.

Confirm that the holdout percentage and test duration match your testing goals before launching.

Optional: Use a manual configuration

To enter a custom holdout percentage or test duration, turn on Manual Configuration.

Triple Whale displays a warning that custom settings may affect test reliability. Review your custom settings carefully before continuing.

When the experiment settings are correct, select Launch.

If Meta rejects the experiment, review the displayed error and correct the underlying account, campaign, permission, or eligibility issue before trying again.

Monitor your experiment

After launch, use the Incrementality table to monitor the experiment’s status, Data Health, and spend delivery.

Avoid making changes based on one unusual day. Look for a sustained warning or a meaningful difference from the experiment plan.

The table can show the following statuses:

  • Draft

  • Running

  • Cooldown

  • Under Review

  • Completed

When an experiment is Under Review, it remains visible, but its complete results are not ready.

Data Health

Data Health helps you identify whether an experiment is progressing as planned.

Possible statuses include:

  • On Track

  • Needs Attention

  • At Risk

  • No spend data available

If an experiment needs attention or is at risk, investigate the issue before relying on its result. Missing data or uneven delivery can weaken the comparison and prevent the experiment from producing a useful answer.

Spend Test Tracker

Use the Spend Test Tracker to compare actual delivery with the experiment plan.

The tracker can show:

  • Cumulative Spend vs. Target

  • Projected % of Target

  • A pacing status

  • Daily Spend vs. Threshold

GeoLift spend pacing uses the following indicators:

  • Green: At least 100% of the planned pace

  • Yellow: 80% to 99% of the planned pace

  • Red: Below 80% of the planned pace

Focus on sustained pacing problems rather than normal changes from one day to the next.

If spend is on pace but delivery is low, review:

  • Bids

  • Audience size

  • Frequency limits

If spend and delivery are both below plan, review:

  • Campaign budget

  • Daily spending caps

When results become available

GeoLift results are organized into:

  • Results Overview

  • Deep Dive

  • Configuration

Revenue results

For a revenue-based experiment, Results Overview can include:

  • Marketing Contribution: The estimated additional revenue created by the marketing you tested.

  • Revenue Lift: The estimated percentage increase or decrease in revenue caused by the tested marketing.

  • iROAS: Incremental return on ad spend, calculated using the incremental revenue and spend included in the experiment analysis.

  • Probability of Direction: How likely it is that the marketing had a positive or negative effect.

Acquisition results

For an acquisition-based experiment, Results Overview can include:

  • Incremental Conversions: The estimated number of additional conversions created by the marketing you tested.

  • Acquisition Lift: The estimated percentage increase or decrease in acquisitions caused by the tested marketing.

  • iCPA: The experiment spend compared with the estimated number of additional acquisitions.

Understand the strength of the result

An incrementality result is an estimate, not a guarantee.

Review the Probability of Direction and the displayed result range before deciding what to do next.

Probability of Direction indicates how strongly the analysis supports a positive or negative effect. A higher probability provides stronger evidence for the displayed direction.

The result range shows the uncertainty around the estimate.

A narrow range means the likely outcomes are closer together. A wide range means there is more uncertainty.

If the range includes both a meaningful positive outcome and a meaningful negative outcome, the experiment may not provide enough evidence to confidently increase or reduce spend.

Compare GeoLift with attribution

The Deep Dive compares the GeoLift result with available attribution models, including:

  • First Click

  • Last Click

  • Linear All

  • Linear Paid

  • Triple Attribution

  • Triple Attribution + Views

Use this comparison to understand whether attribution and GeoLift tell a similar story.

Different results do not automatically mean that an attribution model or experiment is incorrect. Attribution assigns credit for conversions, while GeoLift estimates the additional impact caused by the marketing.

For a deeper results walkthrough, see Reading Incrementality Test Results.

Read Meta Conversion Lift results

Meta Conversion Lift results show the estimated additional impact of the tested campaigns and the confidence in that estimate.

Review both the estimated impact and its confidence before making a decision. A large estimated result with low confidence may look promising, but it does not provide strong enough evidence for a major budget change on its own.

For more information, see Meta Conversion Lift Experiment.

Turn your result into a decision

Apply the result only to the campaigns, markets, audiences, and dates included in the experiment. Do not assume that one result represents every campaign or all activity on a channel.

Positive result

A reliable positive result indicates that the tested marketing created additional value.

Consider increasing spend gradually rather than making one large change. Continue monitoring performance because results may change as you scale.

Negative result

A negative result suggests that the tested activity may not have created additional value during the experiment.

Before reducing spend, review:

  • Data Health

  • Spend delivery

  • The balance between the test groups

  • Probability of Direction

  • The result range

An uncertain negative result is not the same as clear evidence that the advertising reduced performance.

Inconclusive result

An inconclusive result does not mean that the campaign had no impact. It means the experiment did not produce a clear enough answer.

Before repeating the experiment, review whether it had enough:

  • Budget

  • Time

  • Conversion volume

  • Separation between the test and comparison groups

Compare the result with attribution

If attribution looks strong but incrementality is weak, the channel may be receiving credit for demand that already existed or that another activity created.

If incrementality is strong but attributed performance looks weak, your attribution model may be missing part of the channel’s contribution.

Use both perspectives to guide planning, but keep the experiment’s scope in mind.

Troubleshooting

If an experiment cannot launch, falls behind plan, or does not produce a useful result, check:

  • Campaign eligibility and status

  • Account and data-source permissions

  • Primary metric and conversion configuration

  • Budget and spend delivery

  • Test and holdout integrity

  • Overlapping campaigns or major business changes

  • Experiment dates and measurement window

  • Any error displayed during launch or monitoring

For a GeoLift experiment:

  • If spend is on pace but delivery is low, review bids, audience size, and frequency limits.

  • If spend and delivery are both low, review the campaign budget and daily spending caps.

  • If Meta or Google cannot resolve a geographic target, confirm that the location is supported and formatted correctly.

  • If the final result has a wide range, review the budget, duration, geographic balance, and experiment design before repeating the test.

Avoid changing geographic exclusions, budgets, comparison groups, or experiment settings while an experiment is running. Unplanned changes can weaken the comparison and make the final result harder to trust.

Did this answer your question?