Incrementality Testing

Prove what's truly incremental in your marketing

Platform-reported ROAS counts conversions that would have happened anyway. TrustData runs geo holdouts, time holdouts, and platform lift tests to isolate true causal lift — and automatically recalibrates your budget optimizer with real iROAS.
14-day free trial
The incrementality gap

The problem with platform-reported ROAS

Every ad platform claims credit for every conversion it could plausibly have influenced. A Meta retargeting campaign reaches a customer who was already going to buy — and claims the ROAS. The real question is never "how many conversions do we see?" but "how many conversions would we have lost if we stopped spending?"
  • Non-incremental conversions
    Typically 20–60% of attributed conversions are non-incremental — they would have happened without the ad. Platform ROAS is inflated by exactly this share.
  • The calibration factor
    TrustData measures the ratio of your true incremental conversions to platform-claimed conversions. This calibration factor — typically 0.3–0.8 — corrects your optimizer's revenue model.
  • Why it changes budget decisions
    A Meta campaign with reported ROAS 4.0 and calibration factor 0.45 has a true iROAS of 1.8. That changes whether you scale, hold, or cut — and changes your optimizer's recommendations entirely.
Geo holdout

Geo holdout — the gold standard

Pause ads in a set of test regions while keeping them running in matched control regions. The conversion delta between test and control, adjusted for pre-period trends, is the true incremental lift.
  • Matched region control
    Test and control regions are selected to be as similar as possible in baseline conversion rates. TrustData uses your first-party regional conversion data from ClickHouse for matching.
  • Trend extrapolation counterfactual
    The counterfactual is built from pre-period trend extrapolation — not naive averaging. A permutation test with 1,000 shuffles of test/control labels provides the p-value.
  • Direct calibration output
    Result: lift_pct, iROAS, p_value, and calibration_factor = our_iROAS / platform_reported_ROAS. This factor is immediately applied to the budget optimizer.
Time holdout

Time holdout — simplest to run

Pause a channel entirely for 2–4 weeks. Measure the drop in your deduplicated first-party conversions. Compare against a Prophet-forecast counterfactual built on pre-period data.
  • No regional data required
    Time holdouts work with any first-party conversion data — no regional breakdown needed. Best for answering "is this channel incremental at all?"
  • Prophet counterfactual
    TrustData fits a Prophet time series model on your pre-period daily conversion data, forecasts what would have happened during the pause, and compares the forecast to actuals.
  • Seasonal robustness
    Prophet accounts for weekly seasonality and trend changes. Where Prophet isn't available, TrustData falls back to a linear extrapolation with z-test significance calculation.
Platform lift test

Platform lift test — least friction

Use Meta Conversion Lift or Google Conversion Lift to handle holdout randomization. TrustData measures outcomes from your deduplicated first-party data — not from what the platform reports — and calculates how much to discount their claimed lift.
  • Platform handles the holdout
    Meta and Google randomize audiences into test and control groups. You run the test as normal — TrustData reads your first-party results and compares against platform claims.
  • Calibration from first-party data
    calibration_factor = our_measured_lift / platform_claimed_lift. This ratio tells you how much to discount the platform's attribution claims for this audience type going forward.
  • No geo data required
    Platform lift tests work at the audience level — no regional conversion data needed. Best for e-commerce brands without regional tracking or for audience-level holdouts on Meta.
The feedback loop

Calibration feeds directly into the budget optimizer

Every completed incrementality test updates the iROAS calibration factor for that channel. The budget optimizer automatically uses the true incremental ROAS for recommendations — no manual input required.
  • Automatic calibration update
    When a test concludes, TrustData updates the CalibrationFactor for that channel and immediately triggers a response curve rebuild and budget optimization run.
  • Temporal decay model
    Calibration factors decay over time — 50% weight at 90 days, 25% at 180 days. The optimizer degrades gracefully toward uncalibrated rather than relying on stale data.
  • Re-test recommendations
    When a calibration factor becomes stale (>90 days), TrustData automatically creates a low-urgency recommendation to run a new test. Untested channels are flagged in the budget optimizer UI.

What's included

Every component of the incrementality testing engine.
Geo holdout
Regional pause test with trend extrapolation counterfactual and permutation significance testing. Requires regional first-party conversion data.
Time holdout
Full channel pause for 2–4 weeks. Prophet or linear counterfactual. Works with any daily first-party conversion time series.
Platform lift test
Reads Meta/Google test and control group data. Compares against your first-party deduplicated outcomes. calibration_factor = our_lift / platform_lift.
Calibration factors
Per-channel iROAS calibration stored with confidence score and measurement date. Applied automatically to response curve models.
Temporal decay
Calibration weight decays by 50% every 90 days. The optimizer transitions smoothly from calibrated to uncalibrated rather than hard-switching.
ROI waterfall
ROI 1 (reported revenue vs spend) → ROI 2 (gross profit) → ROI 3 (net of non-working costs) → ROI 4 (incremental net profit with calibration). Stops at the highest level with available data.
Stale calibration alerts
Weekly check for calibration factors older than 90 days. Automatically creates re-test recommendation cards. Untested channels flagged in the budget optimizer.
Optimizer integration
Concluded tests trigger automatic response curve rebuild + budget optimization run. iROAS-calibrated channels get materially different budget recommendations than uncalibrated ones.

Frequently Asked Questions

Find out what's actually driving your conversions

14-day free trial. Run your first incrementality test and calibrate your budget optimizer with real iROAS.
14-day free trialView pricing