Threshold Experiment

Threshold Experiment runs a live A/B between your current thresholds and a proposed set. Where the Policy Simulator replays historical data, Experiment shadow-scoresincoming returns through both configurations and compares them in real time.

How it works

1. Set up an experiment with proposed LOW / MEDIUM / HIGH thresholds and a stop condition (target return count or maximum days).
2. Hit Start. The experiment runs in the background — every new return is scored once with current thresholds and once with the experimental thresholds. Customers and automations only see the current zone; the experimental classification is recorded but never acted on.
3. When the stop condition is hit, the experiment auto-concludes and you get the report.

The report

• Status banner— Running / Concluded / Applied / Discarded.
• Current vs experimental threshold cards— Side-by-side numbers.
• Classification comparison chart— The distribution shift between the two configurations.
• Differently classified table— Every return that landed in a different zone under the experimental thresholds.
• Impact projection— Estimated change in Pending Actions, dollar value at risk, and workflow trigger volume.

Apply or discard

A concluded experiment can be Applied (the experimental thresholds become live) or Discarded (the report is kept for the history table, but the live config is unchanged). Past experiments are listed at the bottom for reference.

Simulator vs Experiment

Use the Simulatorwhen you want a fast, one-shot answer using last month’s data. Use Experiment when you need confidence on futuretraffic — especially around seasonal patterns or after a product launch where the historical mix isn’t representative.

Next steps

• Threshold tuning guide— what good numbers look like.
• Chargebacks— the empirical feedback loop for any threshold change.