Two Choices
Configuring an experiment means answering two separate questions.
Algorithm
Thompson Sampling optimises one global winner. Contextual Linear optimises a different winner per visitor. Pick contextual only if you actually send context with your assignment calls.
Memory
Standard counts every result forever. Discounted fades old results by a half-life. Sliding window drops anything older than the window. Both fading modes apply to either algorithm.
Quick Comparison
| Memory | Old results | Best For | Setting |
|---|---|---|---|
| Standard | Counted forever | A winner that will not change | None |
| Discounted | Fade by half-life | Gradual drift, creative fatigue | Half-life |
| Sliding window | Dropped past the edge | Abrupt change, seasonal swings | Window |
Thompson Sampling
A Bayesian algorithm that samples from the posterior distribution of each arm's reward probability.
How it works
Maintain a Beta(successes + 1, failures + 1) distribution for each arm. Sample a value from each arm's distribution and pick the arm with the highest sampled value. As more data comes in, the distributions narrow and the best arm wins more often, automatically balancing exploration and exploitation.
When to use
- Binary outcomes (click/no-click, convert/don't-convert)
- You want the theoretically optimal exploration/exploitation balance
- Any experiment where the same variant should win for everyone
Parameters
None -- adapts automatically from data.
Confidence formula
sample from Beta(successes + 1, trials - successes + 1)Confidence emerges naturally from the posterior distribution.
Distributions narrow with more data
Discounted
Old results fade smoothly instead of counting forever.
How it works
Every observation is weighted by 0.5 ^ (age / half-life), and those weighted counts are what feed the Beta posterior. A one-week half-life means a conversion from last week counts half as much as one from today, and one from a month ago counts about a sixteenth. Because the weights shrink the trial count as well as the success count, the posterior widens as evidence ages — so the bandit starts exploring again rather than just changing its mind quietly.
When to use
- The winner drifts gradually rather than flipping overnight
- Creative fatigues, or the audience keeps turning over
- You want responsiveness without shrinking your effective sample
Parameters
| Name | Type | Description |
|---|---|---|
| halfLifeMs | number | How long it takes a result to lose half its weight. Default: 7 days. |
A result's weight halves every half-life
Nothing is ever thrown away entirely — old evidence just stops outvoting new evidence.
Sliding Window
Only results from the last N days count. Everything older is dropped outright.
How it works
The posterior is rebuilt from the rewards recorded inside the window, at full weight, and nothing else. There is no tail: a result is worth exactly as much on the last day of the window as on the first, and nothing the day after. That makes the sliding window sharper than discounting when change is abrupt — a pricing change, a new season, a campaign switching off — at the cost of a smaller effective sample and noisier odds on low-traffic experiments.
When to use
- The winner changes abruptly rather than drifting
- You can name the period that matters ("the last two weeks")
- Traffic is high enough that a window still holds a decent sample
Parameters
| Name | Type | Description |
|---|---|---|
| windowMs | number | How far back results still count. Default: 14 days. |
Only results inside the window count
Contextual Bandit (Linear UCB)
Uses user context (device, location, behavior) to personalize treatment selection.
How it works
Extracts numeric features from context. Maintains a linear model per arm. Uses UCB-style confidence bounds on the linear predictions. Arms are selected based on predicted reward + confidence bonus, personalized to each user's context. This means different users can receive different treatments based on who they are.
With memory modes
Discounting and sliding windows apply here too: each past observation enters the model scaled by its weight, so an aged observation shrinks its own contribution to the model's certainty as well as to its prediction. The confidence ellipsoid widens as evidence ages, and the model re-explores instead of coasting on a fit it built months ago.
When to use
- Different users respond to different treatments
- You have useful contextual signals (device type, location, time of day)
- You want personalization, not just global optimization
Parameters
| Name | Type | Description |
|---|---|---|
| context | object | Contextual features passed at assignment time (e.g. device, location, time of day). |
Confidence formula
context-dependent linear UCB boundConfidence depends on both the linear model fit and the user's context features.
Context features inform arm selection
Different users get different predictions based on their context.
Choosing Memory
The question to ask is not “which algorithm is best” but “is my winner going to stay my winner?”
Just starting?
Use Thompson Sampling on Standard memory. It needs no tuning and is the right answer for a test you intend to conclude.
Running it forever?
Switch to Discounted. An always-on experiment on standard memory eventually stops learning: after enough traffic no plausible run of new results can move a posterior built from millions of old ones. A half-life keeps it alive.
Change arrives all at once?
Use a Sliding window sized to the period that matters. Better than discounting when the past is not merely less relevant but actively wrong — after a redesign, a price change, or a seasonal turn.
Have user context?
Use Contextual Linear — and pair it with whichever memory mode fits, since the two settings are independent. Best when different users genuinely respond to different treatments.