Audiences: the groups that buy differently, as rules you can read

Is it the headline, or is it who saw it?

A test result is an average. Your gift headline might do nothing overall and still do well with people buying for someone else, while doing worse with people buying for themselves. The average hides both.

The usual fix is a segment report. You slice by device, then source, then country, and you get dozens of rows. Some of them look exciting by chance alone. Most tools leave you to guess which.

Bandit's Audiences page does the slicing for you and only keeps a group when it reacts to your versions differently from everyone else. Each group is a rule you can read, with its counts.

What an audience is

An audience is one or two key = value terms from the context you send with each request, joined by "and". For example:

• device: mobile and source: google
• buying_for: a gift
• returning: true

The rules form an ordered list. Each visitor belongs to the first rule they match, or to "everyone else". Groups never overlap, so the visitor counts on the page add up.

For each audience the page lists the versions that group converts on better or worse than all other visitors, with the counts behind each line. A treemap at the top sizes every group by visitors. Blue marks a group that over-converts on some version, red one that under-converts, and gray one with no signal.

How Bandit finds them

Discovery runs every day at 04:00 UTC. An admin can also choose Recompute on the page, at most once an hour. Each run does this:

1. Read the last 30 days. Bandit reads your most recent 150,000 impressions in that window, newest first. A visitor is your identified user id if you sent one, otherwise the anonymous id. Each impression counts as served, and as converted if a reward arrived for it.
2. Pick the keys worth mining. A context key qualifies when at least 20% of visitors send it. Values are trimmed and lower-cased. Values over 40 characters and numbers are ignored. A key with more than 12 values left is treated as an id and dropped. Bandit keeps the 8 most useful keys.
3. Search for groups. Candidates are single traits, and pairs of traits on different keys. A candidate needs at least max(30, 3% of visitors) visitors. With 4,000 visitors that is 120.
4. Score against everyone else. Each candidate is compared with all other visitors, version by version, counting only versions it was served at least 30 times. The best candidate becomes an audience, its visitors are set aside, and the search runs again on the rest.
5. Stop. Discovery stops at 8 audiences, or earlier when no candidate passes the signal test below.

A group that converts like everyone else scores zero. It never becomes an audience, however large it is. A tight group also beats a broader trait that contains it, because the visitors who react the same as everyone else dilute the broader trait's score.

What counts as a signal

For each audience and version, Bandit asks how likely it is that the group's conversion rate is higher than everyone else's on that version. It keeps the line when that probability is at least 90%, or at most 10%, and the group was served the version at least 30 times.

Say 150 mobile Google visitors saw headline B and 12 converted, while 1,850 other visitors saw it and 74 converted:

The chance that the group's rate is higher comes out at about 97%, so headline B shows as a signal for that group. With 9 conversions instead of 12, the same calculation gives about 89%, just under the line, and the page shows nothing.

Labels and location describe a group

Two kinds of data describe who is in a group but are never used to build one.

Survey answers. Ask a visitor one question, like whether they are buying a gift, and record the answer with client.label. You can send up to 20 answers per call. On each audience card, a question with at least 5 answers gets a chart and a headline such as "Mostly founder (62% of 48 answers)", compared with the share across all visitors. Fewer than 5 answers and the card says there are too few to describe the group.

Location. Bandit stores the country and region Cloudflare reports when it serves a version. The page draws a US state map for all visitors and for each group, plus a country list when some traffic comes from outside the US.

If you want an answer to define groups, send it as context with the request instead. Context is mined. Labels are not.

Limits

• You need 100 visitors in 30 days. Below that, a run finishes with no audiences and the page shows the count and the minimum.
• Numbers are ignored. Cart value and similar numeric context can't form a group yet.
• Ranking is by evidence, not size of effect. A large group with a modest lift can rank above a small group with a big one.
• A single trait can be broader than its slice. When no pair describes the leftover visitors exactly, a rule like device: mobile can also match visitors an earlier group already took. The counts are for the visitors it actually claimed.
• Context can contain personal data. Bandit skips keys named like ids, such as userId, sessionId, url, timestamp, anything ending in Id or _id, and anything starting with __. It has no block list for a key like email, so don't send one.
• A rule only matches requests that send all its keys. Use the same keys every time.

From reading groups to serving them

Audiences start as a report. A standalone experiment, one that is not in a funnel or campaign, can go further. An admin can upgrade it from Thompson sampling to a contextual bandit that picks versions per audience, country, and survey answer.

The upgrade is available only when the experiment scores conversions and has, in the last 30 days, at least 1,000 assignments, at least 100 conversions, and at least 200 impressions per version, more with more features. The latest audience run must also show at least one group with a signal on that experiment. The upgrade freezes the audience rules it learned on, so later runs don't change what the model sees. Reverting is always allowed.

How to turn it on

1. Send short categorical context with each assignment request:

const { config } = await bandit.getAssignment('hero-headline', visitorId, {
  device: 'mobile',
  source: 'google',
  buying_for: 'a gift',
})

2. Record survey answers that should describe groups, not form them:

await bandit.label(visitorId, { role: 'founder' })

3. Open Audiences in the sidebar. The first run happens at the next 04:00 UTC, or right away if an admin chooses Find audiences now.

The SDK guide covers both calls. The growth marketers and developers pages show where audiences fit in a test.

Bandit only shows a group when it buys differently, and it shows the counts behind it.