Skip to main content
Split testing in Senderz happens in two places, and neither of them is the A/B Tests screen.
  • An automation can hold an A/B test step that sends contacts down two different paths.
  • A popup can hold up to four variations of itself.
A/B Tests in the sidebar, under Analytics & insights, is a register. It lists the tests your workspace has running and shows what has been recorded against each one. You cannot build a test there.
Read this page first if you are trying to work out where to set a test up. The two builders are documented in full elsewhere: flow steps for the automation split, and popup testing and results for popup variations.

​
Where a test can run

Automations

An A/B test step splits contacts between Path A and Path B. Two paths only. Registers itself on the A/B Tests screen.

Popups

Two to four variations of the whole popup, split evenly. Results live in the popup builder, not on the A/B Tests screen.

Campaigns

Not available. A campaign sends one version to its whole audience. There is no subject line or content split in the campaign wizard.
The campaigns list has an A/B test filter chip, and the A/B Tests screen has a Campaign source filter. Neither has anything to show, because nothing in the product can put variants on a campaign yet. Use an automation if you need a split send.

​
Setting up a test in an automation

The A/B test step lives in the Logic group of the automation palette.
1

Drop the A/B test step where the split should happen

Everything before the step is shared. Everything after it is what you are testing.
2

Connect both branches

The step has two outgoing paths, A and B. Build the steps you want to compare under each one. An automation will not publish with only one branch connected.
3

Choose a goal metric

Goal metric is the thing you are trying to move. The choices are Open rate, Click rate, Conversion (Placed Order), Revenue per recipient, and Custom event. Pick Custom event and you also name the event.
4

Set the split

The Split slider is the percentage that goes down Path A. It moves in steps of 5 and is limited to a range of 10 to 90, so you can weight a test but you cannot send everyone one way.
5

Set a duration or a minimum sample

Duration (hours) and Minimum sample sit side by side. You need at least one of the two filled in before the automation will publish.
6

Save the automation

Saving is what registers the test. The record appears on the A/B Tests screen straight away, named after the automation and the step, with a status of Running.

​
What you can vary

Anything you can put in an automation. The split is a fork in the path, not a field-by-field comparison, so Path A and Path B can differ by as much or as little as you like.
Change one thing at a time. If Path A differs from Path B in subject line and send delay, a difference in the result tells you nothing about which change caused it.

​
Publish checks

Publishing is blocked until the step is complete.

​
How traffic is divided

A contact is assigned to a branch the first time they reach the step, and the assignment is stored on their journey.
  • Assignment is sticky. Once someone is on Path A they stay on Path A for that journey. Re-running the step does not reroute them.
  • The split is respected as a probability, not a queue. With a 50/50 split and 100 contacts you will get close to 50 each, not exactly 50 each. Small audiences drift further from the setting than large ones.
  • A contact is counted once. The participant count on the A/B Tests screen goes up on first assignment only.
  • Changing the split later only affects contacts who have not been assigned yet. It does not move anyone who is already on a path.

​
The A/B Tests screen

Open A/B Tests in the sidebar. Every member of the workspace can view it. Four tiles sit above the table: Total tests, Running now, Significant winners and Aggregate lift. Below them are pill filters for source (Campaign, Automation, Popup) and status (Running, Completed, Deployed, Paused), plus a search box that matches on the test name. The table columns: Click a row to open it. The detail view repeats the header facts, then shows a card per variant with open rate, click rate, conversion and revenue, and a winner breakdown by segment underneath.
Senderz does not calculate the winner, the lift, the confidence or the per-variant cards for an automation test. Those fields exist on the record and are shown if something fills them in, but nothing in the product computes them today. In practice an automation test reads no winner, +0% lift and 0.0% confidence for its whole life, the variant cards say variant performance data is not yet available, and the four tiles above the table stay at zero apart from the counts.

​
Reading results today

Because the screen does not do the maths, compare the two branches directly in the automation editor. Select a message step on Path A, then the equivalent step on Path B. Each one shows a Performance card for the last 30 days with sends, opens, clicks, unsubscribes, orders and revenue. That is the real comparison. See publishing and monitoring for what each number on that card means. Two things to keep in mind when you read it:
  • The card window is the last 30 days, not the life of the test. A test that has been running longer is not fully represented.
  • Compare rates, not totals. An uneven split makes the totals uneven by design.

​
Deciding a winner

There is no automatic rollout. Moving traffic to the better branch is a manual edit to the automation.
1

Confirm the difference is real

Use the per-step Performance cards, and check the participant count is large enough that the gap is not noise. The sample size calculator below gives you a target.
2

Move the traffic in the automation

Open the automation and either slide the split as far as it goes toward the winning path, or remove the A/B test step and connect the winning branch directly to the step above it. The slider stops at 90, so it cannot send 100% one way on its own.
3

Mark the record, if you want the history

On the A/B Tests screen, open the test and press Deploy the winner. This marks the record as Deployed. It is a bookkeeping action.
Deploy the winner does not change any sending. It sets the record’s status and nothing else. The automation keeps splitting traffic exactly as it did before until you edit the automation itself. The button is also disabled until a winner has been recorded on the test, which for an automation test means it stays disabled.
The Automatically promote winner toggle on the A/B test step is saved with the automation and shown in the inspector, but nothing acts on it. Traffic does not move on its own. Treat it as a note to yourself until it is wired up.
The same is true of Duration (hours) and Minimum sample. They are required before publishing and they are stored with the test, but Senderz does not stop the test or change its status when either one is reached. Diary the end date yourself.

​
Sample size calculator

Sample size calc in the page header opens an estimate of how many contacts each branch needs before a result means anything.
​
default:"20"
What the metric you are testing does today, as a percentage.
​
default:"10"
The relative improvement you care about. A 10% lift on a 20% base rate means moving from 20% to 22%.
​
default:"95"
Shown in the wording under the result.
The result is the number of participants needed per variant, so a two-way split needs roughly double that in total audience.
The number is driven by the base rate and the lift you want to detect. Moving the confidence slider changes the sentence under the number but not the number itself, so read the result as a rough floor rather than a precise threshold.
Popups do not use the A/B test step and do not appear on the A/B Tests screen. The Popup source filter has nothing to show. A popup test is set up in the popup builder’s own A/B panel, holds two to four variations of the entire popup, splits traffic evenly with no weighting, and reports views, sign-ups and rate per variation right there. It also crowns a leading variation once one is ahead on conversion rate with at least 100 views, and Decide winner genuinely promotes that variation’s content onto the popup. Full detail in popup testing and results.

​
Who can do what

​
Limits worth knowing

The A/B test step has Path A and Path B, and no more. The record on the A/B Tests screen can hold up to four variants, and shows an MVT tag when it does, but an automation cannot produce more than two.
Delete the A/B test step from an automation and its record stays on the A/B Tests screen. There is no delete action on the screen, so old records accumulate. Use the search box and the status filter to keep the list readable.
Every save of an automation refreshes its A/B test records, and that includes setting the status back to Running. If you mark a test Deployed and then edit the automation, the status will read Running again.
The record is named after the automation and the A/B test step’s title. Rename the step to something descriptive before you save, or every test in a busy workspace ends up called the same thing.
Saving an automation registers the test, so a draft automation’s split shows as Running on the A/B Tests screen even though nobody is being enrolled yet. Check the automation’s own status to know whether it is live.
Senderz AI can register an A/B test record for you, with a name, source, the variable under test, a hypothesis and a variant count. It creates the record only. It does not build the split step or move any traffic, so you still have to open the automation and wire the branches.

Flow steps and actions

The A/B test step in context, alongside conditional splits and wait timers.

Publishing and monitoring

Per-step performance cards, which is where you actually compare the two branches.

Popup testing and results

Popup variations, the even split, the leading badge and Decide winner.

Campaigns

The five-step wizard, and why there is no split step in it.