> ## Documentation Index
> Fetch the complete documentation index at: https://docs.senderz.app/llms.txt
> Use this file to discover all available pages before exploring further.

# A/B tests

> Where split testing actually runs in Senderz, how traffic is divided, and how to read the A/B Tests screen without over-reading it.

Split testing in Senderz happens in two places, and neither of them is the **A/B Tests** screen.

* An **automation** can hold an A/B test step that sends contacts down two different paths.
* A **popup** can hold up to four variations of itself.

**A/B Tests** in the sidebar, under **Analytics & insights**, is a register. It lists the tests your workspace has running and shows what has been recorded against each one. You cannot build a test there.

<Info>
  Read this page first if you are trying to work out *where* to set a test up.
  The two builders are documented in full elsewhere: [flow
  steps](/en/messaging/flows/steps) for the automation split, and [popup testing
  and results](/en/messaging/popup-testing-and-analytics) for popup variations.
</Info>

## Where a test can run

<CardGroup cols={3}>
  <Card title="Automations" icon="code-branch" href="/en/messaging/flows/steps">
    An **A/B test** step splits contacts between Path A and Path B. Two paths
    only. Registers itself on the A/B Tests screen.
  </Card>

  <Card title="Popups" icon="window-restore" href="/en/messaging/popup-testing-and-analytics">
    Two to four variations of the whole popup, split evenly. Results live in the
    popup builder, not on the A/B Tests screen.
  </Card>

  <Card title="Campaigns" icon="ban">
    Not available. A campaign sends one version to its whole audience. There is
    no subject line or content split in the campaign wizard.
  </Card>
</CardGroup>

<Warning>
  The campaigns list has an **A/B test** filter chip, and the A/B Tests screen
  has a **Campaign** source filter. Neither has anything to show, because
  nothing in the product can put variants on a campaign yet. Use an automation
  if you need a split send.
</Warning>

## Setting up a test in an automation

The A/B test step lives in the **Logic** group of the automation palette.

<Steps>
  <Step title="Drop the A/B test step where the split should happen">
    Everything before the step is shared. Everything after it is what you are
    testing.
  </Step>

  <Step title="Connect both branches">
    The step has two outgoing paths, A and B. Build the steps you want to
    compare under each one. An automation will not publish with only one branch
    connected.
  </Step>

  <Step title="Choose a goal metric">
    **Goal metric** is the thing you are trying to move. The choices are Open
    rate, Click rate, Conversion (Placed Order), Revenue per recipient, and
    Custom event. Pick Custom event and you also name the event.
  </Step>

  <Step title="Set the split">
    The **Split** slider is the percentage that goes down Path A. It moves in
    steps of 5 and is limited to a range of 10 to 90, so you can weight a test
    but you cannot send everyone one way.
  </Step>

  <Step title="Set a duration or a minimum sample">
    **Duration (hours)** and **Minimum sample** sit side by side. You need at
    least one of the two filled in before the automation will publish.
  </Step>

  <Step title="Save the automation">
    Saving is what registers the test. The record appears on the A/B Tests
    screen straight away, named after the automation and the step, with a
    status of Running.
  </Step>
</Steps>

### What you can vary

Anything you can put in an automation. The split is a fork in the path, not a
field-by-field comparison, so Path A and Path B can differ by as much or as
little as you like.

| Comparison                             | How to build it                                                         |
| -------------------------------------- | ----------------------------------------------------------------------- |
| Two subject lines                      | One email step on each branch, same template content, different subject |
| Two templates                          | A different template on each branch's email step                        |
| Email against SMS                      | An email step on one branch, an SMS step on the other                   |
| Send immediately against waiting a day | A wait step on one branch before the message                            |
| One message against two                | A single email on A, an email plus a follow-up on B                     |

<Tip>
  Change one thing at a time. If Path A differs from Path B in subject line
  *and* send delay, a difference in the result tells you nothing about which
  change caused it.
</Tip>

### Publish checks

Publishing is blocked until the step is complete.

| Check                             | Fix                             |
| --------------------------------- | ------------------------------- |
| Fewer than two branches connected | Wire both Path A and Path B     |
| No goal metric                    | Choose one from the dropdown    |
| Split outside 10 to 90            | Move the slider back into range |
| No duration and no minimum sample | Fill in one of the two          |

## How traffic is divided

A contact is assigned to a branch the first time they reach the step, and the
assignment is stored on their journey.

* **Assignment is sticky.** Once someone is on Path A they stay on Path A for
  that journey. Re-running the step does not reroute them.
* **The split is respected as a probability**, not a queue. With a 50/50 split
  and 100 contacts you will get close to 50 each, not exactly 50 each. Small
  audiences drift further from the setting than large ones.
* **A contact is counted once.** The participant count on the A/B Tests screen
  goes up on first assignment only.
* **Changing the split later** only affects contacts who have not been assigned
  yet. It does not move anyone who is already on a path.

## The A/B Tests screen

Open **A/B Tests** in the sidebar. Every member of the workspace can view it.

Four tiles sit above the table: **Total tests**, **Running now**, **Significant
winners** and **Aggregate lift**. Below them are pill filters for source
(Campaign, Automation, Popup) and status (Running, Completed, Deployed,
Paused), plus a search box that matches on the test name.

The table columns:

| Column       | What it shows                                                       |
| ------------ | ------------------------------------------------------------------- |
| Source       | Which surface the test belongs to, with the record name underneath  |
| Test         | The goal metric being measured                                      |
| Type         | A short tag for the same thing                                      |
| Variants     | How many variants the record holds. An automation split is always 2 |
| Status       | Running, Completed, Deployed or Paused                              |
| Participants | How many contacts have been assigned so far                         |
| Winner       | The recorded winning variant letter, or a dash                      |
| Lift         | The recorded lift                                                   |
| Confidence   | Derived from the recorded p value                                   |
| Started      | The date the record was first created                               |

Click a row to open it. The detail view repeats the header facts, then shows a
card per variant with open rate, click rate, conversion and revenue, and a
winner breakdown by segment underneath.

<Warning>
  **Senderz does not calculate the winner, the lift, the confidence or the
  per-variant cards for an automation test.** Those fields exist on the record
  and are shown if something fills them in, but nothing in the product computes
  them today. In practice an automation test reads **no winner**, **+0% lift**
  and **0.0% confidence** for its whole life, the variant cards say variant
  performance data is not yet available, and the four tiles above the table stay
  at zero apart from the counts.
</Warning>

### Reading results today

Because the screen does not do the maths, compare the two branches directly in
the automation editor.

Select a message step on Path A, then the equivalent step on Path B. Each one
shows a **Performance** card for the last 30 days with sends, opens, clicks,
unsubscribes, orders and revenue. That is the real comparison. See
[publishing and monitoring](/en/messaging/flows/publish-and-monitor) for what
each number on that card means.

Two things to keep in mind when you read it:

* The card window is the **last 30 days**, not the life of the test. A test that
  has been running longer is not fully represented.
* Compare **rates**, not totals. An uneven split makes the totals uneven by
  design.

## Deciding a winner

There is no automatic rollout. Moving traffic to the better branch is a manual
edit to the automation.

<Steps>
  <Step title="Confirm the difference is real">
    Use the per-step Performance cards, and check the participant count is large
    enough that the gap is not noise. The sample size calculator below gives you
    a target.
  </Step>

  <Step title="Move the traffic in the automation">
    Open the automation and either slide the split as far as it goes toward the
    winning path, or remove the A/B test step and connect the winning branch
    directly to the step above it. The slider stops at 90, so it cannot send
    100% one way on its own.
  </Step>

  <Step title="Mark the record, if you want the history">
    On the A/B Tests screen, open the test and press **Deploy the winner**. This
    marks the record as Deployed. It is a bookkeeping action.
  </Step>
</Steps>

<Warning>
  **Deploy the winner does not change any sending.** It sets the record's status
  and nothing else. The automation keeps splitting traffic exactly as it did
  before until you edit the automation itself. The button is also disabled until
  a winner has been recorded on the test, which for an automation test means it
  stays disabled.
</Warning>

<Note>
  The **Automatically promote winner** toggle on the A/B test step is saved with
  the automation and shown in the inspector, but nothing acts on it. Traffic
  does not move on its own. Treat it as a note to yourself until it is wired up.
</Note>

The same is true of **Duration (hours)** and **Minimum sample**. They are
required before publishing and they are stored with the test, but Senderz does
not stop the test or change its status when either one is reached. Diary the end
date yourself.

## Sample size calculator

**Sample size calc** in the page header opens an estimate of how many contacts
each branch needs before a result means anything.

<ParamField path="Base conversion rate (%)" default="20">
  What the metric you are testing does today, as a percentage.
</ParamField>

<ParamField path="Lift you want to detect (%)" default="10">
  The relative improvement you care about. A 10% lift on a 20% base rate means
  moving from 20% to 22%.
</ParamField>

<ParamField path="Confidence level (%)" default="95">
  Shown in the wording under the result.
</ParamField>

The result is the number of participants needed **per variant**, so a two-way
split needs roughly double that in total audience.

<Note>
  The number is driven by the base rate and the lift you want to detect. Moving
  the confidence slider changes the sentence under the number but not the number
  itself, so read the result as a rough floor rather than a precise threshold.
</Note>

## Popup variations are a separate mechanism

Popups do not use the A/B test step and do not appear on the A/B Tests screen.
The **Popup** source filter has nothing to show.

A popup test is set up in the popup builder's own A/B panel, holds two to four
variations of the entire popup, splits traffic evenly with no weighting, and
reports views, sign-ups and rate per variation right there. It also crowns a
leading variation once one is ahead on conversion rate with at least 100 views,
and **Decide winner** genuinely promotes that variation's content onto the
popup.

Full detail in [popup testing and results](/en/messaging/popup-testing-and-analytics).

## Who can do what

| Action                                          | Roles                          |
| ----------------------------------------------- | ------------------------------ |
| View the A/B Tests screen and open a test       | Any workspace member           |
| Build or edit an A/B test step in an automation | Owner, admin, editor, designer |
| Press Deploy the winner                         | Owner, admin, editor, designer |

## Limits worth knowing

<AccordionGroup>
  <Accordion title="An automation split is always two paths">
    The A/B test step has Path A and Path B, and no more. The record on the A/B
    Tests screen can hold up to four variants, and shows an MVT tag when it does,
    but an automation cannot produce more than two.
  </Accordion>

  <Accordion title="Removing the step does not remove the record">
    Delete the A/B test step from an automation and its record stays on the A/B
    Tests screen. There is no delete action on the screen, so old records
    accumulate. Use the search box and the status filter to keep the list
    readable.
  </Accordion>

  <Accordion title="Saving the automation resets the status to Running">
    Every save of an automation refreshes its A/B test records, and that
    includes setting the status back to Running. If you mark a test Deployed and
    then edit the automation, the status will read Running again.
  </Accordion>

  <Accordion title="The test name comes from the automation">
    The record is named after the automation and the A/B test step's title.
    Rename the step to something descriptive before you save, or every test in a
    busy workspace ends up called the same thing.
  </Accordion>

  <Accordion title="The record appears before you publish">
    Saving an automation registers the test, so a draft automation's split shows
    as Running on the A/B Tests screen even though nobody is being enrolled yet.
    Check the automation's own status to know whether it is live.
  </Accordion>

  <Accordion title="Asking the Marketing Agent to set one up">
    **Senderz AI** can register an A/B test record for you, with a name, source,
    the variable under test, a hypothesis and a variant count. It creates the
    record only. It does not build the split step or move any traffic, so you
    still have to open the automation and wire the branches.
  </Accordion>
</AccordionGroup>

## Related

<CardGroup cols={2}>
  <Card title="Flow steps and actions" icon="layer-group" href="/en/messaging/flows/steps">
    The A/B test step in context, alongside conditional splits and wait timers.
  </Card>

  <Card title="Publishing and monitoring" icon="chart-line" href="/en/messaging/flows/publish-and-monitor">
    Per-step performance cards, which is where you actually compare the two
    branches.
  </Card>

  <Card title="Popup testing and results" icon="flask" href="/en/messaging/popup-testing-and-analytics">
    Popup variations, the even split, the leading badge and Decide winner.
  </Card>

  <Card title="Campaigns" icon="paper-plane" href="/en/messaging/campaigns">
    The five-step wizard, and why there is no split step in it.
  </Card>
</CardGroup>