search

Publishing & Monitoring Tests

Once your A/B test is designed and tagged, the final step is to launch it and monitor the data until you have a statistically significant winner. Optibase provides flexible scheduling and clear probability metrics to help you make confident decisions.

hashtag
1. Publishing Your Test

When you create a test, it begins in Draft mode. To start showing variations to your visitors, you have two options:

  • Active: Set the status to Active to start the test immediately.

  • Scheduled: Choose a specific date and time for the test to launch automatically. This is ideal for coordinating experiments with marketing campaigns or product launches.

You can manage the lifecycle of your test at any time by switching the status to Paused, Archived, or Finished.

hashtag
2. Setting Auto-Stop and Alerts

To prevent manual monitoring, you can automate the conclusion of your experiment:

  • Auto-Stop: Set a specific threshold (e.g., 2,000 tested users) at which the test will automatically cease.

  • Alerts: If you prefer to review the data yourself before ending a test, set an alert to notify you when your traffic threshold is reached.

hashtag
3. Monitoring Results

The results tab provides a real-time breakdown of how each variant is performing. While you will see charts and analytics immediately, the most important metric to watch is the Probability to be Best.

hashtag
Understanding Statistical Significance

It is tempting to end a test as soon as one variant looks like it’s winning, but early data can be misleading. To ensure your results are accurate, follow these guidelines:

  1. Traffic Volume: Let the test run for at least a few hundred users. If your site has high traffic, aim for a few thousand users to smooth out statistical "noise."

  2. The 95% Threshold: Do not declare a winner until the Probability to be Best reaches 95% or higher.

  3. Timeframe: Ensure the test runs long enough to account for weekly cycles (e.g., weekend behavior vs. weekday behavior).

hashtag
4. Ending the Test

Once a variant has maintained a 95%+ probability over a sufficient sample size:

  1. Set the test to Finished.

  2. Make the change permanent: Update your website's actual code or design to reflect the winning variant.

  3. Remove the Optibase attributes from your site once the winning version is hard-coded.

PreviousURL Split TestingNextBricks Builder

Last updated