Automated A/B Test with Kameleoon

Create A/B tests directly from Contentsquare using AI-powered automation. Transform insights into action in minutes by describing your test hypothesis in natural language.

Automated A/B Test Creation bridges the gap between insight and action, enabling you to launch experiments without leaving Contentsquare. By leveraging agent-to-agent technology, this feature connects Contentsquare's analytics with Kameleoon's experimentation platform, allowing AI to build your tests automatically based on your natural language descriptions.

Why use an Automated A/B Test?

1. Accelerate time-to-test:

Turn insights from Zoning Analysis into live experiments in minutes. No need to switch between platforms or manually configure test parameters.

2. Lower the barrier to experimentation:

Empower non-technical team members to create A/B tests without coding knowledge or deep platform expertise.

3. Act on insights immediately:

Find an optimization opportunity in Zoning? Create the corresponding test on the spot while the context is still fresh.

4. Increase testing velocity: 

Launch more experiments faster by eliminating manual setup steps and reducing the time between insight discovery and test deployment.

5. Maintain Analysis Context: 

Keep your Analysis Context (e.g, segments, date ranges, device filters…) intact as your transition from insight to experimentation.

Before you begin

To use Automated A/B Test Creation, you must have:

• The Kameleoon integration is installed: Your Contentsquare project must have an active Kameleoon integration. Only admins can install integrations. Learn more about the Kameleoon integration in this article.
• Kameleoon PBX add-on: Your Kameleoon account must include the Personalization by eXperience (also known as PBX) add-on, which provides access to Kameleoon’s AI agent capabilities.
• Appropriate permissions: You must have the necessary/required permissions in both Contentsquare and Kameleoon to create Zonings and Experiments.
• Chrome browser with Kameleoon extension: The Kameleoon PBX Chrome extension must be installed and activated for the automated workflow to function properly.

How to create your Automated A/B Test

Automated A/B Test Creation is accessible directly from the Zoning Analysis:

• From the main menu, go to ‘Zoning Analysis’
• To begin an analysis, select ‘Start from Scratch’ or open an existing Zoning from your list.
• Look for the ‘Design A/B Test’ button in the right-side panel.

Create your Automated A/B Test by following the next steps:

Step 1: Identify your opportunity in Zoning: 

Begin by analyzing a page or screen in Zoning Analysis where you’ve identified an optimization opportunity. This could be:

• An underperforming element that needs redesign
• A high-engagement area that could be enhanced
• A confusing layout that needs simplification
• A conversion blocker that needs removal 

Step 2: Initiate test creation

• While viewing your Zoning, click the ‘Design A/B Test’ button in the right-side panel
• A test creation form will appear.

Step 3: Define your test

From the form, you will be prompted to answer a few questions:

1. Name your test: Give your A/B test a clear and descriptive name that helps you identify it later. Examples: “Home page Hero CTA Color Test” or “Product Page Add to Cart Button Size”.
2. Describe your hypothesis: Explain what you’re testing and why. Be specific about what you expect to happen. Examples: “Changing the CTA button from blue to orange will increase click-through rate because orange create more visual contrast with the background.” or: “Making the ‘Add to cart’ button larger and more prominent will reduce hesitation and increase add-to-cart rate”.
3. Specify the expected visual changes: Describe exactly what should change on the page. The more specific you are, the better the AI can build your test. Examples: "Change the primary CTA button color from #0066CC (blue) to #FF6600 (orange), keep all other styling the same" or: "Making the 'Add to Cart' button larger and more prominent will reduce hesitation and increase add-to-cart rate"

Step 4: Let AI build your test

Once you complete Step 3 and submit: 

• Contentsquare connects to Kameleoon, sharing test details and directing you to the test page.
• The Kameleoon Chrome extension activates, sending your prompt to the Kameleoon AI agent.
• The AI agent interprets the description and automatically creates a test variation.
• You can review and refine the AI-generated variation, adjust test settings (traffic, goals, duration), and launch the test within Kameleoon.

Analyzing your A/B test results within Contentsquare

Once your test is active, you can return to Contentsquare to analyze its performance.

View test data in Zoning

1. Create a new Zoning or open an existing one.
2. In the Zoning creation flow, select 'Analyze an A/B Test'.
3. Choose your Kameleoon experiment from the dropdown.
4. Select the variation you want to analyze. Tip: You can choose from different integrations from your own catalog, not just Kameleoon.
5. Contentsquare will automatically:
   • Pull the correct screenshot for that variation.
   • Create or apply the appropriate segment.
   • Display behavioral data for users who saw that variation.

Use A/B Test Dashboards

Monitor your test performance at a glance by creating a dedicated Dashboard:

1. Go to Dashboards and create a new dashboard or use the A/B Test template.
2. Add widgets to track:
   • Conversion metrics for each variation.
   • Engagement metrics (click rates, scroll depth, hesitation).
   • Page performance (bounce rate, exit rate).
   • Zone-level interactions for tested elements.
3. Set up Alerts to notify you of significant changes or anomalies.

Learn more about creating Dashboards in this article. 

Access Session Replays

Watch real user sessions for each test variation:

1. From your Zoning, click 'View Replays'
2. Sessions will be automatically filtered to the selected variation
3. Observe how users interact with your test changes
4. Identify unexpected behaviors or issues

Best practices

Need guidance to write effective test descriptions to make the most of your Automated A/B Test? This section is for you!

General tips for writing effective test descriptions

Be specific with visual details: 

• ✅ "Change the header background from white (#FFFFFF) to navy blue (#001F3F)"
• ❌ "Make the header darker"

Include context about the location:

• ✅ "Move the search bar from the right sidebar to the center of the hero section, below the headline."
• ❌ "Move the search bar"

Specify exact text changes:

• ✅ "Change button text from 'Learn More' to 'Get Started Free' and add an arrow icon (→) after the text"
• ❌ "Update the button text"

Mention what should stay the same: 

• ✅ "Change only the CTA button color, keep all other page elements unchanged"
• ❌ "Change the CTA button color"

Tips for writing effective prompts:

• Be specific about colors, sizes, text changes, and element positions
• Include CSS selectors or element descriptions when possible
• Mention any elements that should NOT change
• Describe one clear variation; you can create additional variations later in Kameleoon.

Test strategy tips

Start with high-impact areas:

Focus on elements with high visibility and engagement in your Zoning data.

Test one variable at a time: 

For clearer insights, isolate individual changes rather than testing multiple changes simultaneously.

Define success metrics upfront: 

Before creating your test, decide which metrics will determine success (conversion rate, click-through rate, engagement time, etc.).

Ensure sufficient sample size: 

Check that your page has enough traffic to reach statistical significance within a reasonable timeframe.

Monitor early and often: 

Review test performance in the first 24-48 hours to catch any implementation issues quickly.

FAQ & Troubleshooting

Go further

• Kameleoon Integration
• How to create Heatmaps and Zonings
• How to setup Contentsquare for your A/B test analysis
• How to request an integration
• Introduction to Zoning Analysis