Features used: Impact Quantification, Page comparator, Dashboards
Time: 20 min.
How to quantify the potential impact of a change
Understanding and quantifying the potential impact of a UX change is crucial for making data-driven decisions and prioritizing design efforts. It helps reduce the risk of implementing changes without significant impact.
How to use Impact Quantification to quantify the potential of an impact
With Impact Quantification you can prioritize issues based on their impact on conversion and revenue.
1. To start you analysis, first formulate the purpose of the optimisation:
- What behaviour have/will you improve(d)?
- What is the journey you want to encourage?
- What friction have/will you reduce(d)?
- Which error do we want to solve?
2. Define the scope by taking into consideration:
-
Page/feature to optimize: Note down the key metric or user journey for improvement.
- Your optimization goal:
- Improve behavior: Do you want to increase conversions/conversion rate, reach next step rate, feature clicks, and X% scroll rate?
- Reduce frictions: Do you want to decrease click recurrence, looping behavior, and time spent?
3. Create your comparison segments based on the segment of users that have achieved the above goal ('good segment') vs. those who didn't ('bad segment').
You can create them in the Analysis context directly within Impact or any other Contentsquare capability, and then click on the 'Quantify' CTA.
Ex. 'Clicked on content slot on the List page' vs. 'Didn’t click on content slot but viewed the List page'
Note! When comparing your segments make sure to target the users that have viewed the page you’re focusing your analysis on in both segments.
4. Compare your segments on the following metrics:
- Detected opportunity: Are the results statistically significant? Is there a difference between the conversion rates of the two segments? What is the potential revenue that can be generated from having more people achieve the goal you've set (e.g., clicking on a content slot) or from the change you've planned/made?
- Goal conversions (E-commerce or another set goal) and actual transactions (Revenue and Median cart): Is the your 'good segment' having a higher conversions or not? Are they more likely to reach a specific goal or not? Are they more likely to transact or not? What revenue are they generating? How high/low are their carts?
- Other UX metrics (Bounce rate, Average Page views, Average session time): Are there any observable differences in how the two segments consumed the site? Is one of the segments experiencing a higher bounce rate? What about their browsing depth? Are your 'good segments' viewing fewer or more pages? Are they spending less or more time on the site, which might indicate smoother journeys?
Here you can learn more about Impact Quantification and how to adjust your opportunity parameters when calculating ROI.
How to measure the impact of the change
Measuring and reporting on the outcomes of a change enables data-driven decisions, effective UX optimization, and the opportunity to make smarter choices on what to optimize next and how to iterate further.
Impact Quantification
With Impact Quantification you can get a view of the session level view of the impact of the changes based on pre- and post-launch main KPIs progression.
1. To do so, navigate to Impact Quantification and select the Comparison mode in the Analysis context. Then, apply the time period pre and change for comparison. Key metrics to look at are:
- Check the revenue. Is there an impact on conversion and revenue?
- Compare the 2 segments' bounce rate. Are there any spikes or sudden drops of the bounce rate?
- Look at the conversion rates and conversion over time. Is the change having a positive or a negative impact on the conversions? Are there any spikes or sudden drops of the conversion rate?
- Have a look at how the other global metrics are tracking? Is the change improving in some indicators but not translating to an overall improvement in conversion? If so, you need to start analyzing why this is so. Now is the time to start really drilling down and make a deep dive analysis in the rest of Contentsquare's capabilities.
Page Comparator
Page comparator is useful for getting an overview of different segments’ activity on a given page: Did it impact any of the key UX metrics on the page?
1. To begin your analysis, open the Page comparator and select the Comparison mode in the Analysis context . Then, select the two timeframes for comparison - before and after the launch.
2. Favor the key page/s that you made a change on and look at the following metrics:
-
Check the exit/bounce rate of your key pages to figure out if the changes made are helping in retaining your visitors.
-
Look the load time to understand If the change had an impact on the loading speed of any of your key pages
-
Apply different conversion goals (link those to the predefined objectives of key pages) to understand how the changes are impacting the micro conversions of your page: Are your visitors now more likely to achieve the goal you’ve set?
Dashboards
Dashboards are designed to meet various needs, allowing you to easily grasp the performance of different campaigns, the scope of different teams, and specific pages, all in a single view.
Below you can find a few examples of different types of Dashboards, representing some common use cases for KPI monitoring.
Dashboards for E-Merchandising
Dashboards for Category performance
Dashboards for Acquisition channels
How to track and share insights
Tracking and sharing insights and actions is crucial for ensuring that your business is making informed decisions based on data, rather than assumptions or guesswork.
What tools to use?
There are many tools available to help you track and share insights and actions, depending on your specific needs and preferences. Here are some popular tools that you might find useful:
Monday board: to help track progress
Monday is a customizable project management platform that helps users track insights. Users can organize and prioritize their insights and actions using visual boards and columns, and collaborate with team members using communication tools. Monday enables users to assign ownership, set deadlines, and track progress on each task, as well as measure the ROI of their insights and actions. |
Google Sheets: track & prioritise value
Google Sheets is a cloud-based spreadsheet tool that allows you to track and organize your insights and actions. You can use Google Sheets to create custom dashboards, track progress against your KPIs, and collaborate with your team |
Trello: to help prioritise optimisation ideas
Trello is also a useful tool for tracking insights. It allows users to create boards and lists to organize their insights and actions, assign ownership, and set deadlines. Users can create cards for each insight, attach relevant files, and add comments and checklists to track progress. Additionally, Trello's integration with other tools and services, such as Google Drive and Slack, enables users to share their insights and collaborate with others seamlessly |
PowerPoint/Slides: to communicate a more focused message
PowerPoint/Slides is a presentation software that enables users to create dynamic, engaging presentations for a variety of purposes. With PowerPoint, you can easily create slides that incorporate text, images, videos, and other multimedia elements to communicate their message effectively. |
Airtable: to house all optimisation initiatives and track progress
Airtable can help track insights by providing a central and customizable database that allows users to record and organize their insights in a structured and accessible manner. You can create custom fields for different types of insights, such as descriptions, categories, sources, and tags, and use them to filter, sort, and group their data in different ways. |
Figma: more visual way to track insights
Figma is a design collaboration tool that enables users to track insights related to design projects. With Figma, users can create shared design files, wireframes, and prototypes, collaborate with team members in real-time, and track changes and feedback. Figma's integration with other tools and services allows for seamless sharing and collaboration. |
KPIs to follow
- Decrease share of negative / non significant changes
- Number of positive changes per month / quarter / year
- ROI generated per changes
- Time-saved (per team, per decision, per positive change)
Go Further
Discover how to foster a data-driven mindset across your organiziation with a webinar recording featuring a Contentsquare expert sharing valuable insights, tips, and best practices.