Features used: Error analysis, Journey analysis, Session replay, Impact quantification
Time: 20 min.
Common use cases for analysing errors and error messages
Identify the impact of errors on your site performance
Identifying the impact of errors on your site performance is crucial for understanding user experience and prioritizing fixes to enhance usability and retain visitors.
Set up
1. Select Error Analysis from the menu.
2. Choose the analysis context to set up your analysis scope (device, time period, and segment).
Learn more about using Error Analysis
Analysis
3. Scroll down to view all errors in the list view. This table lists all the Grouped JS, API, and Custom Errors that occurred for the selected analysis context, along with impact KPIs.
4. Use the search bar to find a specific error, or select the 'Errors by Page Groups' tab to see which of your pages are being most impacted by errors, and then click to view the related errors and their individual impacts.
Example of Search Results Page Errors and their Impact
5. Click an error and from the side panel, click on 'See in Journey analysis'
to understand and compare the paths of user journeys for those who experienced the error, versus those who didn't.
6. To start validating any hypothesis you make, shortcut to Session Replay directly from any step in the journey comparison, or return to the Error Analysis module and review session replays corresponding to each table entry. This will enable you to visualize the impact of the error on the user experience.
Example of a Journey analysis of journeys with API errors preventing users from logging-in
Journey analysis revealed looping behavior after the 'Checkout Address' stage, with users stuck in a loop between Checkout, where an API error occurred, and the basket and Login steps. Session replays confirmed that logging in and returning to the basket resulted in product erasure, leading to multiple exits and further looping behavior
7. Quantify the error's impact on business KPIs by clicking 'Quantify'. This feature automatically compares users who encountered the error with those who didn't, but reached the page where the error occurred.
8. Dig into the impact of the error by looking at the following:
- Check the Change over time tab to better understand when users started to encounter the error
- Check the Localize tab to get more context about the error and see how the error is impacting across browsers and devices
Learn more about drilling into the impact of the error.
Analyze the impact of error/stress-marketing/ warning messages on your site's KPIs
Analyzing the impact of key messages on your site's KPIs is crucial for understanding user behavior, optimizing the user experience, and improving business performance.
With Text search you can look for impactful text-based messages and error messages (e.g., "Oops, this page doesn't exist") and assess their effect on conversion and revenue.
Set up
1. Navigate to Impact quantification, set the Analysis context and type in the message you want to analyze in the search bar (e.g., "Something went wrong", "The price of your holidays has increased").
Learn more about Text search and How to use Text search.
Analysis
2. To calculate the actual impact on conversions and user journey, consider the following:
- Use the Opportunity widget/Metric widget to gauge the impact of the text on revenue and conversion.
- Use the Traffic Distribution widget to determine which Devices, Browsers, and Operating Systems are most affected by the text.
- Check to the Most Viewed Pages widget to identify where text strings and errors are most frequently viewed. Take note of these pages for further investigation in Journey Analysis (refer to step 4 below).
3. Navigate to Journey Analysis, activate comparison mode, and select segments that encountered versus those that didn't encounter the message (e.g., 'Users who saw: "The price of your holidays has increased" versus those who didn't). Compare based on:
- Where do the users land? Are there differences between the two segments?
- Are there standout pages among users seeing the message (e.g., looping back or exiting after a certain page)?
4. If you've identified a page where the message frequently appears (found in the 'Most Viewed Pages' widget) or a problematic page through your analysis in the previous step, select that page from the 'After a page group' drop-down to see the navigations following that page. Look at the following:
- Are there any unusual journeys after the page? For example, users returning to the previous page and adjusting search settings or changing resorts.
- Is there any looping behavior among users exposed to the messaging that could indicate friction or confusion? For instance, are they more prone to looping back to a search results page and starting a looping behavior?
Example of a Journey analysis of journeys after a page with a warning message
Journey analysis showed that users unable to purchase their desired travel due to unavailability (exposed to the "Sorry, there’s no more availability" message) were much less likely to convert. They often returned to the search results page, leading to looping behavior, as they didn't want to restart their search.