# AI and Personalization Are Revolutionizing E-commerce Search ## Traditional search no longer meets consumer expectations in the digital era. But as AI, personalization and semantic, intent-based search come together to deliver sophisticated consumer experiences, search has the opportunity to adapt and reclaim its position as an exciting gateway to product discovery. **Summarize this article** **Here’s what you need to know:** - Traditional keyword-based search is outdated and struggles to understand user intent, leading to irrelevant and frustrating eCommerce experiences. - AI and personalization now enable smarter, context-aware and visually driven product discovery that aligns with how people naturally search. - Product discovery is defined by three key consumer behaviors — browsing, purpose-driven and product-specific exploration. Advanced semantic search, personalization, visual analysis, generative AI and deep learning can enhance all three. - This intelligent, unified approach has the potential to boost conversions and engagement by delivering hyper-relevant, intuitive shopping experiences tailored to each user. Search has been an integral part of our lives for decades — but it’s due for an overhaul. Let’s go back to the days of early search engines to understand why. Say you were looking for up-to-date information for a research report. Your initial instinct was to type a question into the search box. The results appeared, but they were mostly irrelevant. You had to learn how to translate your thoughts into a few searchable keywords, which even then would cast a wide net that required sifting through pages of results before you could piece together the right information. As SEO evolved, the search process became less arduous, but conceptually, it hasn’t undergone any meaningful evolution in decades. Instead of search dynamically adapting to people as they engage, humans have adapted to the logic of machines. This lack of nuance has left users frustrated and disengaged. Today, we’ve reached an inflection point. A growing number of users are turning to AI tools like ChatGPT — which receives over a billion queries each day — to find what they want in seconds. Now search is finally evolving to meet consumers’ ever-growing wants and needs. ## Changing consumer expectations are forcing a paradigm shift in search AI opened the floodgates to new ways of engaging with brands and product discovery. Shoppers now expect more sophistication, speed and intelligence in their search experience. Here’s how we got here: **The way people search conflicts with traditional search capabilities** Let’s consider how a person typically engages with search. Imagine someone looking for a dress to wear to a friend’s wedding. Even though 50% of questions are more than three words (according to data from Dynamic Yield), consumers typically use queries of 2-3 words to zero in on what they want. So, this person enters the query, “dress for wedding,” but the search engine only surfaces white dresses, failing to recognize that guests should not wear white. Now, picture a situation in which the shopper just asks for what they want: “I need a dress for a friend’s al fresco wedding in Florida.” The additional context and specificity contained in those extra words could save consumers a lot of time — if the search function were sophisticated enough to understand them. Consumers have adjusted because historical keyword search is not designed to handle complex queries, since product feeds are often poorly tagged and search hasn’t been able to draw from real-world knowledge. Such queries can even lead consumers to unrelated products, as the meaning behind the words matters just as much as the words themselves. **AI-driven guidance and recommendations are more personal** Traditional search relies on user input and filtering but is unable to leverage potentially valuable data about consumer preferences, leading to generic results. To get around the frustration, some 70% of people have opted for generative AI over traditional search for guidance and recommendations. Further, most trust these AI recommendations and accept them without additional research since they address their needs so specifically. For brands to avoid losing valuable opportunities like these to engage (as well as a lack of control around how their products show up across Gen AI tools), they’ll have to evolve their search experience from generic to tailored based on context and data. **Consumers have come to rely on visual information when shopping** Online shoppers that struggle to describe what they’re looking for prefer to use visual information to bridge the gap. In fact, 85% of respondents to a Pinterest survey said visual information was more important than text when searching for clothes and furniture online. Traditional keyword searches simply can’t deliver results based on visual analysis. ## How AI and personalization came together to redefine search Today, AI-driven algorithms and personalization are elevating search to new heights. Search isn’t just one-size-fits-all anymore; it can dynamically adjust to the various ways that people look for the products they need. In fact, there are three common ways that people discover products. Let’s unpack each one and delve into how these groundbreaking new search capabilities can better meet consumer needs. 1. **Personalized navigation for browsing-driven consumers** These are shoppers interested in exploring what’s in a particular category rather than navigating to a specific product. To streamline the search experience based on this high intent browsing behavior, brands can use personalization to identify navigational search queries (like “men’s shoes”) and direct these queries to a tailored category page rather than the site’s default search experience. _A search for men’s shoes surfaces a category page with personalized results._ These category pages can also be sorted and optimized using sophisticated, deep learning algorithms to surface the most relevant products for each user according to their preferences and predict what they might be most interested in next. Different algorithms and merchandising rules can also be targeted to different pages and audiences to determine the best possible combination for maximum engagement. 2. **AI-powered assistants for purpose-driven consumers** Purpose-driven consumers know what they want and are ultimately seeking guidance. For instance, they might know they need clothes for the gym but have yet to narrow in on specific products. Personalization and AI can now help retailers understand the intent behind colloquial, question-based search queries. When paired with advanced semantic search, which can interpret the meaning behind a query (not just the words), the experience gets even better. A great example of this in action: generative AI-powered conversational experiences like [Shopping Muse](https://www.dynamicyield.com/shopping-muse/). Whether a shopper searches for “running jacket for winter” or “What should I wear jogging in cold weather?”, these AI chatbots use natural language processing and deep learning models to answer direct questions, recommending the most relevant products every time. And by analyzing contextual and behavioral data, the chatbot is better suited to anticipate what the consumer might need next. In our experience, retailers have found that shoppers are more inclined to buy and have higher cart values than those who do not engage with tools like this. It can also be an easy way for consumers who are shopping for someone else, or simply lack product knowledge, to find the perfect gift. 3. **Advanced semantic search and visual analysis for product-driven consumers** These are high-intent shoppers who are trying to navigate directly to a particular item, like “high-top basketball shoes.” With such a specific query, we might assume traditional search methods are enough. But advanced search capabilities enable consumers to filter through the noise of large product catalogs and reach those shoes in an instant. Drawing from user history, affinity and contextual clues, personalized search can surface high-top basketball shoes with the colors and features they prefer. Personalized autocomplete can also drive shoppers to related products they wouldn’t have discovered navigating through category and product listing pages. Through visual analysis tools, physical attributes of items in a brand’s product catalog can now be recognized, eliminating the need for third-party catalog enrichment and enabling retailers to surface the right products quicker. For instance, even if an item is not tagged “striped trousers,” AI can “see” the stripes in the image and know to include it in the relevant search results. Further, consumers who spot something they like — for instance, a stylish pair of sneakers in a window while out running errands — can upload a photo and instantly be matched with lookalike products from the retailer’s catalog. Even when a certain item in the catalog is out of stock, visual search can return the next most relevant item. _A shopper uploads a photo of a woman in a red dress, and visual search delivers multiple products that resemble the original._ Overall, these visual analysis tools improveaccuracy and reduce manual product feed management, benefiting both brands and consumers alike. ## Intelligent search has arrived The days when search, personalization and AI held separate roles in e-commerce are coming to pass, and as they merge in exciting new ways, consumers can expect faster, simpler and more impactful digital experiences. On top of that, brands can tap into a host of benefits, including: hyper-relevant search results that lead to higher engagement and average order value; the ability to scale search globally by enabling consumers to submit queries in almost any language; and the power to align search experiences with business objectives, ranking results based on profit margins, return rates, inventory levels and more. And the benefits multiply as more people begin to turn to search as a truly helpful tool for product discovery, with each interaction enabling deeper personalization across other touchpoints. A frictionless search experience maximizes conversions, drawing on what retailers know about their customers — from previous purchases to loyalty memberships — to deliver an ultra-tailored, smooth experience. With AI at the helm, search is primed to not only reclaim the fundamental role that it once held for consumers — but to revolutionize the way they discover and purchase products. Interested in how you can get your hands on the next generation search capabilities? [Get in touch with us](https://www.dynamicyield.com/search/?utm_source=thoughtleadership&utm_medium=xp2&utm_campaign=search) to schedule a tailored demo of Experience Search, which can help your brand deliver smarter, more personalized search experiences that convert. Dynamic Yield by Mastercard is not affiliated with the research cited in these sources. 1 Lee, Kristian Kask and Joel. “Believe It or Not, Chatgpt Gets over 1 Billion Messages Every Single Day.” PCWorld, December 5, 2024. https://www.pcworld.com/article/2546712/believe-it-or-not-chatgpt-gets-over-1-billion-messages-every-single-day.html. 2 Indig, Kevin. “New Data: What Consumers Really Think about Generative AI.” www.growth-memo.com/p/new-data-what-consumers-really-think-about-generative-ai. 3 Pinterest. “Upgrading Lens for More Online to Offline Inspiration.” Pinterest Newsroom Archive, 17 Sept. 2019. newsroom-archive.pinterest.com/upgrading-lens-for-more-online-to-offline-inspiration. #### Read next - [Conversational Commerce: Your Guide to This Market-Shifting Technology](https://www.dynamicyield.com/article/conversational-commerce-guide/) JR Moore - [When Words Fall Short, Let the Image Speak: How Visual Search Is Redefining Product Discovery](https://www.dynamicyield.com/article/how-visual-search-is-redifining-product-discovery/) Lior Delouya --- # 3 Tips to Integrate GenerativeAI into Your Personalization Workflow — Dynamic Yield _Welcome to our new column [Dynamic Voices](https://www.dynamicyield.com/articles/?type=dynamic-voices&order=date&gate=or), a regular series on XP² featuring timely perspectives on personalization, from thought leaders within our network. In this edition, [Harry Hanson-Smith](https://www.linkedin.com/in/hhansonsmith/), Vice President of Northern Europe, shares practical tips that personalization practitioners can implement to better reach their goals with the help of AI._ From the looks of my LinkedIn feed, I’ve begun to sense that the market is feeling a bit overwhelmed by the noise around generative AI. And after speaking with [personalization](https://www.dynamicyield.com/article/personalization-guide/) teams and business leaders, it’s clear that AI is being adapted faster in cultural conversation than it is being implemented in businesses and personal processes. I’m here to cut through the noise and tell you not only that teams _can_ integrate AI into their workflows today and optimize both their results and efficiency—for their individual careers and overall business impact—but that they _should_ ASAP. Though AI has widened the possibilities of what a single person can do in terms of creation and analysis, it still relies on that one person’s expertise. For example, though a marketer can use GenAI to create hundreds of different copy options to A/B test for in a new campaign, that marketer still needs to write a prompt that communicates information regarding a brand and its audience and edit its outputs to ensure they feel natural and rooted in genuine human empathy. And though a marketer can use advanced machine learning tools to anticipate and fulfill customer needs, it can only do so with pinpoint accuracy if the data feed is error-free. In sum: People need to implement generative AI into their workflows for any real gains to be made. While all this prep work and QA’ing can feel too expensive and difficult for [personalization teams](https://www.dynamicyield.com/lesson/roles-of-a-personalization-and-optimization-team/), it’s imperative to do the work now. But luckily, there’s much that can be done to make the journey feel less daunting. Here, three key strategies that I’ve seen work for personalization practitioners, teams, and leaders. ## **Invest in Your Data Now:** There are already many AI-powered recommendation tools baked into your operating system (we’ll get to them later), but to ensure maximum impact, you’ll need to collect and utilize your data effectively. AI capabilities hinge entirely on the quality and quantity of data it’s exposed to, so your team needs to make sure all data is relevant and consistent. It’s not an easy task, but it’s worth it. Let’s look at [home24 as an example](https://www.dynamicyield.com/article/home24-product-feed/%20rel=) : As the company grew their personalization program, they noticed their product feed was less-than-perfect, due to a previous strategy that focused on time to market over data integrity. To maintain their competitive advantage, home24 had to completely reorganize the structure of their product feed, purging any duplicated, irrelevant, or inconsistent attributes, and adding those that are timely. While the project was an incredibly technical, time-consuming, and complicated project, it was also the only way to improve AI-powered recommendation quality and ensure the data feed’s integrity as the program scaled. While teams may want to delay this work, the time to [tidy up the product feed](https://www.dynamicyield.com/article/fashion-product-feed-cleaning-recommendations/) is now as it will guarantee a competitive advantage. ## **Explore AI Prompt Training:** Marketers can supplement their workflows with text-based AI tools like Gemini or ChatGPT, or image-based tools like Canva to create different copy and visual variations for [A/B testing](https://www.dynamicyield.com/lesson/introduction-to-ab-testing/). I’ve heard from my network that the most common challenge teams face around AI adoption was knowing how to nail the perfect prompt—and this is easily solved with proper training. You need to be explicit to create any usable asset that aligns with your brand and goals. The context you provide for a text-based AI tool is just as important as the type of tool you’re using. As we know, AI is still a fallible technology, but continuing to experiment with writing AI prompts will give you a better sense of the level of specificity the tool requires to suit your needs. _You can unlock new ways to fuel creativity and produce captivating campaigns in seconds within Dynamic Yield’s Experience OS. Learn how to break free from repetitive messaging and visuals with_ [_generative AI-powered offerings_](https://www.dynamicyield.com/ai/) _._ Marketers need practical tips for using AI, and I found [this resource](https://huit.harvard.edu/news/ai-prompts) from Harvard University to be an excellent jumping-off point. You can provide any number of directions to the tool, like what you do or do not want to be included and how you want it to be presented. Feedback is essential, too. If you aren’t satisfied with an output, let the tool know so it can correct the mistake. And if you’re stuck on creating a prompt altogether, ask AI to help generate one for you. When given the appropriate context and direction, AI can yield incredible results. _These two prompts were written by an online retailer’s personalization team. The first prompt is generic (could describe any number of t-shirts) and leads to a long, vague output. The second exemplifies how more parameters makes the output more concise, effectively communicating critical details about the product._ ## **Experiment with Different AI Tools:** Beyond GenAI, there are advanced AI-powered tools that can plug directly into your personalization provider and improve the user experience and [product recommendations](https://www.dynamicyield.com/lesson/product-recommendations-guide/). For example, in a world where customers seek highly personalized digital experiences, sophisticated [generative AI-powered chatbots](https://www.dynamicyield.com/shopping-muse/) can create a conversational commerce experience that mimics the in-store consultative experience, using machine learning capabilities that identify and surface visually similar products. You can also improve recommendations with [deep learning](https://www.dynamicyield.com/adaptml/) that processes data inputs across users to identify trends and patterns across your customer’s behavior. ## **The Future Is Now** As AI continues to revolutionize how marketers can interact with consumers, there’s pressure to leverage it as much as possible to stay efficient and ahead of the game. Experimenting with what’s available will take time and patience, but once you cut through the fluff, the benefits you will see will far outweigh any growing pains, plus help you discover more innovative and elevated personalization strategies. I hope these practical examples give you a place to start in your everyday role. --- # From Fragmentation to Connection: Mastering User Identification for Personalization — Dynamic Yield ## Discover practical strategies to improve user identification and drive loyalty in an omni-channel world. **Summarize this article** **Here’s what you need to know:** - User Identification: Crucial for providing a seamless and personalized experience across multiple channels, forming the cornerstone of a successful omni-channel strategy. - Omni-Channel Opportunity: Brands should view omni-channel interactions as a positive sign of brand awareness and loyalty, using each touchpoint to strengthen connections and learn user preferences. - Benefits of User Identification: Enables consistency across channels, data-driven personalization, enhanced customer experience, and efficient marketing strategies. - Boosting User Identification: Implement omnichannel events, use targeted campaigns to increase subscriptions and logins, and identify users via external email campaigns with custom codes. ## **The Importance of User Identification for Personalization in an Omni-Channel Environment** In today’s hyper-connected world, customers interact with brands across multiple channels, from mobile apps and websites to physical stores and social media. To provide a seamless and personalized experience, it is crucial for businesses to identify users accurately and consistently across all these touchpoints. This process, known as user identification, forms the cornerstone of a successful omni-channel strategy. ## **Embracing Omni-Channel as a Positive Opportunity** Rather than fearing fragmented data and inconsistent user experiences, brands should view omni-channel interactions as a positive sign of brand awareness and loyalty. The more a user engages with your brand on different channels, the more connected and invested they are. Each touchpoint offers an opportunity to strengthen this connection and learn more about the user’s preferences. By focusing on building robust user identification throughout the customer journey and across multiple channels, businesses can turn the perceived challenge of fragmentation into an asset. Developing seamless identification strategies not only enhances personalization but also nurtures long-term relationships with loyal customers. ## **Why User Identification Matters for Personalization** Personalization has become a key factor in customer satisfaction and loyalty. When users feel that their preferences are acknowledged and catered to, they are more likely to engage with the brand and make purchases. User identification enables: 1. **Consistency Across Channels:** When a user moves from a mobile app to a website or visits a physical store, their preferences and data are preserved. 2. **Data-Driven Personalization:** Businesses can leverage data collected from different interactions to predict user needs and offer timely recommendations. 3. **Enhanced Customer Experience:** Personalization fosters a sense of connection, making customers feel valued and understood. 4. **Efficient Marketing Strategies:** By understanding user journeys, businesses can target promotions and content more effectively. ## **How can you Boost User Identification?** 1. **Implement! Make sure Omnichannel Events are implemented throughout the user journey.** These events trigger identification when a user: Signs up to Account, Logs in to Account, Subscribes to a newsletter or identifies themselves during checkout 2. **Increase Subscriptions & Logins with Targeted Campaigns –** Using the [Dynamic Yield Empathic Personalization](https://support.dynamicyield.com/hc/en-us/articles/17653567078173-Empathic-Personalization) framework, you can personalize your strategy based on the needs of your users in their customer journey. - Curious Users – Use exit-intent pop-ups to offer a gentle introduction: “Unlock early access to exclusive collections!” - Interested Users – Encourage deeper engagement with a “Save your favorites” prompt when adding to the wish list to capture intent and strengthen the relationship. - Focused Users – Facilitate a seamless checkout and build trust with the prompt: “Create an account for express checkout & order tracking.” - Satisfied Users – Encourage repeated logins and loyalty with post-purchase sign-up benefits: “Track orders & unlock VIP perks.” 3. **Identify Users via External Email Campaigns** Create a _Custom Code_ campaign to identify users arriving from external email campaigns via CUID (eg hashed email, custom CUID. When users click links in your ESP-sent emails, a parameter captures their CUID, enabling identification. Check out [how to set this up](https://support.dynamicyield.com/hc/en-us/articles/360034332473-Matching-Users-Across-Channels#identifying-users-from-external-email-campaigns-0-3) in your account and start increasing your identified users today with our ready-to-use templates! 🚀 #### Read next - [Breaking the Personalization Barrier for Banks](https://www.dynamicyield.com/article/banking-personalization-issuers-breaking-barrier/) Parks Daniel - [How Personalization Fuels Success in Latin America’s Digital Boom](https://www.dynamicyield.com/article/personalization-growth-latin-america/) Karin LaHalle - [AI and Personalization Can Close the Empathy Gap](https://www.dynamicyield.com/article/ai-personalization-empathy/) Yaniv Navot --- # Choosing the right traffic allocation in A/B testing **Summarize this article** **Here’s what you need to know:** - Choosing the right traffic allocation method is crucial for A/B testing success. - Manual allocation offers precise control and is ideal for long-term tests requiring statistical significance before implementation. - Automatic allocation prioritizes data exploitation and conversion optimization, making it suitable for short-term tests. - A/B testing should leverage both methods strategically, depending on the specific test goals and timeline. A vital tool for marketers, [A/B testing](https://www.dynamicyield.com/blog/introduction-to-ab-testing/) allows businesses to make impactful, data-driven decisions regarding the customer experience. Whenever you set up an A/B/n test with multiple variations, it’s important to determine how you want the traffic to be distributed between the variations. The behavior of each traffic allocation option is as follows: ## Manual traffic allocation (The classic A/B testing approach) ## In a nutshell, with manual allocation, traffic is split evenly between variations until a single winner is declared. De-facto a standard A/B/n test, the assumption is that once results are significant, the test administrator will assign solely the best variation to all visitors. For example, if you launch a test with four variations, you may decide that all variations should have equal exposure, 25% of traffic each. Alternatively, you can favor certain variations over the other and go for any other combination of allocation rates that amount to 100%, such as 50/20/20/10. Manual Allocation tests are by definition tests between variations (and control group, if relevant), in which ultimately one variation will be declared the winner with high confidence levels. ## Automatic allocation (Multi-Armed Bandit) With automatic allocation (also called dynamic allocation or multi-armed bandit method), the highest-performing variation is gradually served to a larger percentage of visitors as more data is collected. Over time, the system dynamically routes traffic to the best performing variation based on available data. This means, even if variation A is the best performer today, a month from now, a different variation may potentially outperform it. The following diagram illustrates the manual versus automated traffic allocation behavior over an ongoing experiment between two variations, where a decision is required by the eighth day: ## How to choose the right traffic allocation for your A/B tests? Each of the traffic allocation methods is optimized for distinct use-cases. Ask yourself which of the following assertions better describes the use case at hand: **1)** I am looking for the best variation so I can present it to all users in the long run. In this case, choose manual allocation. Use case example: Layout and UX changes. **2)** I am looking to make the most out of several variations during the limited time the test will run. In this case, choose automatic allocation. Use case example: Promotions on the hero banner. Manual allocation should be used when [statistically significant results](https://www.dynamicyield.com/glossary/statistical-significance/) are required for making the decision to carry out a stark permanent change to the website, and time is not of the essence. Manual allocation tests can run for as long as required collecting data that will result in highly [conclusive statistically significant results](https://www.dynamicyield.com/glossary/conclusive-results/). The downside of such tests is that while you are waiting for significant results – which may take time – there is no exploitation of the data collected. Visitors will still be exposed to the poor-performing variations in the mix. In cases where promotion variations are updated frequently, there may not even be enough time to [reach significant results](https://www.dynamicyield.com/blog/statistical-significance/), and therefore, any optimization opportunity is lost. If you are managing campaigns in which the variations have a short shelf life, or if they change and are updated frequently, then [multi-armed bandit](https://www.dynamicyield.com/glossary/multi-armed-bandit/) is the optimal way to go. Automatic allocation has a much higher exploit rate of readily available data and is much more aggressive when driving traffic allocation decisions. Automatic Allocation knows to weigh in on new variations, variations that perform differently, different time periods, and more. ## Timeliness isn’t best in all matters A/B testing and optimization should be viewed through the lens of time – you either have it, or you don’t. To explain a little further, when looking to test the impact of a long-term change, such as a new page layout, or for the purpose of this post, an email capture message, one wouldn’t want to prematurely make a big or strategic decision without the data to support it. Doing so could have major repercussions on KPIs and the overall customer experience. Given that, a test would need to accumulate enough data about the variation so the team could confidently declare a winner, a process that can take, at minimum, two weeks. In this case, should a business have the luxury of waiting for statistically significant results, an A/B test is ideal. An A/B test set up to find out which message resonates more for long-term deployment But what if the lifespan of a variation is short and there’s isn’t time to wait for a winner? In a scenario where a hero banner is changing on a weekly basis, for example, during a sales event, the main objective is to increase a particular KPI by engaging users with the better performing variation. Sending traffic to a losing variation, therefore, actually reduces CTR, conversions, or whatever other primary metric is being used to measure the test’s success. This is exactly why automated allocation is well suited for shorter-term decisions, seeing as the variation driving the highest results is served more frequently, allowing teams to optimize conversions at much quicker rate. A short-lived test run with dynamic allocation to increase exposure to leading variation We can think of dynamic allocation in terms of Explore vs. Exploit, which addresses how much is “wasted” on learning and the opportunity to capitalize on what has already been learned. Because 10% of traffic in dynamic allocation is always served to a random variation and 90% to the winner, the 10/90 rate of Explore and Exploit allows for traffic to be directed to the leading variation while the algorithm continues to learn about “losing” variations, allowing them to bounce back. ## Balancing data and conversions In the end, the two traffic allocation methods available provide businesses with the flexibility to gain the intelligence needed for sound, long-term decision-making as well as the power to optimize on the fly. Not a question of one or the other, manual and dynamic allocation should both be used for testing, and hopefully, this article helped to clarify when exactly to do so. ##### Continue reading --- # eCommerce navigation optimization best practices and examples You walk into your neighborhood grocery store in search of some ingredients for a new recipe you plan to tackle this evening: rice, chicken, potatoes – your staples. But you’re also in search of saffron, a less familiar ingredient in your household. But knowing it’s a spice, you’re able to quickly navigate to the aisle, locate it on the shelf ordered alphabetically, and complete your checkout without a hitch. While online shopping, brands should aim to facilitate the most straightforward discovery experience possible. In many regards, the eCommerce experience mimics that of brick-and-mortar shops. Users can browse through available inventory, typically accessing items they’ve purchased before with ease and comparing similar products before finalizing a decision. However, digital shops have a challenging task at hand. Unable to fully replicate the in-store shopping experience where consumers often take their time to interact with dozens of products, online shopping tends to be very transactional. To remedy this reality, brands must [personalize](https://www.dynamicyield.com/article/personalization-guide/) the shopping experience to maintain user engagement. From the imagery, button colors, and homepage banners to email subject lines and the cart page design, every decision can dramatically affect how long a shopper remains on-site and whether or not they will complete a purchase. And it’s not just what you offer and how you present products on your site that’s important; how you organize your site can make-or-break your business. Not only will a well-designed site ease the overall shopping experience, but when done right, it will also increase your primary metrics. Below, learn more about navigation optimization, the elements of all eCommerce sites should take into consideration and test, and examples of brands that have designed exceptional navigation experiences. Navigation optimization refers to the process of improving how visitors and search engines find and access information within a given website. This includes the site’s taxonomy, how pages are structured, and how menus are labeled on both desktop and mobile. The design of all of these components can have a tremendous impact on the overall end-user experience, increasing or decreasing metrics like search ranking, [bounce rate](https://www.dynamicyield.com/glossary/bounce-rate/), [pageviews](https://www.dynamicyield.com/glossary/pageview/), time on site, return visitors, [conversions](https://www.dynamicyield.com/glossary/conversion/), and more. So to give you a leg up, let’s take a closer look at the details marketers should pay attention to. ## Breaking down the various components and examples of site navigation ### Primary navigation How you present your header is the backbone of your eCommerce navigation strategy. The two most common formats are menus fixed either horizontally or vertically, exposing a handful of key product categories. Web designers have long debated which presentation is most optimal, but the truth is, it varies from site to site. What works well in one context doesn’t always work equally as well in another. And this is true when looking at design by the channel as well – mobile and web navigation experiences typically vary. Let’s assess the variables at play that will likely impact which approach you take: 1. **Page space:** On desktop devices, a horizontal navigation menu conserves more page space than a vertical one, narrowing the content area available on both your homepage and across site pages. However, on mobile, where space is limited, vertical navigation via a hamburger menu allows you to hide and expose menu items quickly. 2. **Menu item priority:** Typically, the leftmost and top menu items carry the most weight, as these positions are seen as primary areas visually. Additionally, because most users read from left to right, there’s a stronger case for a horizontal navigation experience on desktop sites, where more site real estate is available (the same cannot be said for mobile experiences, where most devices are optimized for vertically-formatted pages). 3. **Scanning:** Many users find the experiences of quickly scanning pages vertically the more natural experience, making a case for vertical navigation menus. ### Categories Navigation structure and labeling should be clear and concise across all pages, and part of this includes deciding how to display product categories. If your product inventory is vast, a navigation menu bar consolidated by category type is integral. [34%](https://baymard.com/blog/mobile-ecommerce-search-and-navigation) of mobile eCommerce sites do not offer “thematic” product browsing, making it difficult for users to find what they are looking for. Therefore, brands should aim to display a few, top-level categories rather than overwhelm shoppers. There are exceptions to this advice, of course. For example, if you only sell hats, it’s probably best to categorize your menu bar by hat type rather than merely listing one option. If you know users tend to primarily shop within a specific product category during a season (i.e., boots in the winter), restructure your menu to prioritize this section above other categories. And it doesn’t stop with how you present your parent categories. Identifying how to showcase sub-categories within your navigation menu is also an essential part of the navigation experience. There are two primary ways brands typically go about this design: 1. **Tiered menus:** List parent categories and only expose sub-categories upon hover or click 2. **Mega menus:** Lays out all parent and subcategories upon initial menu dropdown _eCommerce navigation menu example of a tiered approach where sub-categories are only exposed upon click_ _Menu design inspiration for a mega menu_ Tiered navigation allows a user to make one, simple choice within a given moment by limiting the list of categories and options to choose from. A mega-menu releases an overwhelming sea of possibilities, which risks a user experiencing choice paralysis. While there may seem to be an obvious choice here, we encourage every organization to test both experiences and iterations of each, running experiments to determine which variation works best both for the average visitor, as well as for different audiences. Doing so will instill a sense of confidence in the ultimate decision you make. The order in which you present menu items is also a way to optimize your site navigation. While you may have a default order you present to the average visitor or new user, using affinity data, you can tailor the order of menu items to personalize the experience on a more individual basis based on a user’s preferences. Not only will it expedite the discovery process, but it can also drive conversions more efficiently. _Re-sorted navigation menu example based on each visitor’s affinities_ ### Additional elements to take into consideration Aside from nailing down your main navigation layout, eCommerce teams have plenty of other decisions to make. These include: ##### Sticky navigation These are fixed menus that help users navigate through pages on a site page. To simplify and facilitate a positive online shopping experience, navigation menus, product filters, and sorting menus should always be visible to users and appear while they browse and scroll through a webpage. _Sticky eCommerce menu example_ ##### Design style The design of your menu options can also play a significant role in the navigation experience. From the button colors and look of the primary navigation menu to the fonts used and size of the sorting menu, design decisions can impact how easily users can navigate a site. Testing different looks, colors, sizes, and styles will inform which variations work best for your brand. ##### Menu rendering Two main types of menu styles currently exist: hover pop-down and click pop-down. The hover menu expands when the user’s mouse hovers over the navigation menu, and the latter option only pops down upon click. Similar to design decisions, test both options to identify which one to employ on your site. Additionally, with both tiered and mega menus, brands can showcase personalized product recommendations once fully expanded, further maximizing site real estate. _Featured product recommendations in an eCommerce navigation menu example_ ##### Footer The footer of your site also presents an opportunity to serve as a medium to navigate users to pages they are interested in, encourage email sign-ups, establish your credibility through privacy-related information, and more. Consider displaying links to popular category pages in the fixed footer of your site so users can easily access additional site areas if they’ve reached the bottom of a page. _An example of a detailed footer experience_ In addition to facilitating the product discovery experience, building a more comprehensive footer can positively affect your site’s SEO. Including the right amount of hyperlinks in your footer, even if you don’t opt for a more robust footer experience, will positively impact your search ranking. Evey link listed is analyzed for SEO rankings, so while you shouldn’t pack too many keywords into the footer, we suggest you add the phrases that add value to your brand or encourage user action. ## The various types of search functionalities Besides your primary menu navigation design and stickiness, eCommerce vendors have access to several additional strategies to optimize site navigation. When it comes to search functionality, organizations have two primary options: faceted search and semantic search. ### Faceted search [Faceted search](https://www.dynamicyield.com/glossary/faceted-search/), or guided navigation, helps users analyze, organize, and filter large sets of product inventory based on filters such as size, color, price, and brand. Faceted search options are the result of a search query; therefore, options displayed are solely related to filtering options relevant to the query. _Example of faceted search on an eCommerce site_ On the other hand, [search filters](https://www.dynamicyield.com/glossary/search-filter/) allow users to sort through product attributes based on a particular category, e.g., by size, color, price, or brand. Filters differ from faceted searches because they aim to help browsers narrow down their queries without having to type in a manual search at all. Often seen on the left-hand side of the category or product listing pages, this functionality is especially useful to users browsing through a site with massive amounts of inventory available. _Example of search filters on an eCommerce site_ ### Semantic search Semantic search uses geolocation, a user’s (and global) search history, and spelling variations to improve search queries. This includes programming smart, autocomplete options when a user types items into a search menu. By understanding regional and behavioral trends at the individual and audience level, brands can prioritize the suggested queries that appear when a user begins typing in a search query. _Search results when typing while in incognito mode_ _Search results when typing while logged in_ Additionally, brands may want to conduct additional experiments centered around search functionality. Suppose you notice users are abandoning your site after browsing a few categories or PDPs. In that case, you may want to encourage them to conduct a search on-site as an additional effort to surface a product they are interested in. Especially useful for companies with massive product catalogs, consider testing different variations of the search menu design so it appears more prominently on the page and captures users’ attention. _Two variations of the search box to encourage goal-oriented shoppers to find what they are looking for_ ## Navigating between category pages A best practice for designing [product listing pages (PLPs)](https://www.dynamicyield.com/glossary/product-listing-pages-plps/) and [product detail pages (PDPs)](https://www.dynamicyield.com/glossary/product-detail-page/) is optimizing eCommerce navigation between pages. Once a user arrives on a PDP, they may want to easily return to their initial search results page if they haven’t found what they are looking for. Be sure users don’t lose their place on your site when navigating back and forth between pages by integrating breadcrumbs to help ease site navigation. Breadcrumbs are a linear navigation display, typically visible at the top of a webpage, that display a user’s path through pages to arrive on the current one (i.e., Women > Shoes > Boots > Ankle Boots > Block Heels). Each item in the breadcrumb trail should be clickable and take a user back to that specific page. _Website navigation example: Breadcrumbs seen on the Target website_ ## Additional tips catered toward mobile navigation design It’s integral that your mobile site and app experiences are as seamless as the desktop experience. With more shoppers making purchases on mobile devices than ever, be sure all navigation optimizations render correctly across your mobile inventory and incorporate mobile-first optimizations. Some simple things to consider are menu design, whether menus should be fixed or hidden, vertical or horizontal, whether the menu should render on the right or left side, and more. Another consideration is assessing if your brand should use a hamburger menu to quickly expose and hide menus, maximizing real estate on your mobile site or app. _A vertical menu on the Nike app (left) and Amazon’s lefthand hamburger menu (center: before click; right: after click)_ And as you experiment with maximizing the site real estate, there are some additional strategies teams can rely on specifically for mobile design. If breadcrumbs don’t fit well on a mobile screen, consider using category banners to drive users back to the main category page when browsing a PDP, for example. This will encourage them to continue discovering products rather than abandon the mobile site or app if the product in view doesn’t suit their interests. _Example of a category banner on a mobile PDP page to navigate users back to the primary category_ Take best practices into account when designing how to properly utilize the little inventory you have on your mobile site. Always ensure users have easy access to their carts and avoid intrusive popups that will distract them as they flip between pages. Take advantage of more visual ways to navigate between different product category pages (i.e., using homepage cards to direct users toward popular product categories). Small tweaks, such as incorporating icons instead of text, testing different CTA button messages and colors, encouraging app downloads for richer shopping experiences, and supporting mobile payment options (i.e., Apple Pay) are just a handful of the additional ways to ensure your eCommerce navigation experience is as stellar as your site experience is. ## Real eCommerce navigation experiments from brands in our customer base A leading sports and outdoor brand was looking to personalize its eCommerce navigation experience, focusing specifically on highlighting relevant menu items. Looking to maximize returns, they struggled with a tradeoff between promoting full-priced products and items that were on sale. They knew that users that navigated to New Arrivals using the site menu were more likely to purchase full-priced items but were also less likely to convert overall compared to users that navigate to the Sale page. In order to find a solution that would account for both of these behaviors, they used affinity-based targeting to highlight “New Arrivals” in the menu for customers with an affinity toward full-priced items and highlighted the “Sale” menu item for budget-forward shoppers. Another customer in our customer base that has relied on navigation optimization best practices is a leading beauty brand. Unsure how to effectively organize their primary navigation experience, they especially grappled with whether to showcase brand names or product categories. To make a confident decision, the brand began testing both options and uncovered some important learnings. The first was that new users prefer navigating using familiar categories: lips, eyes, foundation, etc. The second was that returning users preferred exploring the site by navigating to familiar brand name product listing pages. As a result, the beauty retailer tailored the navigation experience according to each type of user that arrived on the site. ## Start testing your way to site navigation success There’s no universal blueprint for building a site navigation experience. Especially in the eCommerce industry, where brands deal with users from various audiences with varying tastes and preferences, it’s essential to test different navigation design elements. Always experiment, and don’t hold back on what elements you test. Play with different menu placements and designs, test which product categories to display in tiered or fixed menus, personalize the order of items in filter menus according to a user’s browsing history, try out different messaging variations, and, of course, always QA your site experiences to ensure they are working in the ways you intend. --- # A/B testing guide by CRO experts, with examples **Summarize this article** **Here’s what you need to know:** - A/B testing is a scientific method for comparing two versions of a webpage or app to see which performs better for a specific goal, like increasing conversions or user engagement. - It’s a powerful tool for optimizing websites, mobile apps, emails, and more, and can help solve UX issues, improve performance, and boost engagement. - To run an A/B test, you first define a problem or user behavior you want to address. Then, you create variations of your original element and split website traffic between them. Finally, you collect and analyze data to see which variation performs best. - Common A/B tests include testing different navigation menus, optimizing landing pages, and experimenting with promotional messages. A/B testing is a method of comparing two versions of a webpage or app against each other to determine which one performs better against a specific objective. It is one of the most widely used techniques for maximizing the performance of digital assets such as websites, mobile applications, SaaS products, emails, and more. Controlled experiments provide marketers, product managers, and engineers with the agility to iterate fast and at scale, leading to data-driven, thoroughly informed decisions about their creative ideas. With A/B tests, you can stop wondering why some things are not working, because the proof is in the pudding. It’s the perfect method to improve conversion rate, increase revenue, grow your subscribers base, and improve your [customer acquisition](https://www.dynamicyield.com/glossary/customer-acquisition/) and lead generation results. Some of the most innovative companies, like Google, Amazon, Netflix, and Facebook, developed lean business approaches, allowing them to run over thousands of experiments each year. As Jeff Bezos has once said: _“Our success at Amazon is a function of how many experiments we do per year, per month, per week, per day.”_ [Netflix wrote](https://netflixtechblog.com/its-all-a-bout-testing-the-netflix-experimentation-platform-4e1ca458c15) in one of their technology blogs back in April 2016: _“By following an empirical approach, we ensure that product changes are not driven by the most opinionated and vocal Netflix employees, but instead by actual data, allowing our members themselves to guide us toward the experiences they love.”_ And Mark Zuckerberg [once explained](https://www.youtube.com/watch?v=Lb4IcGF5iTQ&feature=youtu.be&t=10m9s) that one of the things he is mostly proud of that is really key to their success is their testing framework: _“At any given point in time, there isn’t just one version of Facebook running. There are probably 10,000.”_ ## What is an A/B Test? In a classic A/B testing procedure, we decide what we would like to test and what our objective is. Then, we create one or more variations of our original web element (a.k.a. the [control group](https://www.dynamicyield.com/glossary/control-group/), or the baseline). Next, we split the website traffic randomly between two variations (i.e., we randomly allocate visitors according to some probability), and finally, we collect data regarding our web page performance (metrics). After some time, we look at the data, pick the variation that performed best, and cancel the one that performed poorly. If not done correctly, tests can fail to produce meaningful, valuable results and can even mislead. Generally speaking, running controlled experiments can help organizations with: 1. Solving UX issues and common visitor pain points 2. Improving performance from existing traffic (higher conversions and revenue, improve customer acquisition costs) 3. Increasing overall engagement (reducing [bounce rate](https://www.dynamicyield.com/glossary/bounce-rate/), improving [click-through rate](https://www.dynamicyield.com/glossary/ctr/), and more.) We must keep in mind that the moment we pick a variation, we are generalizing the measures we collected up to that point to the entire population of potential visitors. This is a significant leap of faith, and it must be done in a valid way. Otherwise, we are eventually bound to make a bad decision that will harm the web page in the long run. The process of gaining validity is called _[hypothesis testing](https://www.dynamicyield.com/glossary/hypothesis-testing/)_, and the validity we seek is called _[statistical significance](https://www.dynamicyield.com/blog/statistical-significance/)_. **Some examples of A/B tests:** - Testing different sorting orders of the site’s navigation menu ( [Like in this example from a large electronics retailer in Germany](https://www.dynamicyield.com/use-case/personalized-navigation-menu/)) - Testing and optimizing landing pages ( [Like in this example from a European leading airline passenger protection company](https://www.dynamicyield.com/use-case/data-driven-landing-page-optimization/)) - Testing promotional messages, like newsletter subscription overlays and banners ( [Like in this example from an international boutique retailer of natural bath products](https://www.dynamicyield.com/use-case/tailored-newsletter-signup-offer/)) ## How an A/B test is born: Constructing a hypothesis An A/B test starts by identifying a problem that you wish to resolve, or a user behavior you want to encourage or influence. Once identified, the marketer would typically conclude a hypothesis – an educated guess that will either validate or invalidate the experiment’s results. **Example hypothesis**: Adding a [Social Proof](https://www.dynamicyield.com/glossary/social-proof/) badge to your [Product Detail Pages (PDP)](https://www.dynamicyield.com/glossary/product-detail-page/) will inform visitors of the product’s popularity and increase add-to-cart events by 10%. In this case, once the problem is identified (low add-to-cart rate, as an example) and a hypothesis is worked out (adding a social proof badge to encourage more website visitors to add items to their carts), you are ready to test it on your site. ## The classic approach to A/B testing In a simple A/B test, traffic is split between two variations of content. One is considered the control and contains the original content and design. The other functions as a new version of the controlled variation. The variation may be different in many aspects. For example, we could test a variation with different headline text, call-to-action buttons, a new layout or design, and so on. In a classic page-level experiment, you don’t necessarily need two different URLs to run a proper test. Most A/B testing solutions will let you create variations dynamically by modifying the content, layout, or design of the page. However, if you have two (or more) sets of pages that you’re looking to include in a controlled test, you should probably consider using a split URL test. ## When to use split URL tests Split URL testing, sometimes referred to as “multi-page” or “multi-URL” testing, is a similar method to a standard A/B test, which allows you to conduct experiments based on separate URLs of each variation. With this method, you can conduct tests between two existing URLs, which is especially useful when serving dynamic content. Run a split URL test when you already have two existing pages and want to test which one of them performs better. For example, if you’re running a campaign and you have two different versions for potential landing pages, you can run a split URL test to examine which one will perform better for that particular campaign. ## An A/B test is not limited to just two variations If you want to test more than just two variations, you can run an [A/B/n test](https://www.dynamicyield.com/glossary/abn-testing/). A/B/n tests allow you to measure the performance of three or more variations instead of testing only one variation against a control page. High-traffic sites can use this testing method to evaluate the performance of a much broader set of changes and maximize test time with faster results. However, although it is useful for any testing, from minor to dramatic changes, I recommend not making too many changes between the control and variation. Try making just a few critical and prominent changes to understand the possible causal reasons for the results of the experiment. If you are looking to test changes to multiple elements on a web page, consider running a [multivariate test](https://www.dynamicyield.com/glossary/multivariate-testing/). ## What are Multivariate tests? Multivariate tests, sometimes referred to as “multi-variant” tests, allow you to test changes to multiple sections on a single page. As an example, run a multivariate test on one of your landing pages and change it with two new elements. In the first version, add a contact form instead of the main image. In the second version, add a video item. The system will now generate another possible combination based on your changes, which includes both the video and the contact form: Total test versions: 2 x 2 = 4 **V1** – Control variation (no contact form and no video item) **V2** – Contact form version **V3** – Video item version **V4** – Contact form + video item version Since multivariate tests generate all possible combinations of your changes, it is not recommended to create a large number of variations unless you’re running the test on a high-traffic site. On the other hand, running multivariate tests on low-traffic sites will provide poor results and insufficient data to draw any significant conclusions. Be sure to have at least a few thousand monthly visitors to your site before choosing to run a multivariate test. Example of a multivariate test on an eCommerce product-listing page ## When to use each test type A/B tests will help you answer questions such as: which of the two versions of my page perform better in terms of the visitor’s response to it? Multivariate tests will answer questions like: - Do visitors respond better to a video item next to a contact form? - Or to a webpage with just a contact form and no video item? - Or to a webpage with a video item but no contact form? ## How to measure the effectiveness of the A/B testing platform One method of determining the effectiveness of an [A/B testing platform](https://www.dynamicyield.com/ab-testing/) is to perform an A/A test. This means that you create two or more identical variations and run an A/B test to see how the platform handles the variations. Successful results should show that both variations yield very similar results. You can read further about [A/A tests](https://www.dynamicyield.com/glossary/aa-testing/) here. ## The road to A/B test success _“I didn’t fail the test, I just found 100 ways to do it wrong.”_ / Benjamin Franklin When running an A/B test, using a valid methodology is crucial for our ability to rely on the test results to produce better performance long after the test is over. In other words, we try to understand if tested changes directly affect visitor behavior or occur due to random chance. A/B testing provides a framework that allows us to measure the difference in visitor response between variations and, if detected, establishes statistical significance, and to some extent causation. ## Questions and answers What are the key considerations when designing an A/B test to ensure reliable results? When designing an A/B test, it’s crucial to define clear objectives and select the right metrics that align with these goals. Randomization is essential to avoid bias, and calculating the appropriate sample size ensures statistical significance. The test should run for an adequate duration to capture variations in user behavior over time. Additionally, controlling for external factors, such as marketing campaigns or seasonal trends, helps in obtaining reliable results. By addressing these considerations, businesses can ensure that their A/B tests provide meaningful and actionable insights. How can businesses effectively interpret the results of an A/B test to make data-driven decisions? Interpreting A/B test results involves statistical analysis to determine significance and calculating confidence intervals to understand the range of the true effect size. It’s important to consider the broader context, including user behavior patterns and external influences. Segmentation analysis can reveal variations in performance across different user groups. The insights gained should be translated into actionable strategies, and iterative testing should be used to refine hypotheses and optimize outcomes continuously. This approach ensures that businesses make informed, data-driven decisions. What are some advanced techniques in A/B testing that can enhance the accuracy and depth of insights? Advanced A/B testing techniques include multi-armed bandit testing, which dynamically allocates traffic to better-performing variations, and sequential testing, which allows for continuous monitoring and early stopping of tests. Bayesian methods provide a flexible approach to decision-making by updating outcome probabilities as data is collected. Personalization tailors variations to different user segments, and integrating machine learning algorithms can predict outcomes and optimize test designs. These techniques enhance the accuracy and depth of insights, leading to more effective optimization strategies. How can A/B testing be integrated into a broader digital marketing strategy? Integrating A/B testing into a digital marketing strategy involves aligning testing initiatives with business objectives and fostering cross-functional collaboration between marketing, product, and analytics teams. Data from A/B tests should be integrated with other marketing analytics to provide a comprehensive view of performance. A/B testing should be used as a tool for continuous improvement, regularly testing and optimizing various aspects of the digital experience. Developing a scalable testing framework allows for efficient execution and analysis of multiple tests, ensuring that insights are actionable and impactful. What ethical considerations should be taken into account when conducting A/B tests? Ethical considerations in A/B testing include ensuring user consent, especially when personal data is involved, and protecting user data in compliance with regulations like GDPR or CCPA. Transparency about the purpose of the tests and how data will be used is crucial. Tests should be designed to avoid causing harm or negative experiences for users, and fairness should be maintained to ensure no particular group of users is disadvantaged. By addressing these ethical considerations, businesses can conduct A/B tests responsibly and maintain user trust. What role does hypothesis formulation play in the success of an A/B test? Hypothesis formulation is a critical step in the A/B testing process as it provides a clear direction and purpose for the test. A well-defined hypothesis outlines the expected outcome and the rationale behind the changes being tested. This helps in setting measurable goals and ensures that the test is focused on addressing specific issues or opportunities. A strong hypothesis also aids in interpreting the results and making informed decisions based on the findings. What are the limitations of A/B testing, and how can they be addressed? While A/B testing is a powerful tool, it has limitations such as the potential for inconclusive results if the sample size is too small or the test duration is too short. Additionally, A/B testing may not account for long-term user behavior changes or external factors influencing results. To address these limitations, it’s important to ensure adequate sample sizes and test durations, complement A/B testing with other research methods, and continuously monitor and iterate on the findings to adapt to changing conditions. How can businesses ensure that their A/B testing practices are scalable and sustainable? To ensure scalability and sustainability in A/B testing practices, businesses should invest in robust testing platforms that automate the process and provide comprehensive analytics. Developing a structured testing framework with clear guidelines and best practices helps in maintaining consistency and efficiency. Training teams on the importance of A/B testing and fostering a culture of experimentation encourages continuous improvement. Regularly reviewing and updating testing strategies based on learnings and technological advancements ensures that the practices remain relevant and effective. ##### Continue reading --- # Free Bayesian A/B Testing Calculators — Dynamic Yield A while back, [we explored](https://www.dynamicyield.com/blog/bayesian-testing/) a less restrictive and more reliable approach to [A/B testing](https://www.dynamicyield.com/lesson/introduction-to-ab-testing/) in the form of a newer, Bayesian testing method. With its simplicity, reliability, and intuitiveness, the Bayesian framework is a superior A/B testing methodology which will provide marketers a quicker and more robust statistical engine. Therefore, we’ve been working long and hard to make it easier for marketers and [conversion rate](https://marketing.dynamicyield.com/benchmarks/conversion-rate/) optimizers to utilize this new approach. And for us, that’s meant taking the complex math out of the equation. So, today, we’re excited to announce the official release of two free and useful tools… ## Bayesian A/B Testing Calculator **Our [Bayesian-powered A/B testing calculator](https://marketing.dynamicyield.com/bayesian-calculator/)** will help you find out if your test results are statistically significant. For each variation you test, all you have to do is input the total sample size and number of conversions. Then, based on statistical significance, the statistical engine will declare a winning variation. Here’s a quick breakdown of the terms and metrics we run: - **Sample Size** – The number of users, sessions, or impressions depending on your KPI. - **Conversion** – The number of clicks, even purchases or goal completions (e.g. purchases or video views). - **Conversion Rate** – The number of completed actions (i.e. conversions) divided by the sample size. - **Probability to be Best** – Each variation’s long-term probability to outperform all other live variations (given collected data since the creation or change of any variation included in the test). - **Expected Loss** – The percent you are expected to lose in the long term if you declare the wrong variation as a winner versus the variation which is actually the best. - **Posterior Simulation of Difference** – The distribution of conversion rates given the sample size collected so far. To better understand your results, consider the winning variation above. Across the board, Variation A dominates in terms of conversion, conversion rate, Probability to Be The Best, as well as Expected Loss. Remember, the higher the PTBB and lower the EL, the better. This indicates confidence in long-term performance as well as lower expected loss if the variation declared a winner ends up being wrong (which it doesn’t look like it will be). Learn more about [the importance of statistical significance in A/B tests](https://www.dynamicyield.com/blog/statistical-significance/). ## Bayesian A/B Test Duration & Sample Size Calculator Not sure how long you will have to run your experiments in order to get statistically significant results? No worries. As the first of its kind, our free online **[Bayesian-powered A/B test duration](https://marketing.dynamicyield.com/ab-test-duration-calculator/) and sample size calculator** will help you avoid false positives and increase the validity of your A/B testing. There’s no hard limit on how many variations you can test against the control and we offer a few suggestions for how to approach your testing. The calculated output provides range estimations of the time required to run the test in order to get statistically significant results, and the minimum required sample size to support that. While we know there are many other variables involved, these calculated estimations can be used for planning experiments in advance, as well as for analyzing ongoing tests. Here’s a quick breakdown of the terms and metrics we run: - **Baseline conversion rate** – The current conversion rate (number of successful actions divided by the number of visitors, sessions, or impressions) for the experience you’re testing. - **Expected uplift in conversion rate** – The X% change in conversion rate you are aiming for from your baseline rate. In the example below, if the baseline conversion rate is 2.5%, a 5% expected uplift would result in a new conversion rate of 2.625%. - **Number of variations** – The number of variations compared in a single test. - **Average sample size per day** – The number of visitors to be served in the experiment over the course of one day. Now that you have everything you need to get started, we know you’ll be deploying Bayesian A/B tests at higher confidence intervals. And with a higher degree of self-confidence. It’s important to note that while these tools support binary objectives (user converted or not) when measuring conversion, the real statistical engine behind the Dynamic Yield platform behaves differently, and supports more advanced, non-binary calculations, such as maximizing completions number, or revenue. To learn more about our statistical engine and automated optimization approach, [request a product demo](https://www.dynamicyield.com/request-demo/). --- # How to Deliver a Less Frustrating Online Shopping Experience _This content originally appeared in our XP² newsletter._ [_Subscribe here_](https://www.dynamicyield.com/newsletter/) _to receive experience optimization insights like these, straight to your inbox._ Let’s say you’re out with friends when the question of where to go to dinner arises. A search on your phone for what’s “nearby” greets you with a buffet of options, and you spend what feels like forever deciding on the right one, only to give up and opt for a place you’ve been a million times before. This is the paradox of choice: The more choices we have, the more overwhelmed we feel. Eventually that overwhelm leads to “decision fatigue,” or settling for a subpar option because it’s easier. But this problem isn’t limited to restaurant selection. As eCommerce platforms rapidly expand, smaller brands are entering the fray alongside established, global retailers. Now that consumers have more to choose from than ever—and the online web experiences of these brands are largely identical—shoppers are struggling to find exactly what they want. This can negatively impact their business KPIs, especially as shoppers jump to competitors’ websites or [abandon their carts](https://www.dynamicyield.com/lesson/shopping-cart-abandonment-strategy/). Abandoned carts are abandoned revenue, so what can retailers do to combat decision fatigue? I chatted with Lior Delouya, Product Manager at Dynamic Yield by Mastercard, about streamlining the decision-making process with [personalization](https://www.dynamicyield.com/article/personalization-guide/). Drawing from his 8+ years of experience in leading teams in customer success, he has a unique perspective on alleviating choice overload. **JR: What are some factors that pile up and contribute to decision fatigue?** Lior: Even seemingly minor tasks can significantly impact the shopping experience. You may not think of the following as decisions, but they are: - Selecting filter options to narrow down product searches - Determining the value of product reviews - Evaluating promotional offers - Contemplating whether to wait until an item goes on sale - Considering an order with multiple items to reduce shipping costs - Finding a balance between price vs. quality - Choosing different payment methods at checkout - Selecting the right shipping method - Ensuring your items have the proper dimensions via size guides Each decision, however small, can collectively drain the consumer’s energy, leading to mental exhaustion and dissatisfaction with the shopping process over time. This exhaustion can spur customers to take long breaks (averaging about a week) in between product searches, according to a [joint paper from NYU and UCLA Anderson](https://anderson-review.ucla.edu/search-fatigue-online-shoppers-grow-weary-take-a-break/). That break time didn’t necessarily lead to a sale—76% of shoppers didn’t make a purchase after their initial click on a fashion website. **Why might people give up on buying when searching from one website to another?** The effort and time required to navigate multiple websites—and compare prices, features, and reviews across those sites—can frustrate shoppers, especially if the process doesn’t quickly lead to a gratifying decision. Some have the nagging feeling that they are paying too much, which can create a situation in which they obsessively search for a better deal. The problem is, this comparison process could go on endlessly, considering the wealth of options at our fingertips. Eventually, those shoppers become overwhelmed and paralyzed, abandon their shopping carts, and leave sites empty-handed. **Typically, expanding a brand’s product catalog is in its best interest. But how can brands achieve that goal without stressing out their customers?** Brands should strike a balance between offering a broad selection and organizing their offerings effectively. Of course, one of the best ways to do that is through personalization, which enables retailers to deliver bespoke [product recommendations](https://www.dynamicyield.com/lesson/product-recommendations-guide/) that streamline the shopping experience. Personalized conversational experiences can play a pivotal role here. They excel in providing inspiration and recommendations, which is particularly useful for customers who don’t have a specific product in mind. They can also mitigate the negative impact of a sprawling product catalog by ensuring customers don’t have to sift through irrelevant options to find what they’re looking for. By doing so, conversational AI also reduces the impact of the paradox of choice: Shoppers have a limited number of options to pick from, curated based on prior interactions and their own preferences. **How does AI help narrow down decisions for users with no prior data?** AI can start by making initial recommendations based on contextual data (such as geolocation or [local weather conditions](https://www.dynamicyield.com/lesson/weather-based-targeting/)), basic demographic information, and even comparisons with similar user profiles. It can then adapt when it interacts with the user. **Some believe AI is pulling us further away from the human experience—but it may actually be the other way around. How will the interplay between humans and AI evolve in the future?** As advancements in [deep learning](https://www.dynamicyield.com/lesson/deep-learning-recommendations/)—and [large language models (LLMs)](https://www.dynamicyield.com/article/generative-ai-personalization-marketing-transformation/) in particular—gain momentum, the online shopping experience will begin to resemble a brick-and-mortar store. Imagine a personal stylist who is not only well-versed in the latest trends and fits but also deeply understands your unique style preferences and needs, meticulously selecting items tailored just for you. With the power of AI, every customer will have access to an expert guide that can lead them through their purchasing journey. It could spell the end for our decision-heavy shopping woes—and the impersonal, transactional nature of shopping online. In many ways, [AI shopping assistants](https://www.dynamicyield.com/shopping-muse/) will be more intuitive than the older model of endlessly browsing [product listing pages (PLPs)](https://www.dynamicyield.com/glossary/product-listing-pages-plps/) once consumers try them out. We’re all used to searching for items in “Googlish”—brief two-word search queries—because of older technological limitations. We grew up with them. Now everything is turned upside down. We’ll have to relearn how to communicate with tech in a more human way. But [AI is driven by humans](https://www.dynamicyield.com/article/ai-personalization-human-advantage/) as well. It’s crucial for brands to nurture internal talent to operate and refine these technologies effectively. Investing in skilled professionals who can manage, analyze, and drive innovation with these systems will keep brands one step ahead of the competition over the coming years. **What advice do you have for companies interested in using AI-driven shopping assistants but hesitant about whether shoppers will embrace them?** As with any new technology, there’s an adoption curve. Well-defined entry points to the chat experience, targeted marketing campaigns that emphasize their usefulness, and clear, straightforward tutorials can help educate shoppers. Seamless integration into the online shopping environment is essential to ensure a smooth user experience. By showcasing how AI assistants can surpass the limitations of conventional search tools and offering incentives like discounts or personalized deals, companies can motivate users to embrace these innovative search methods, leading to a more efficient and satisfying shopping experience. ## Overcome Choice Overload with a Human-Centered Shopping Experience Decision fatigue is now a fact of modern life—and its especially pervasive in the eCommerce industry. As customers encounter a mounting number of infinitesimal shopping dilemmas, they are more apt to abandon their carts and take their business elsewhere. But the adoption of highly capable, empathetic AI shopping assistants could spell the end of consumers’ shopping woes—if retailers are willing to take the plunge. --- # Gartner Magic Quadrant for Personalization — Definition by Dynamic Yield Gartner Magic Quadrant for Personalization Engines refers to a specific Gartner report which evaluates competing players in the major technology market for personalization engines. Gartner is one of the world’s leading research and advisory companies, whose rigorous research process and proven methodologies have provided businesses with objective insights they need to make the right decisions. Every year, Gartner releases its Magic Quadrant, a report positioning technology players within a specific market. To date, we’ve seen solution providers from Web Content Management, Content Collaboration Platforms, CRM Customer Engagement Centers, Data Integration Tools and more experience the evaluation process. In its fourth consecutive market research report on Personalization Engines, the 2021 Gartner Magic Quadrant evaluated 12 solution providers based on their ability to execute as well as completeness of vision. ### Why Personalization? According to Gartner: “ _Personalization remains a priority for digital marketing leaders. Relevant and timely messaging is key to educating customers, minimizing friction and building purchase consideration. Use this research to assess personalization engines that will enable you to deliver measurable results.”_ — Gartner ### How Does Gartner Define Personalization Engines? _Personalization Engines are software that enables marketers to identify, deliver and measure the optimum experience for an individual customer or prospect based on their past interactions, current context and predicted intent. Personalization engines help marketers identify, select, tailor and deliver messaging such as content, offers and other interactions across customer touchpoints in support of three primary use cases: marketing, digital commerce, and service and support._ — Gartner ### What Evaluation Criteria does Gartner Use? By applying its standard Magic Quadrant graphical treatment and a uniform set of evaluation criteria, Gartner’s differentiated each personalization engine by: **Leaders** execute well against their current vision and are well positioned for tomorrow. **Visionaries** understand where the market is going or have a vision for changing market rules, but do not yet execute well. **Niche Players** focus successfully on a small segment, or are unfocused and do not out-innovate or outperform others. **Challengers** execute well today or may dominate a large segment, but do not demonstrate an understanding of market direction. _“Vendors are judged on Gartner’s view of their ability and success in making their vision a market reality that customers believe is differentiated and are prepared to buy into. Delivering a positive customer experience — including sales experience, support, product quality, user enablement, availability of skills and ease of upgrade/migration — also determines a vendor’s Ability to Execute.”_ — Gartner Gartner has positioned Dynamic Yield in the Leaders quadrant for the fourth consecutive year based on our ability to execute and completeness of vision in its [annual Magic Quadrant for Personalization Engines](https://www.dynamicyield.com/guides/gartner/), so here’s a little bit about what we do best and why we believe we were recognized as a Leader in the Magic Quadrant: 1. **Open architecture** – Prioritizes flexibility, [security](https://dynamicyield.com/security), connectivity, and [governance](https://www.dynamicyield.com/enterprise-grade-personalization/) 2. **Best-in-class algorithms** – Predict customer intent and affinity in real-time through self-trained [deep learning recommendation](https://www.dynamicyield.com/adaptml/) models. 3. **Ease of use and agility** – Empowers teams to start small, then scale across a brand’s channels at their own pace and deploy personalization and [A/B testing](https://www.dynamicyield.com/ab-testing/) where it will drive impact. 4. **Superior UI and streamlined workflows** – Allows marketers to run [omnichannel personalization](https://www.dynamicyield.com/personalization/) programs across web, mobile, email, and ads, through a simple UI and streamlined workflows – without having to rely on developers. To many, this report confirms the increasing reliance on personalization technology by marketing leaders for business advantage, solidifying its place within the marketing landscape at large. _Gartner, Magic Quadrant for Personalization Engines, 19 July 2021, Jason McNellis, Claire Tassin, Jennifer Polk._ _Gartner and Magic Quadrant are registered trademarks of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission. All rights reserved. Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose._ _GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally, and is used herein with permission. All rights reserved._