Robert Dyas chose to use Nosto’s A/B Testing & Optimisation to uncover valuable insights into how different onsite experiences affect the way customers behave onsite. To find out how category and brand affinity affects the way customers engage and convert, they deployed tests on their homepage and product detail pages in order to directly compare these variations in experiences against their status quo.
Test 1: Recommending Products Based on Brand Affinity on Product Description Pages
The first test, which was aimed at discovering how brand affinity affects the effectiveness of product recommendations on PDPs, was set up to compare product recommendations that matched the brand of the product whose page the shopper was currently on to recommendations that varied in brand but focused on similar products. The purpose of the brand affinity test was to determine if brand affinity is a strong indicator of how effective product recommendations will be on a PDP. To yield the highest test engagement, the test was conducted on the YMAL block on the product description page—a high-traffic area of the retailer’s website. Two variations were set:
- Variation A which showcased standard YMAL recommendations
- Variation B which displayed the YMAL filtered by brand of the viewed product which allowed the customer to view alternative products from the same brand.
The tests results would answer the critical question of whether brand affinity or product utility is more important when shoppers are shopping at a multi-brand homeware retailer.
‘Salter’ pan, sold by merchant:
Test #2: Personalising the Home Page with Category Specific Banners
For the second test, Robert Dyas tested how personalised homepage experiences affect shopper engagement compared to their standard homepage experience. Robert Dyas wanted to test if personalised homepage experiences caused different audience segments to engage more. To form the test, the retailer created different variations of their homepage content that had messaging and visuals directed toward specific categories that were tied to the customer’s previous behaviour.
Homepage general banner:
Homepage deals segmented banner:
Test #1 Results: Brand Affinity on Product Description Pages
Overall, customers were 28% more likely to convert when exposed to a recommendation block containing products that were the same brand as the product whose detail page the shopper was currently on. Each segment saw an increase in conversion performance with the new variation and the prospect segment had an exceptional 30% increase in conversion compared to the standard variation. Incidentally, the retailer did not think that this would be the case—which is a critical example of the impact testing can have on a retailer’s performance expectations.
Test # 2 Results: Category Specific Banners on the Home Page Results
Introducing the same kind of campaign test for multiple different segments can be a strong indicator that not all groups of shoppers behave in the same way—which allows for more streamlined content segmentation. Personalised messaging won out for the category ‘Kitchen Electricals’ by a margin of 88%, whereas the shoppers in the ‘Kitchenware’ segment were 78% more likely to convert on the default content. This shows a successful testing strategy involves more than simply setting up and running a test.
‘Kitchenware’ banner test results:
‘Kitchen Electricals’ Banner Results:
