
Background
Hypothesis
By making information regarding subscription easier to discover at PDP once the option has been selected, we expect to see an uplift in the percentage of subscription orders.Design Changes
- Subscribe option has updated purple background colour when selected.
- “ i “ icon is removed on the far right of the subscription option.
- New content slot appears below the subscription option when it’s selected.
- Title for content slot: “ What is a vuse subscription?”
- “More info” CTA should trigger the same modal as used in control, which is triggered using the “ i “ icon.
- If users changes the selection back to one time, then the subscription content slot should be removed and the purple state is also removed.
- Functionality is identical for desktop and mobile devices.
Targeting – Pages
- UK Flavours PDPs
Test Goal
- Conversion Rate
- Subscription Orders
- Add to Basket
- PDP Subscription Modal Interaction
Devices
- Mobile
- Desktop
Audiences
- All visitors
#1. Which one do you think is the winner?
Results
Congrats! You got it right.
Winner: Variation 1
Start: 12/01/22
End: 07/02/22
Duration: 28 days
Control visitors: 8,244
Variation visitors: 8,338
Highlights
- We observed an uplift in the primary metric of subscription orders of 5.65% vs. the control at 88% statistical significance. We believe this is as a direct result of the introduction of the grey content slot added to product pages when the subscription purchase option was selected.
- The uplift in subscription orders impacted overall transactions, which were 6.5% vs. control at 92% statistical significance. This was ultimately reflected in an uplift in overall revenue per visitor of 14.7% from £3.20 to £3.67 because of the increase in subscriptions.
Recommendations
There is clear evidence that the addition of the subscription info content slot on PDP for users that have selected the subscription option drives a significant uplift in subscription orders and therefore overall transactions.
The 6.5% uplift in overall transactions observed in this experiment is worth an estimated £40.5k in annual incremental revenue, from an additional 1.1k transactions. This has been calculated by applying the uplift in conversion rate to the last 12 months of GA revenue data for flavour PDP visitors. This calculation assumes that the volume of traffic will be level year on year. We recommend a follow up experiment to highlight the new subscription info content slot to all PDP visitors rather than just those that have selected the subscription option. We believe this could result in an even higher uplift in subscription orders due to showing the benefits of a subscription to visitors that have not yet showed an interest in this purchase method.
Unfortunately, you didn’t get it right. Variation 1 was the winner.
Winner: Variation 1
Start: 12/01/22
End: 07/02/22
Duration: 28 days
Control visitors: 8,244
Variation visitors: 8,338
Highlights
- We observed an uplift in the primary metric of subscription orders of 5.65% vs. the control at 88% statistical significance. We believe this is as a direct result of the introduction of the grey content slot added to product pages when the subscription purchase option was selected.
- The uplift in subscription orders impacted overall transactions, which were 6.5% vs. control at 92% statistical significance. This was ultimately reflected in an uplift in overall revenue per visitor of 14.7% from £3.20 to £3.67 because of the increase in subscriptions.
Recommendations
There is clear evidence that the addition of the subscription info content slot on PDP for users that have selected the subscription option drives a significant uplift in subscription orders and therefore overall transactions.
The 6.5% uplift in overall transactions observed in this experiment is worth an estimated £40.5k in annual incremental revenue, from an additional 1.1k transactions. This has been calculated by applying the uplift in conversion rate to the last 12 months of GA revenue data for flavour PDP visitors. This calculation assumes that the volume of traffic will be level year on year. We recommend a follow up experiment to highlight the new subscription info content slot to all PDP visitors rather than just those that have selected the subscription option. We believe this could result in an even higher uplift in subscription orders due to showing the benefits of a subscription to visitors that have not yet showed an interest in this purchase method.
Get a copy of the full report
You can download the full report in pdf format from the link down below.

