How a one word change increased product demo conversions by 139%

This post looks at an A/B test with a simple copy change and how it improved conversion rates by 139%. The idea behind the A/B test was to give users immediate access to the product, via a recorded demo, instead of having them receive a personal phone call from our sales team later on.

Theory

Using a framework, when A/B testing potential improvements to landing pages, is helpful. Sean Ellis has a simple one for understanding the broad levers that can been affected: Conversion Rate = Desire - Friction. Desire and Friction can be further broken out and the LIFT Model by WiderFunnel describes this well:

Hypothesis

Our goal was to improve the conversion rate for demo requests so that customers could access the content they were interested in as soon as possible and with minimal friction. We wondered if the “Request a demo” button might be causing some users anxiety (as described by the LIFT model) and, as such, artificially lowering conversion rates. We then tested whether the wording “Watch a demo” would outperform the original “Request a demo” wording.

We had further reason to believe that immediate access to a recorded demo would be beneficial as only 1/5th of leads ended up watching the live demo which has been scheduled by our sales team.

Website Modifications

The original GoCardless user experience with the “Request a demo” was the following:

• Call to action (CTA) on the homepage was “Request a demo”

• Users were taken to a request a demo form to fill out and submit

• Upon completing the form, users were given a date and time of an upcoming live demo that they could participate in.

We altered the user experience by giving users immediate access to a recorded demo:

• Call to action (CTA) on the homepage is “Watch a demo”

• Users are taken to a “Watch a demo” form to fill out and submit

• Upon completing the form, users are shown a 10 minute recorded demo in their browser

Most of these changes were implemented within Optimizely’s Multi-page Experiments feature (aka Conversion Funnel Testing). However, we did build the recorded demo page ourselves as we didn’t need to A/B test this page directly.

Although some time and effort went into thinking about reducing friction, the work required to implement and instrument these changes was very easy because of Optimizely and Mixpanel. Optimizely has a very useful single toggle option for sending super properties to Mixpanel so that we can track what happens deep within our funnel.

Results

Given our acquisition channel characteristics, we ran the A/B test for a full 7 days. We then looked for a statistically significant winner with at least a 95% confidence level. Optimizely’s report panel below shows that the “Watch” version consistently outperformed the original version:

We also ran the numbers through Mixpanel’s split test calculator based on event data we track in our conversion funnels:

This shows that the “Watch a demo” version is more than twice as effective as the “Request a demo” version (139% increase in conversion). With this simple copy change, derived from the idea of reducing friction for new users, we’ve dramatically increased the number of users who watch a product demo and are therefore more likely to become customers.