Motorcycle News A/B Split test
Skills: UX Design, Figma, HTML5, CSS3, Bootstrap4, Google Analytics
Timeline: 1 week
Duration of the test: 1 month
Objective:
To create a test with two different paywalls showing on the article pages on MCN, with one paywall advertising joining free for one month and the other to pay 99p for a one month's subscription, and seeing which paywall was the most successful in users signing up.
Step 1. Decide upon the two tests to be used
In order to undertake the test, I put forward the idea of the two paywall payment amounts and this had to be agreed by the senior team.
Step 2. Create the A/B split test
The 3rd party software used by the members sites allowed us to create a split test and the percentage each test should be shown to the user. I created the test to enable each paywall to be shown randomly 50% of the time.
Step 3. Design the two paywalls
The paywall would use the same as the existing design although there would be slight changes to the payment amounts. After researching by looking at paywall designs used by other competitor, I felt it was important to emphasise the fact that the user could cancel the deal at anytime as this would provide some comfort to the users if they didn't want to continue the deal. These are the two designs of the paywalls:
Step 4. Code changes
The languages I used to create the changes live to the site were:
HTML5 | CSS3 | JavaScript(vanilla) | Bootstrap 4 | mdBootstrap
Step 5. Gathering the data in Google Analytics and in the 3rd party software
The data in the 3rd party software enabled me to see how many times each paywall was shown to the user. I had also created custom dimensions in Google Analytics to show how successful the user journey was through paywall in users signing up. The custom dimensions were placed on each of the CTA's and then as the user entered the payment process.
Step 6. Analysing the end result of the test to see how successful the test was.
The test was successful as it was found 69% of users who was shown the free month paywall signed up and continued their membership after the trial ended. This has encouraged us to try the offer again in the future.
