Online A/B testing
Make an objective decision about which version works best
What is online A/B testing?
Online A/B testing is a form of usability testing where we test two or more versions of a webpage or screen to identify which version works best. At the end of the test, we'll provide a detailed report outlining the findings of the test, our assessment of the usability of the preferred version, and recommendations for improving its usability.
Benefits of online A/B testing
While you can conduct A/B testing by simply uploading the new version of your site or app to your business web address and monitoring your 'before' and 'after' business statistics, this is a highly risky approach for your business. If your site or app is not very usable, your business is likely to suffer the immediate and direct effects of the lack of usability in the form of a drop in sales, a decrease in customer satisfaction, an increase customer support calls, and negative PR coverage.
By contrast, our online A/B testing service enables you to test your site or app at a test location that is not linked to your business web address, making it a low-risk way of getting feedback from a pool of potential users without jeopardising your business income or alienating your existing customer base. You'll find out if your proposed changes will actually improve or reduce the usability of the site or app before you develop or launch the new version.
Online A/B testing enables you to:
- Make an objective decision between two or more versions of a webpage or screen
- Get users' reactions to proposed designs before coding begins
- Get user feedback about labelling, navigation, and expected content/functionality
- Assess if a proposed change improves or reduces the site or app's usability
- Find out which features users like and dislike about competitors' sites or apps, to inform the design of your own site or app
When to test
It's never too early to test. Online A/B testing can be used at multiple points of the development cycle:
- During the website planning phase, to identify possible features and content for your own site or app, based on user feedback gained by conducting an A/B test of your competitors' sites or apps.
- During the design/pre-coding phase, to evaluate two or more potential designs, by conducting an A/B test with mockups or sketches (known as wireframes) of the proposed webpages or app screens. Testing with mockups or sketches before a single line of code is written can significantly reduce your development costs and time.
- After the coding phase, to test if the changes improved or reduced the usability of your site or app.
Your site or app does not need to be public to allow us to conduct A/B testing—as long as it's possible to access the site or app (or the mockups/sketches of the webpages or screens) remotely, we can conduct an online A/B test.
What it involves
When you contact us to conduct online A/B testing, the process generally runs as follows:
- Free, no-obligation quotation. We're always happy to provide a no-obligation quotation. If, like most customers, you need to get user feedback on your site or app sooner rather than later, we can talk through your test over the phone or send you our information-gathering questionnaire by email.
- Designing the test. We review your test requirements and design an appropriate test for your site or app.
- Running the test. We conduct the test with an agreed number of test participants recruited by us or selected from your own customer base.
- Documenting the results. We produce a detailed report outlining the findings of the test, our assessment of the usability of the preferred version, and recommendations for improving its usability.
How to get started
To request your free, no-obligation quotation or to simply find out more about our online A/B testing service, just call us at +353 (0)1 5475145 or email hello@userstory.ie and one of our usability analysts will be happy to help you.
Steve McConnell, 1996.
Rapid Development: Taming Wild Software Schedules
"...a requirements defect that is left undetected until construction or maintenance will cost 50 to 200 times
as much to fix as it would have cost to fix at requirements time.