Last week we profiled five apps for split testing (or A/B testing) that cater to those with little or no development expertise. The tools profiled in that summary are attractive for their ease of use, straightforward implementation, and simple dashboards.
Those tools generally work by dropping a line of Javascript into your site and changing the visitor experience at the browser level. While they’re easy to install and maintain, there are some potential disadvantages for power users looking to do beta testing, price testing, and other types of experiments. Here’s a great summary by Phil Whelan:
Managing [split testing] in JavaScript makes it easy to setup, but it also means that all the A/B versions are available to all users and are just obscured by the browser. The server has to potentially render the full content for both versions for each visitor. This is not ideal if you are testing a beta-version of a feature and you do not want it to affect all your users. Or you are testing pricing or new features where you really do not want any chance of someone hacking your JavaScript to expose the unpublished features. Your marketing team would not be happy.
In other words, some situations require more advanced frameworks to complete sophisticated split testing. Today we’re highlighting split testing programs that require a bit more up front investment from a dev perspective, but that have the capability to offer up some enhanced flexibility, functionality, and security once up and running.
If you’re not familiar with the concept of split testing, check out this recent article by Eric Siu. For inspiration on split testing possibilities once you get a proper infrastructure in place, this piece by Neil Patel is a good place to start. (From there, you’ll probably want to check out our suggestions for dead simple split testing apps.) And for even more tips on split testing and other Web monetization tips, subscribe to our free email newsletter.
Got nothing to split-test? Start your blog today!
If you’re looking to get more advanced, read on:
App #1: Split
This Rack-based split testing framework is designed to work with Rails, Sinatra, or any other Rack-based app. This tool is designed to be developer friendly, allowing for maximum customization.
The Split wiki has all sorts of documentation, including a step-by-step tutorial to get you up and running. Once you’ve got Split set up, you’ll have an extremely flexible interface for running A/B tests:
App #2: Vanity
Vanity is an experiment-driven development framework for Rails. This tool can be used to split test prices, new features, tweaks to existing features, and just about everything else imaginable. The Vanity dashboard can be tweaked to include Google Analytics.
Similar to Split, there’s a ton of documentation on how to get the most out of this framework, starting with the 2-minute demo and including details on defining metrics and interpreting results:
Here’s a look at a sample pricing experiment:
App #3: A/Bingo
A/Bingo is an open source split testing framework developed by Bingo Card Creator. Some nice features include a collection of actual split test results and a presentation on how to make your own A/B testing framework. There’s also a screencast that will walk through the steps needed to get up and running.
An additional line of code allows users to test for statistical significance of experiments, helping to indicate when you’ve achieved a large enough sample size to wrap up your test. Check out this comparison of A/Bingo vs. more traditional split testing apps.
There’s also a variation of this testing framework developed specifically for the Khan Academy. GAE Bingo includes many of the same features as well as performance optimized for App Engine and a Javascript API. GAE Bingo is framework agnostic; it works with Django, Flask, and countless others.
App #4: Genetify
Genetify, based on the concepts of genetic algorithms and evolutionary computation, works a bit differently than most split testing tools. Specifically, this framework scans a selected site for specifically named groups of elements, and then varies those to produce different results. Genetify is powered by an algorithm that “basically says that the weight given to a variant is equal to its share of past goals plus a declining constant.”
It works with CSS and Javascript, and can search through a matrix of variables to come up with the best combination (for example, of text and images). There’s an interactive demo of the capabilities, as well as a summary by John Resig that walks through some use cases.
App #5: FluidFeatures
The aforementioned Phil Whelan developed this Ruby on Rails testing framework. FluidFeatures allows users to set custom breakdowns of variations; for example, you’ll be able to test a new SQL query against 5% of your user base if you want to determine whether it’s faster than your existing process.
Once installed, a dashboard lets you add new tests and goals, and monitor the progress:
Check out the video tutorial with step-by-step instructions to get up and running on FluidFeatures.
Bottom Line
No matter what your level of sophistication or needs, odds are there’s a split testing app out there that can help you more efficiently monetize your site. Take a look at the ten tools we’ve profiled (the other five are here), find one that fits your needs, and start testing!