16 Comments

June 11, 2013 ,

 Updated October 7, 2021

Last week we profiled five apps for split testing (or A/B testing) that cater to those with little or no development expertise. The tools profiled in that summary are attractive for their ease of use, straightforward implementation, and simple dashboards.

Those tools generally work by dropping a line of Javascript into your site and changing the visitor experience at the browser level. While they’re easy to install and maintain, there are some potential disadvantages for power users looking to do beta testing, price testing, and other types of experiments. Here’s a great summary by Phil Whelan:

Managing [split testing] in JavaScript makes it easy to setup, but it also means that all the A/B versions are available to all users and are just obscured by the browser. The server has to potentially render the full content for both versions for each visitor. This is not ideal if you are testing a beta-version of a feature and you do not want it to affect all your users. Or you are testing pricing or new features where you really do not want any chance of someone hacking your JavaScript to expose the unpublished features. Your marketing team would not be happy.

In other words, some situations require more advanced frameworks to complete sophisticated split testing. Today we’re highlighting split testing programs that require a bit more up front investment from a dev perspective, but that have the capability to offer up some enhanced flexibility, functionality, and security once up and running.

If you’re not familiar with the concept of split testing, check out this recent article by Eric Siu. For inspiration on split testing possibilities once you get a proper infrastructure in place, this piece by Neil Patel is a good place to start. (From there, you’ll probably want to check out our suggestions for dead simple split testing apps.) And for even more tips on split testing and other Web monetization tips, subscribe to our free email newsletter.

Got nothing to split-test? Start your blog today!

If you’re looking to get more advanced, read on:

App #1: Split

This Rack-based split testing framework is designed to work with Rails, Sinatra, or any other Rack-based app. This tool is designed to be developer friendly, allowing for maximum customization.

The Split wiki has all sorts of documentation, including a step-by-step tutorial to get you up and running. Once you’ve got Split set up, you’ll have an extremely flexible interface for running A/B tests:

Split in action

App #2: Vanity

Vanity is an experiment-driven development framework for Rails. This tool can be used to split test prices, new features, tweaks to existing features, and just about everything else imaginable. The Vanity dashboard can be tweaked to include Google Analytics.

Similar to Split, there’s a ton of documentation on how to get the most out of this framework, starting with the 2-minute demo and including details on defining metrics and interpreting results:

Vanity In Action

Here’s a look at a sample pricing experiment:

Vanity Pricing Tests

App #3: A/Bingo

A/Bingo is an open source split testing framework developed by Bingo Card Creator. Some nice features include a collection of actual split test results and a presentation on how to make your own A/B testing framework. There’s also a screencast that will walk through the steps needed to get up and running.

An additional line of code allows users to test for statistical significance of experiments, helping to indicate when you’ve achieved a large enough sample size to wrap up your test. Check out this comparison of A/Bingo vs. more traditional split testing apps.

ABingo-Screencast

There’s also a variation of this testing framework developed specifically for the Khan Academy. GAE Bingo includes many of the same features as well as performance optimized for App Engine and a Javascript API. GAE Bingo is framework agnostic; it works with Django, Flask, and countless others.

App #4: Genetify

Genetify, based on the concepts of genetic algorithms and evolutionary computation, works a bit differently than most split testing tools. Specifically, this framework scans a selected site for specifically named groups of elements, and then varies those to produce different results. Genetify is powered by an algorithm that “basically says that the weight given to a variant is equal to its share of past goals plus a declining constant.”

It works with CSS and Javascript, and can search through a matrix of variables to come up with the best combination (for example, of text and images). There’s an interactive demo of the capabilities, as well as a summary by John Resig that walks through some use cases.

Genetify

App #5: FluidFeatures

The aforementioned Phil Whelan developed this Ruby on Rails testing framework. FluidFeatures allows users to set custom breakdowns of variations; for example, you’ll be able to test a new SQL query against 5% of your user base if you want to determine whether it’s faster than your existing process.

Once installed, a dashboard lets you add new tests and goals, and monitor the progress:

FluidFeatures Dashboard

Check out the video tutorial with step-by-step instructions to get up and running on FluidFeatures.

Bottom Line

No matter what your level of sophistication or needs, odds are there’s a split testing app out there that can help you more efficiently monetize your site. Take a look at the ten tools we’ve profiled (the other five are here), find one that fits your needs, and start testing!

  • Everyone loves what you guys tend to be up too. This sort of clever work and exposure!
    Keep up the good works guys I’ve added you guys to our blogroll.

  • Hello there! Do you have Twitter? I’d like to follow you if that would be
    ok. I’m undoubtedly enjoying your blog and count on fresh
    posts.

  • Hello Deb,

    Yes, you can find us on Twitter @MonetizePros.

    Follow us and you’ll never miss a post!

  • Hi! Would you mind if I share your blog with my Facebook group?
    There’s a lot of people that I think would really enjoy your content.
    Please let me know. Cheers

  • Hi Cynthia! Yes, you are very welcome to share our content on social media!
    And don’t forget to sign up to our newsletter.

  • I’d like to thank you for the efforts you have put in writing this site.
    I really hope to view the same high-grade content from you later on as
    well. In fact, your creative writing abilities has encouraged me to get my very
    own blog now 😉

  • My programmer is trying to convince me to
    move to .net from PHP. I have always disliked
    the idea because of the costs. But he’s tryiong
    none the less. I’ve been using WordPress on a variety of websites for about a year and am anxious about switching to another platform.
    I have heard fantastic things about blogengine.net.

    Is there a way I can transfer all my wordpress posts into it?
    Any help would be really appreciated!

  • I have needed to contact fitbit a number of instances up to
    now and each time I obtain a less than heelpful non reply to my query.

  • Excellent goods from you, man. I’ve be mindful your stuff previous to and you are just extremely magnificent.

    I really like what you’ve received right here, certainly like what you’re stating and the way by which you assert it.
    You make it entertaining and you continue to care for to keep it smart.
    I cant wait to read far more from you. This is actually a
    tremendous website.

  • What’s up all, here every person is sharing such knowledge,
    therefore it’s pleasant to read this website, and I used
    to visit this webpage everyday.

  • {"email":"Email address invalid","url":"Website address invalid","required":"Required field missing"}

    Sign up for How to Sell on Shopify

    Get access to our FREE full Shopify Course and product monetization. 

    >