Second-screen application zeebox is using A/B testing to segment specific groups of consumers who are watching TV and plugged into their mobile devices at the same time.

Since launching in the United States in 2012, zeebox is now moving towards becoming more of a social community for watching television shows versus solely focusing on the second-screen experience, and the company is using A/B testing to better understand which features consumers are most interested in. Zeebox is working with Swrve to run the A/B tests within its iPhone and Android app.

?There are two parts to A/B testing ? one of them is obviously testing A and B and using analytics to change to the right answer in your next release,? said Anthony Rose, cofounder and chief technology officer at zeebox, London.

?And secondly, it?s to actually even avoid having to do a new release, which is to build A and B options into a given mobile app and then use configuration to choose whether A or B is served to users,? he said. ?It?s that which will really help avoid some of the frustrations with having to commit to an app that has been locked in the market for several weeks and avoid the pitfalls of getting it wrong.?

Second-screen engagement
Zeebox is currently testing top versus menu navigation in the Android app. The company initially found that consumers were less receptive to accessing navigation through the menu tab even though the homepage navigation takes up more space.

The company is still running this navigation test and will make a decision on which feature to use in the future, but the theory is that certain groups of consumers favor utility-based features to be prominent within the app instead of being hidden behind a menu button.

Another test played up social media and encouraged consumers to share a piece of content across their networks. This resulted in a 570 percent increase in engagement within the app, showing how a direct call-to-action can be effective in getting consumers to interact with content.

Additionally, zeebox may test the copy within a button that reminds consumers when a show is starting. 

One set of copy may read, ?Remind me? while another says, ?Bookings.? Zeebox then splits up the test so that a certain percentage of users are served one message or the other to see which performs better.

Zeebox runs about 10 different tests at a time to better understand how consumers use the app. 

Customized messages
To take A/B testing up a notch, zeebox can then theoretically serve only a select group of consumers a specific message.

For instance, the app acts as a remote control for Comcast set-top boxes. The company would like to segment out Comcast subscribers specifically with a message prompting them to use the app to control their TV.

Besides zeebox, Rue La La and Urban Outfitters are two other examples of brands leveraging A/B testing to retain users and personalize content.

In November, Rue La La reported a 35-40 percent increase in conversions as a result of tweaking in-app creative (see story).

Additionally, Urban Outfitters added a newsfeed to its app in December, resulting in a 30 percent engagement rate (see story). 

The idea is that marketers already pouring money into developing apps now need to find creative ways to push consumers back into customized content consistently.

?What we believe is the marketer needs to fully be in the loop so that the app can be changed, it can be tested, it can be varied, it can be personalized after it?s been in the app store and after it?s been downloaded,? said Hugh Reynolds, CEO of Swrve, San Francisco.

?We shouldn?t be aiming for this one microsite,? he said. ?We should be able to have the app respond and actually cover a greater section of the audience as they come in. That?s really the idea of hyper personalization ? you target groups and then you hyper personalize the app experimentally.?

Final Take
Lauren Johnson is associate reporter on Mobile Marketer, New York