Introducing Tokyo Otaku Mode’s growth hacking techniques - A/B testing examples -

Hi. This is Hajime, COO of Tokyo Otaku Mode Inc.

Welcome to our official blog. My colleagues and I will be sharing regularly on how we work here at Tokyo Otaku Mode. I hope you enjoy it!

Japanese people frequently describe Tokyo Otaku Mode (‘TOM’) as an enigmatic company when asked about it. Although we have over 15 million fans on Facebook, only less than 1% are from Japan. In order to help everyone understand what we do, we have started this blog to introduce the techniques and methods that our team uses to support and grow TOM.

To start things rolling, I’d like to talk about TOM’s growth hacking strategies.

TOM started its Facebook page on March 24, 2011 and our main objective is to share with the world the latest Japanese pop culture news about anime, manga, cosplay, etc. Taking advantage of Facebook’s growing popularity and the internet’s hunger for more otaku culture, we now have a Facebook fanbase of over 15.5 million (Thank you very much!).

Despite having over 15 million followers from around the world on our Facebook page, it is extremely risky to run a company which offers services that depend entirely on this social network. Therefore, we launched otakumode.com (‘.com’), our own native web site.

We want to grow .com rapidly with our large Facebook following. In order to achieve that, we focused on converting each of our Facebook fans into a .com user.

More specifically, we post trending content on Facebook that attracts ‘likes’ and ‘shares’, and then re-direct users to the web site by using a ‘read more on .com’ message. Just before the full content is displayed, users are prompted to register.

image

To simplify the explanation, let’s just say our goal here is to lead the user to complete our member registration process. We want to optimize the appearance and functionality of the website to prevent abandonment. As you can see in the flowchart above, if we can prevent people from abandoning the registration process, the rate of membership conversion will increase.

Starting from a potentially repercussive Facebook post is an extremely effective way to increase that conversion. In TOM, we have a set KPI for each post. We quantify all our post repercussions and analyze our data in detail. We put a lot of effort in creating and timing our Facebook posts. I’ll explain more about how we produce our viral content in a future blog post.

In any case, for today I am going to introduce to you how we increase membership conversion by optimizing the process of going to .com from Facebook.

image

First, we held a kickoff meeting with engineers and designers to decide on the critical components. We came up with the following list of things that we wanted to verify with regards to the login page.

<Things to bear in mind>

  • Logo size
  • White background
  • Background opacity
  • Site loading speed
  • Click location
  • Login button color
  • Login button shape
  • Login button name
  • Login button animation
  • The entire layout
    And many more.

After that, we made many wireframes based on the list above and selected the most optimal combination using an A/B test. We needed accurate and reliable data from the A/B tests so we determined the results using a 95% confidence interval (Google to learn more about A/B testing and confidence intervals).

Now for the results - here’s a quiz for you! Which turned out to be the most effective version (in terms of registration rates) for each A/B test we conducted?

First question:
This test will determine the optimal transparency to partially hide the contents of Facebook’s post which are in the background. The percentage of people accessing the site will depend on the viewable content in Facebook. What is your answer?

First question

Answer:
The test results showed that the optimal transparency was 80%. Additionally, we refined the results by testing the opacity in intervals of 5% and found that our optimal transparency was 90%.

Answer

Second question:
We believe that there is an optimal shape that would lead to the most number of click-throughs. We tested the visibility and clickability of the ‘Log in with Facebook’ button. Which do you think was the most effective button?

Second question

Answer:
The final result was B, the flat and elliptical button. I personally thought that the 3-D buttons would lead users to click on them so I was surprised by our test results. This goes to show that A/B tests are very useful for an objective take on things.

Answer

All right, last question:
We want to decide on an overall layout. Which layout did the best in our A/B test?

last question

Answer,

B and C were victorious by far. Just to be sure, we compared the registration rates instead of just clicks. Less is more. It is always important to not confuse your users..

Answer

These are the results after rounds of testing in .com. You would have to conduct testing for different products and services to find the optimal setting for them. However, I think with this you would have a better idea of how A/B testing work. Here at TOM we are looking for growth hackers to join our ranks. If you are a growth hacking ninja, click here to join our ranks.

From now on we’ll be regularly updating this blog. If you had enjoyed our first post, do subscribe to our RSS feed. I hope you’ll be looking forward to our next update!