As "a simple experiment," Obama brought $ 60 million [translation] Bashny.Net

October 21, 2010, President Barack Obama visited Palo Alto. To attract money to his campaign was organized dinner party participation cost is 30 $ 400. This is an extremely effective method of raising funds, though it may work only if you have the opportunity to invite the President. How can raise funds, if you only have a website that is not seen no one but your grandmother? It is in this situation, we were back in 2007, when Obama was nominated for president, and the gap from the nearest competitor double-digit percent.



In 2008, when I worked as the director of the analytical department of the Obama administration, in my duties included making decisions based on data analysis. We started with a simple experiment in December 2007. On the basis of this experiment, we concluded that every visitor of our website was a unique "opportunity to raise funds" and, more importantly, taking advantage of this "opportunity" through optimization and A / B testing, we can draw the tens of millions dollars i>.

This experiment tested the two blocks of our site: block with image and video (Media section) and power button (Call-to-action).

For the experiment, we used the Google Website Optimizer and spent a full factorial multivariate test. In simple words, we tested all combinations of buttons and media content. We had 4 different buttons, 3 images and 3 video, which meant 24 (4 * 6) combination for the test. Each visitor to the landing page at random demonstrated one of the combinations, and we, in turn, track the number of visitors who have left their e-mail.



Before you see the results, try to suggest what button and media content in your opinion have been most successful?

Results h4> metrics that we use to measure the success of the experiment, the pace was getting subscriptions. To be precise, it is the number of people subscribing, divided by the number of people who saw a certain version of the page. Since the total time for the experiment on the landing page has gone 310,382 visitors, this means that each option has seen approximately 13 000 people.

These are the values ​​we observed for each of the variants of the landing page:

And these values ​​we observed in the evaluation of combinations of different options:

Winner h4> The best combination of buttons and media content has become «Combination 11", which consisted of a button «Learn more» photos and family:

Before we started the experiment, the people working on the campaign, chose «Sam's Video» (the last in the list of the slideshow above). If we had not started the experiment, we would likely just put this video on our landing page. As a result, such a trifle could be a fatal mistake, because as it turned out, any of the movies affects the conversion is weaker than any image.

11, 6% - that such a conversion have been subscription winning embodiment, unlike 8, 26% conversion of the original page. Thus, we managed to get to 40, 6% more signatures. Let's try to understand where this figure. If we assume that this effect will keep improving throughout the campaign, then later, towards the end, we can estimate the total figures and identify the changes that have led to this particular experiment. Roughly 10 million people have signed up on the landing page for the campaign. If we had not started this experiment and just left the original page, this number would be about 7, 1 million people. And it's smaller by as much as 2, 9 million e-mail addresses.

Sending emails to people who have left their mail with a proposal to become volunteers, has attracted 10% of them. The result is that we were able to draw from 2,880,000 email addresses of 288,000 volunteers, and this is much more than originally planned. On average, each email address that was obtained via our landing page donate $ 21 for all the time of the campaign. Total additional 2,880,000 email addresses of our list resulted in an additional $ 60 million donation.

The knowledge h4>

Every visitor to your website - it is an opportunity. Do not miss this opportunity, use optimization and A / B testing, it will give you an extra edge. Check the assumptions. All participants liked the campaign videos. But eventually video worked worse than the picture. We would never have known about it if I had not checked our assumptions. Experiment at an early stage, experiment more. We launched the experiment in December 2007, and the benefit it brought down to the end of the presidential campaign. Since the first experiment was so effective, we continue similar experimentation throughout the site throughout the campaign. Translation done by project staff Generate.club - the first service in Russia for optimizing landing pages automatically. We will be happy to answer all your questions.

P.S. Article is переводом History help A / B testing in the campaign of Barack Obama. Unfortunately, due to unknown bug Habré possibility to specify this when you create the publication was not. We apologize for the confusion!

Source: habrahabr.ru/company/generateclub/blog/233141/