I was talking with a client about a current mailing they were running. They told me that letter B was converting at a better rates than letter C. When I asked how they ran the test I was baffled at the response. I thought this was common knowledge. Ultimately, it became a lesson in market testing factors.
When you test your marketing piece is just as important as what you test. Here’s what they did in a nutshell.
They took a list of prospects and cut it in half. They sent sales letters to a group A and measured the results. They sent group B a letter with a different headline and tracked the results. This is not bad. It’s a good way to start testing your marketing however, the second group was sent letters one month after the first group received their letters.
The easiest way to test your marketing is to test one variable at a time and measure the results. If headline X pulls better than headline Y then use headline X as your control and continue testing to see if you can beat those results. This process is called split testing and is nothing new to anyone with direct marketing knowledge.
One factor commonly overlooked when performing split testing is time. The split test has to be run at the same time to be considered an effective test. If you send one mailing this week and another mailing four weeks later then you’ve essentially thrown in another testing variable called “time.”
“So, what’s the big deal it’s only a couple of weeks difference?” he asked. Imagine mailing a mortgage marketing piece right before the recent financial meltdown and a month later after all the headlines hit the news. If you’re selling earthquake insurance you’ll see a difference in response from a mailing before and a mailing after the recent Haiti and Chile earthquakes. As you can imagine, awareness in society changes over time and is another piece of the marketing puzzle you need to put in place.
What About Online Marketing?
The same rules apply online. If you’re marketing through email, most email responders allow you to perform split tests. Run tests changing the subject to test open rates. If your email system allows you to run more than two then test two different offers with each headline to test click-through rates too. Make sure it’s the same mailing and you’re not sending one email blast today and one email blast tomorrow. (that’s time and sending a second mailing to the same list, another variable)
What if you’re testing landing pages or opt-in forms? You want to run split tests as people visit the site. Testing one page for 3 weeks and another page for 3 weeks will not give you accurate results. Time can skew your statistics plus your source of visitors could change.
Most email providers offer a form of split testing your emails but, do you use it? If you have a sales page, opt-in form, squeeze page, etc. are you split testing to see what works or are you going by what the gurus tell you works? (I recommend SendPepper for reliable email and landing page split testing and marketing.)
Marketing Is Not A Science
Marketing is not a science but it does require one to step back, analyze the variables and, look at the results. The only scientific rule I know that works every time is test and measure.
What’s your experience?
Photo by Redstamp
Where I work we dont scientifically test but we do change things up, note the differences then stick with what works. Example, one year we surveyed customer and had about 900 respondants. The next year we lead with a personalized email to each customer telling them the survey was coming and asking if theyd spend a few moments to take it. Survey response was over 1400, an increase of more than 55%. That was a great return for an extra hour of our time. There were several principles of influence at work in the pre-contact email and we seek to employ all of them as much as we can now.
Brian,
That’s a great difference in response. You also have the time variable mixed in because they were a year apart from each other. However, that email was probably a huge factor.
While I deeply enjoy the psychological and persuasive aspects of marketing, effective marketing is math. It’s not easy. In my opinion, if you’re not testing you’re guessing and risking a lot of time an money.
Couldnt agree more. Testing is crucial. Without it youre flying blind.
Different things work for different people. Also, time of day and week are important as well. Trying to target people based on what’s happening in the world today can occasionally be difficult at times as well, as sometimes recipients don’t receive messages until days later when things may not be as relevant. It’s all a matter of testing.