Last night I watched the Bill Pullman classic movie The Serpent and the Rainbow about Haiti and Voodoo. And, I thought about Web Page Split testing voodoo. Many people are offering web page split testing as a service. They are knowledgeable about lots of things. However, they are getting themselves into something deep, just like the Bill Pullman character in the movie.
It should be noted that the Anthropologist who Serpent and the Rainbow is based upon, despises the movie. In part this is because what he did as a PhD scientist is discounted and a bunch of fantasy and horror is added to make a Hollywood movie. Since I know some ethnographers, I’d not be surprised that an Ethno-Botanist doesn’t like how he’s characterized. Most don’t like the sound of their recorded voice in my experience. However, you might not like it if you were the basis of Serpent and the Rainbow and found in the “Horror” section at Blockbuster (call first, I had to go to a distant Blockbuster). Having your life’s work show up in the Horror section is a bit like having 60 Minutes do your life story.
I watched the movie to remind myself of some of the extreme characterizations of Hispaniola where Haiti sits (West is Haiti and East is the Dominican Republic on the Caribbean Island). Hispaniola is also where the first whites settled in North/South Americas. A place of pirates and gold, and still a place of “a bottle of rum.” Many of the Haitians and Hispanics are descendents of people who were brought to Hispaniola, often against their will. Many Haitian people reside and work places you might NOT expect, like Princeton University, where Michelle Obama’s brother went to school. And there are many misperceptions about the Hispaniola. There are beautiful resort areas and some of the best Kite Boarding in the World. This is also an island that the wealthy to vacation, such as Bill Clinton, A. Jolie, etc.
So what’s this got to do with Web Page Split testing? Well, it takes some digging to understand what is really going on the island and with split testing. If you’ve never been to a wealthy resort on Hispaniola, you might never realize that there are many beautiful resorts and that the island is hundreds of miles long. If you have never taken a statistics class, you may think that you know how to split test a web page, yet you don’t.
Even people who know everything about how to test web pages often test too few “respondents” on their web page and then they “change” their web page to the one they think works best, based on bad testing. One example I’ve seen is people testing web pages exclusively online and during business hours. Is this a good idea? Well maybe not. You might be making changes based upon people that are “out of work” and who should be working during business hours. Good split page testing means you should allow people enough time to get a representative sample of your target market. Is your target market available during business hours? Doctors, lawyers, accountants and other wealthy individuals may be left out.
Also, people misuse a lot of the free tools out there. For example you can mis-configure Google analytics, right? May be you think your conversion rate is pretty good, but what about the people who never made it to your site? To test people that are in your target market that are not currently getting to you on the web you’ve got to develop another strategy. This is where a research firm like ours can help. We can recruit people to come to your site, either online or in-person and give you a representative sample and the correct number to make meaningful assessments.
Or you could just use the voodoo that you do so well.