A/B Testing: A lesson in cringe
March 15th, 2009 | Published in SEO | 6 Comments
Having heard Caelen talk a little about A/B testing on his site, I decided to finally have a go for myself. The results were cringe-inducing for two reasons. The first reason is that you get to see exactly how important copy is on your site. The second is that what works isn’t necessarily what your mother would have wished for you. I decided to A/B test our Piehole ebook/newsletter page. The page had some pretty straight talking copy on it so I dug out some ’sales copy’ that I had left over from the Commission Blueprint course I bought a couple of weeks ago. The difference in the copy is as clear as night and day. Our original version was pretty understated …
The second ‘new’ version was hardcore. There was more than a little controversy amongst the piehole team when I posted it up.
I really thought that the original – less salesy copy might do the trick. But …
the new variation seems to be beating the crap out of our clever copy.
UPDATE: Caelen reminded me this morning of the importance of sample size when it comes to A/B testing. I cheked back on my A/B test this morning and the results are not as clear cut as they first appeared. You can see below that the ’scary’ copy is still working better but by a far lower margin. I still haven’t collected enough data for a ‘high confidence’ result though.
March 16th, 2009 at 11:05 am (#)
I sometimes find it disturbing how well hyped sales copy does on the interent. I interact with a lot of guys from the US and their sales copy is much pusher – it works. Whereas us Europeans think that less is better…
March 16th, 2009 at 12:07 pm (#)
Okay – commenting on your own blog post has to be wrong – but I have a follow up to this. A couple of weeks ago I sent out a ’sales copy’ driven letter to the goose database. It was a long form sales letter promoting http://www.3r.ie/futurizeyourenterprise.htm. I had an unsubscribe and have just had a coffee with someone I respect who really pushed back against this type of letter.
I’m sitting here wondering which way to go when it comes to sales copy. The above blog post and the bulging attendance at our workshop last Friday make it hard to ignore the effectiveness of writing strong sales copy. Personally I rail against this type of copy. You could say it ‘insults the intelligence’ of those who read it. On the other hand – we had great feedback from the course content and ultimately higher signups to our newsletter.
I want to have more people engaged with Piehole and I want to have more people come along to our courses. I wonder can softer copy achieve the same results?
We also had a brief discussion about the type of person that would be attracted by the more brash approach. Does ‘get rich quick’ style copy attract a ‘get rich quick’ mentality? Are we tricking people into signing up to our newsletter? Would we rather they didn’t sign up in the first place?
March 18th, 2009 at 8:24 am (#)
Quote “Are we tricking people into signing up to our newsletter?”
This is the main problem I think. For most of the US guys this would not go through their head.
I am only tricking someone if my product if crap – that is what my thinking has come down to. My product is not and neither is yours – so if the “sales” copy gets better results it is the way to go, and I mean long term better results.
March 23rd, 2009 at 3:47 pm (#)
[...] the difference between untold riches and the poor house. If you don’t believe it – read James‘ article on what a big difference it can make. This month we have Denise Faye from Achiever [...]
March 23rd, 2009 at 8:02 pm (#)
Emotionally the “strong copy” has a huge response in my body. Sometimes I find myself clicking on the buy or subscribe button without having made a conscious decision to do so … All this comes as a huge surprise to me as I wasn’t brought up in the West and like to think that I was not subjected to the strong consumerist brain-wash from an early age which is taken for granted around here.
After I realised how strong the emotional manipulation of strong copy is, I only skim over offers using it. The longer term internal effect is even worse … Probably this is where its power comes from – the destabilising effect it has on the inner peace of a person, which in turn comes from the discrepancy between truth and reality. The unconscious picks up on it but the conscious can’t, this inner conflict brings the person into a hypnotic state. Then the strong calls for action do the rest.
Some day there will be laws against strong copy, hopefully.
January 20th, 2010 at 4:53 pm (#)
[...] page on Piehole.ie. We are a pretty low traffic site but I didn’t let that stop me drawing conclusions early on. My thought was that hard as nails sales copy was outperforming a very much more [...]