Brave Tech World

July 15, 2008

TUE 15 JUL

A 10% improvement on conversion with one change By Marcelo Calbucci

 

    If you look back at the Sampa sign up form, it has been through 4 or 5 different versions. At one point we asked you about 15 questions. Then we worked with one local UX expert to simplify a bit, then we worked with a Portland-based UX firm to simplify it even more and we had just 8 inputs:

1. Name
2. Email
3. Confirmation of email
4. Password
5. Confirmation of password
6. Agree to Terms of use
7. Web address
8. CAPTCHA

    After reading the book "Web Form Design Fillin in the Blanks" (by Luke Wroblewski) over the 4th of July, I decided to take a closer look at our sign up form again.

 

    I am always afraid of making any change to our sign up pipeline since we have such fantastic conversion rates that is easy to assume something, make a change and it to backfire.

 

    Well, there was at least one change I've made that was a no-brainer. The Terms of Use checkbox wasn't checked by default and I changed that.

 

    But that's not the interesting part. On the book, Luke mentions that a lot of the things we ask on forms are limitations of the application itself and that it doesn't add value to the customer ("inside out").

 

    You can clearly see some of that on Sampa. Why do we ask twice for your email? Why do we ask twice for your password? Why do we need your name? Why do we need CAPTCHA?

 

    We have good answers for all of the above, and we A/B tested several scenarios over the last 2 years to know that we are close to the right combination that yields maximum conversion and retention. But one piece was never tested and it was off: CAPTCHA.

 

    Why do we need CAPTCHA? Like any other free web service that you sign up to (Hotmail, GMail, Ticket Master, Blogger, etc.) CAPTCHA is necessary to prevent automated bots from creating hundreds or thousands of fake accounts, either for Spam or Link Farm. So it's a necessary evil that good people have to go through so we can keep the bad people out, or is it?

 

    Well, last Monday at 5:13 PM we removed CAPTCHA from Sampa. It wasn't easy and clean, but we created a set of tests and rules that will make us not display a CAPTCHA about 99% of the time. I can't really disclose the tests we make since that's the secret you want to keep out of the hands of bot creators.

 

    The result: 9.2% improvement on our conversion rate!

 

    The entire team is just shocked that such a huge percentage of users wanted to sign up to Sampa and either didn't pass the CAPTCHA (less likely) or found the form more intimidating with a CAPTCHA (more likely). The third explanation which we've been talking about is that with less vertical space, the "Continue" button is above the fold more often and give users a better sense of how short this form is.