Smarty

The Setup

Smarty is a new mobile network built with simplicity at its core.

In a team of just myself, a developer and a data guy. We harnessed customer feedback and data insights to find where to focus. Striking fast and true, with an arsenal of the kinds of things you’d most commonly associate with “growth hackers”. Think, running a/b tests, building propositions, honeypot tests with mailchimp and fake targeted ads. It was awesome.

Notable accomplishments;

⬊ Launched the cheapest plan ever, to hack comparison sites
⬊ Changed password requirements from 9 to 8
⬊ Created a new customer loyalty programme

But we'd had our fill of low hanging fruits, it was time for something more juicy.

The needle in a stack of simcards

Drop off in the payment journey was at a staggering 45% and we'd set out to find why. In the early days of Smarty, some bright spark had set up an abandon basket survey to collect responses. But no one had been brave enough to go through them... yet.

After two long days wading through a swamp of 3000 responses, I had outlined 5 distinct problem areas. We presented these to the client and began to explore ways to tackle these areas. It quickly became clear that giving away free sims, would be our fruity three course meal. Helping us win the hearts of customers and improve that pesky drop off statistic.

But how could we, a humble team of three, find a way to test a free sim card offering, without disrupting their entire checkout and distribution flow?

How the test worked

I don't want to understate how much work went into this, but for now just use your imagination. In Google Optimise we set up an A/B test on the order summary page. Variant A remained exactly the same, but variant B gave users the option to pay or order a shiny new free sim card.

If users were feeling a little unsure on paying, they'd click Get my free sim. This would then take them to a mailchimp form, styled by myself to look exactly like part of the existing Smarty website. Here we'd gather there details and send them an automated 'Your Sim is on it's way' email. It was that easy...

Once their details were in the system, I'd manually feed a CSV into the royal mail online postage service to create prepaid labels. Print them off and add each to an envelope, with a trackable sim card inside. Trackable sims were all scanned by me, by hand. I'd then walk them 3 minutes down the road to the nearest postbox. Oh what a thrill.

Licking and sticking

Over the next 2.5 days I personally sent over 700 sim cards to all corners of the UK. By the end of day 2 we'd doubled the overall number of sims being dispatched. We found that by removing less committed users from the funnel, we had improved conversion by 8%.

I'd just finished a huge halloumi wrap and was in a bit of a post lunch haze, when we were called up for a quick catchup. It was all small talk and pleasantries, until they broke the news. My heart sank. Black Friday was coming and we had to kill the test.

We were never able to reach statistical significance and it still keeps me up at night.