5 1/2 steps to securing your idiot-proof optimization program


We’ve all been there. You’re starting a new job as a conversion optimizer in a forward-thinking company. You see huge potential and you’re finally in a position to implement all your ideas. You’ve done a heuristic analysis on the website and… oh shoot! The site messaging isn’t addressing the customer’s core needs! What’s even worse, your advertising is misaligned and doesn’t communicate the true value of your product. You take a few deep breaths and come up with hypotheses that you think may be worth testing. All you need is a month or two for an in-depth research & the autonomy to make changes to the website. Your company prides themselves in being data-driven so that shouldn’t be a problem.

You pitch the idea to your boss... You have a well structured 11-step plan... and… nothing.

“You’re over-complicating things.”

“We’ve already brainstormed all the things we want to change.”

“These are the changes that are allowed within our branding.”

Basically, all you need to do is set up 3 A/B tests per week to validate their opinions. Starting immediately.

3 months in, you’ve have had 2 winning tests that resulted in a small uplift, 1 losing test, and 33 tests that made no difference whatsoever. The board doesn’t see the value of the CRO program, so you get pushed into the “Creative” team - you already have experience in this area anyway - where you’re tasked with creating Facebook campaigns that have been crafted by the marketing strategist. And you wait for the cycle to repeat itself...

(source: Momoko Price @ Digital Elite Camp 2018)

Luckily, the speakers at Digital Elite Camp 2018 in Estonia had a lot to share about why management and conversion optimizers have such a hard time speaking the same language and what you as an optimizer can do about it.



pdf-icon No time to read this now? Get the PDF sent to your inbox!


Every Company Is “Data-Driven”


Nowadays, you’ll have a hard time finding a company that will admit to not being data-driven. That’s because we all think that data is the ultimate diet pill, the silver bullet that will result in hockey stick growth through its mere collection. - Click to Tweet


But let’s get real.

Collecting & using data isn’t special. It’s a best practice at most. To quote Peep Laja in his keynote speech at Digital Elite Camp:

“You’re an idiot if you’re not doing at least this.”

Then why do companies still have such a hard time using data effectively to inform their decision making? How is it possible that they invest thousands into a conversion optimization program just to take away any decision-making power the specialist could have and instead force him to implement their own random ideas.


Like what you're reading? Get regular growth hacking tips sent straight to your inbox!
Sign Me Up!


Ego Is The Enemy

Yes, that is the title of some RockBoosters’ favourite book. But there’s so much truth attached to these 4 words.

When companies sign up to become more data-driven, there’s 1 thing they often don’t realise. To let your decisions be informed by data means to let go of your own experience about what’s the best practice.

So? What’s wrong with this? If there’s data to back it up, why is this even a question?

Most high level executives got to where they are because of the decisions they’ve made over the years. They’ve been rewarded for their great intuition and ability to spot the right opportunities at the right time. But they often don’t know sh*t about analysing data.

“The c-suite doesn’t give a shit about a/b-testing, ux improvement or web analytics” , André Morys

(source: André Morys @ Digital Elite Camp 2018)

When you’re asking them for permission to make changes based on some insight that you may find after months of doing research, they’re thinking “Why should I waste money & time on petty research? I got the company to where it is now, I already have a good feeling for what works and what doesn’t.”


But… Even the big players get it wrong sometimes


Take a look at Booking.com. They are notoriously famous for their state-of-the-art A/B testing program. Which is why Lukas Vermeer’s story about the danger of assumptions was so powerful:

Lukas was challenged with optimizing the order in which hotel listings were displayed on a landing page.

(source: Lukas Vermeer @ Digital Elite Camp 2018)

After 6 months of working on an algorithm, can you guess what delivered the largest uplift?

The winning solution was...

Remove all listings & get rid of the algorithm that was computing things in the background.

Whaaaat? Why?

The listings didn’t matter. When visitors got to this landing page, they went straight for the search bar. The computations were actually slowing the site down, leading visitors to bounce before they could even take action.

Read more on this subject in this blog post.

The whole project had been based on the assumption that what was on the page actually mattered. But it didn’t.

So, what can you do as an optimizer to make sure that your A/B tests are all based on strong, data-backed hypotheses & avoid setting yourself up for failure from the onset?


Follow these steps


Step 1- Just do it

If you can get the budget/time to do some proper research you’re blessed. But oftentimes your boss will not care about this if this isn’t your primary responsibility. Don’t let this discourage you. Your goal is to prove the importance of research for A/B testing. Sometimes it really is easier to beg for forgiveness than to ask for permission.

You may need to stick to what you already have access to, but the following data sources will (almost) always be available to you:

    1. Analyse GA data to find the largest drop-off
    2. Check whether there are devices or browsers that convert far below average
    3. Watch Hotjar recordings
    4. Interview customer service
    5. Ask for a chat transcripts review
    6. Conduct a forum mining
    7. Pay per response surveys (Momoko Price’s secret weapon)

The goal is to figure out what your target audience really values. What motivates them to become a customer? And what barriers and anxieties drive them away?

Step 2 - Compare your findings to your website

How are you already addressing visitors’ motivation, perception of value, fears, and anxieties? And more importantly, in which areas are you scaring them away?

Things to look out for:

  • Are you highlighting the features instead of the benefits for customers?

(source: Momoko Price @ Digital Elite Camp 2018)

  • Are the messages on your website the ones that customers actually care about the most?

(source: Momoko Price @ Digital Elite Camp 2018)

Step 3 - Engage with Storytelling

The problem with research is that it sounds absolutely boring. Who truthfully wants to hear about customer surveys, or focus groups, or exit polls…?

(source: Els Aerts @ Digital Elite Camp 2018)

What you need is the journalist approach. What can a data analyst learn from a journalist, you ask? Think about how (good) journalists report. Do they simply state the facts and bluntly lay out what they believe to be true? NO! They tell a story and allow the reader to form their own conclusion.

How do they do this? One of their most powerful tools is EMPATHY.

Most managers never see how actual users engage with their website. So if all you do is TELL them about the things that are wrong with your website, it’s incredibly easy for them to think “Ah well, that can’t be too bad”.

However,  if you’re able to SHOW them how frustrated users become while navigating your website, they’ll be much more likely to empathize with the user and actually feel their pain.

The ideal tool for this would be to show them recordings of actual user testing sessions. Yeah, sure… If only it were that easy. Again, stick to what you’ve got. If there’s a problem with your website simply find a Hotjar recording that clearly demonstrates the user’s frustration. Make your manager watch this. It won’t be as easy to turn a blind eye anymore. Check out our code to link Hotjar to Google Analytics to find your answers. 


pdf-icon Click here to download our Code to Track Hotjar ID in Google Analytics



Step 4 - Calculate the ROI

Now that you’ve got their attention, show them the numbers they won’t be able to resist.

(source: Momoko Price @ Digital Elite Camp 2018)

You don’t even need to promise enormous increases. Take a look at how increasing your conversion rate by just 2% every month will result in an almost 12% increase in revenue by the end of the year.

How can you make sure to consistently increase your conversion rate by 2% every month? By making meaningful changes. And how do you know what these meaningful changes are? RESEARCH!

If you want to calculate the exact €€ value you’ll be earning the company, you can download our quick & dirty ROI calculation sheet here.


pdf-icon Click here to download our Optimization ROI Calculator Template


Step 5 - Ask for empowerment

You’re almost there. Besides research, conversion optimization is all about speed. The faster you can test, the faster you will learn & the faster you will reach those results you just promised.

However, you’ll never have the necessary speed to allow your A/B testing to really take off if you have to ask for approval on every single test you want to run.

Hopefully, by now, you will have convinced your boss that you’re pretty darn good at identifying those opportunities for improvement. But if they’re still not convinced, check out how Booking’s culture of empowerment is what makes their optimization program so successful.

Step 5 and a half - Quit

We were surprised to hear how many of the speakers at Digital Elite Camp indicated they had left an employer because they could not handle their ignorance towards listening to the data that matters. (Momoko Price quit her job in rage after her boss insisted on describing their product as a “Multi-Dimensional Data Universe” - what do you mean, you don’t know what that’s supposed to be?!)

But I sincerely hope that you are not forced to take such drastic measures in order to secure the optimization program you love. Are you?

Can’t get enough of our Digital Elite Camp stories?

Not a single speaker at this 3-day conference managed to finish their presentation without mentioning the importance of research when doing anything related to optimization. But there was so much more discussed that I couldn’t possibly fit into this post.

Luckily, we have 2 more exciting blogs about the inspiring insights gained at Digital Elite Camp 2018:

Natalie Jamieson

Growth Hacker @ RockBoost