Select Variables and Create Versions

It’s time to work out the designs related to your hypothesis. You can make different variants based on the original.

We have opted for 2 dynamic content formats for DIDI, whereby we have created a dynamic feed with model images and a dynamic feed with product images, without models.

Start experiment

The experiment can be started. Have all your test data ready for checking.

Analyze and determine result

When your experiment is finished, you dive into the results achieved. Look at the data you have collected and test it with your hypothesis to formulate a conclusion.

Ultimately, the carousel with the model achieved a better CPA than the carousel with the product image. The CPA is 33% better. There is a 91% chance that the difference is not accidental. This confirms the hypothesis, but the advice is to continue testing until we can say for sure.


Keep in mind that you can never achieve maximum results in one Engineering Directors Email Lists go with evidence-based marketing. By continuously testing and starting again at step 1, you can continue to realize improvement after improvement. Even if there is no clear result, you must continue to optimize.

Engineering Directors Managers Email Lists

This test is not an isolated one for DIDI, but we want to test it more often over a longer period of time and we are going to work on follow-up tests, for example on content format; carousel vs. canvas or whether/no price in the visuals.

Tips for evidence-based marketing

Finally, a few tips to get started with evidence-based work.

  • Keeping a log is crucial
    If you want to be successful in the long term, it is important to document all ideas, elaborate test plans, tests that are live and completed tests. This can already be done in an Excel file, but can also be done in various online tools. Over time, you can analyze the completed tests to find out what determines success.
  • Proper setup of UTMs and campaign structure
    This allows you to find everything you need to do your analyses.
  • Don’t draw conclusions too quickly.
    Provide test optimizations and do tests over a longer period of time.
  • Do not make too many different versions
    . The chance that you will subsequently find significant differences is small.
  • Be critical of which KPI you evaluate
    . For example, sometimes a test has more effect on order value than conversion rate (or both).
  • Segment your analysis
    Sometimes your overall result does not score significantly, but a specific segment does! Think of demographics, placements, devices, resources and browsers.
  • Do not be demotivated by a negative test result, this is also learning!
    You can also test again with a new version of variables. You may find a difference now.
  • Use a Customer Data Platform as a starting point for evidence-based marketing
    If you have good insight into customer data, this can serve as a breeding ground for Evidence-Based Marketing.

If you have a brain, you have a bias

Everyone has a subconscious that is driven by preferences and impulses. The unconscious is much more powerful than the conscious. That is why it is so important to pursue an evidence-based approach!

Leave a comment

Your email address will not be published.