What is Amazon’s “Manage Your Experiments” And How To Use ItMay 10, 2020 in Blog
After years of closely guarding their domain, causing a testing “black hole” for sellers with ambitions of optimization, Amazon is finally starting to roll out tools to enable sellers to squeeze every dollar of effectiveness out of their Amazon marketing efforts.
The latest tool in that growing seller arsenal is Amazon’s “Manage Your Experiments” feature.
In this post I’m going to walk through what Amazon’s “Manage Your Experiments” is, who has access to it, and how to get the most out of it.
What is Manage Your Experiments?
Manage Your Experiments is a new tool that allows eligible sellers to run A/B tests on their A+ content.
In the words of Amazon:
“You no longer have to spend months testing different product marketing strategies when creating A+ content on your brand’s listings. The Manage Your Experiments tool helps you test two versions of A+ content simultaneously, and shows you the results in an easy to understand dashboard. Through the use of A/B tests you can compare the two versions of content and see which version resonates with Amazon customers.”
This means you’ll be able to test many different aspects of your A+Content and get data back that will show you definitively what is and isn’t converting the best for your listing (i.e. producing more sales).
By testing different elements of your A+ Content one-by-one, over time you’ll start to get a picture of the kind of text and images that appeal most to your target audience.
The results will also give you insights about what should work best across all your A+ Content.
Some of the A+ Content elements you’ll be able to test are:
- The order in which your A+ modules are stacked
- Adding extra modules to your existing A+ content
- Adding charts, graphs and graphic in place of photography for your images
What is A/B or Split Testing?
A/B testing (aka split testing) is when you simultaneously test two variables against each other under controlled conditions and then use the data to show you which was more effective.
Say you’re sending an email and you want to test two subject lines to see which is more likely to get the email opened (i.e. be more effective).
Split-testing software will allow you to send BOTH at the same time. So when you send out the email to 200 people, 100 will get subject line A and 100 will get subject line B.
The data will then tell you which subject line got the most opens and therefore was the most effective.
Who Can Use Manage Your Experiments?
Right now, Manage your Experiments is only available for vendors and brand registered sellers on the U.S. marketplace (If you’re not part of Amazon’s Brand Registry, you can learn all about how to do that here).
However, in order to access it you must have listings with existing A+ content that get a high enough volume of visitors so that Amazon can “provide statistically sound results for your A/B tests.”
As with a lot of things Amazon, there are no specifics as to what qualifies as a high enough volume of visitors, so if you’re brand registered and want to check if you have eligible ASINs you can either go to the A+ Content Manager in your Seller Central account:
Once there, A+ Manager will show which of your current ASINs that have A+ Content are eligible.
Or go to this Amazon news post and hit the link at the bottom that says ‘Go to Manage Your Experiments and set up your experiment.’ If you have no eligible ASINs you’ll get a message like this one:
And if you do have eligible ASINs you’ll be taken to the Manage Your Experiments dashboard where you’ll be able to create and manage all of your A+ Content A/B tests.
Once there, simply hit the button that says “Create a New Experiment” and Amazon will let you choose which of your ASINs with A+ Content you’d like to run a A/B test on.
A workflow will then appear that will take you through the steps you need to take to set up your experiment. Once done, you’ll have two versions of your A+Content ready to test against each other.
Then schedule the experiment, leave it to run its course and come back when it’s done to check on the results.
Best Practices For A/B Testing In Manage Your Experiments
When A/B testing you should only ever test one element at a time.
The reason for this is, if you have version A with one headline, one set of images, and one set of text, and you have Version B with a completely different headline, different images, and different text, when the results come back and you learn Version B was the most effective, you’ll have no idea why.
Was it the different headline that made it more effective? Or was it the different images or the different text?
The only way to know is to test one element at a time.
So if you want to test the headline, run two identical pieces of A+Content and only change the headline.
If you want to test your images, same thing. Use the same headline and text, and only change one of your images.
It can be a slow and somewhat tedious process, but that’s the point of optimization. The goal is to squeeze every last drop of ‘conversion juice’ out of every element until you end up with the perfect combination of elements.
And then you can test that against something else.
How Long Should I Run Amazon A+ Content Experiments?
Manage Your Experiments allows you to choose any duration between 4 and 10 weeks. Within that, you can cancel your experiment at any time.
However, as with the best practices section above, the whole point of A/B testing is to get as much data as possible before you make a decision. So for the most reliable results you’re going to let your experiments run for as long as possible.
One Of The Big Flaws Of Amazon’s Manage Your Experiments
I’d be remiss if I didn’t mention one big flaw with Manage Your Experiments before concluding this article, and that is that Manage Your Experiments currently only allows you to A/B test your listing’s A+Content.
That means sellers still cannot split test their listings regular title, images, description or bullet points without using a third party software like Split.ly.
Given these are the first thing anyone sees when they first land on your listing, this is a pretty major flaw that will hopefully be addressed as Manage Your Experiments evolves.
Manage Your Experiments is an excellent new feature for optimizing your A+ Content and testing variables to create your highest converting listings possible.
Whether it’s worth the time and effort necessary will depend on where you are with your brand right now – if you’re just starting out there may be lower hanging fruit that will give you a better ROI for your time.
However, if you’re a more established brand, testing each individual element of your A+Content could give you the data that helps you to squeeze every last dollar out of your listings.
John Robb has been writing copy for almost 20 years for brands like Google, Beats By Dre and Sony Playstation, to name just a few. He currently lives in Los Angeles where he spends his days consulting for brands on digital marketing and copywriting, and tinkering with classic cars.