Frequently Asked Questions

Does Really Simple Testing automatically optimize my web pages?

Yes it does. You provide different versions of parts of your page such as headlines, or background colors, or testimonials or images etc. and Really Simple Testing will use these different versions, measure how effective they are at generating the response you want, whether it is a sale or a sign-up or a visit to another part of your site, and then use that data to determine dynamically which is the best version to use on your page in the future.

Does Really Simple Testing use the relative success rates of different versions of my copy to determine which version gets displayed?

Yes it does, but it is not as simple as just displaying versions according to how successful they are. For example, if version A converts at 5% and version B converts at 10%, you don’t want version A being used half the time because that will seriously reduce your success rates. Instead, you want B all the time. Why use A ever again if you know B is much better?

In practice though, you never know for sure which version is the best one to use unless you have an infinite amount of data to analyse and no-one does. So instead, Really Simple Testing uses an intelligent, complex optimization algorithm that examines the outcomes of tests and assesses how confident it can be that one particular version is better than the other and as its confidence grows, it displays the best version more and more. You can see this process going on in your reports.

How often does Really Simple Testing re-calculate the weightings of each test version

Really Simple Testing re-calculates weightings every time a web page is viewed. This means that you start to benefit from increasing conversion rates almost immediately. However, you do need to be aware that proper statistical analysis does require a reasonable body of data and so it is quite possible that in the early stages of optimization, sometimes a version that in the long run turns out to be a weak version may actually be used more, simply because, by chance, it may have been very effective in the early stages of a test. Over time, Really Simple Testing will apply the appropriate weighting though.

How does the optimization algorithm work?

The optimization process is quite complex because it needs to be able to deal with a shortage of data in the early stages of a test, whilst allowing you to introduce new versions later on. It also needs to react to changes in traffic behaviour over time. It is possible that in certain circumstances a poor version may start becoming more effective due to market or seasonal changes. For example, in October a few highly organized people may purchase Christmas presents whereas most people won’t. These organized people may respond to one headline better than those people who leave Christmas shopping to the last minute. So a headline that said “Get your Christmas presents now and avoid the rush” could appeal to October shoppers, whereas “Guaranteed delivery before Christmas” may appeal to December shoppers. The Really Simple Testing algorithm is capable of dealing with such situations automatically.

Can I delete or edit versions at any time?

You have total control over your tests and can add more tests, pause or delete tests any time you want. However, to do testing properly you need to acquire a reasonable body of data and the system is designed to make it unnecessary for you to intervene.

If you do too many tests on say, a headline, at once, it will take longer to get enough statistically valid data in to make optimization decisions. This may not matter if you have a very heavily trafficked site but if you don’t get a huge number of visitors, you should stick to only a few.

What can I test with Really Simple Testing?

Probably a lot more than you think! For example, you can test headline copy, headline colors and fonts. Different calls to action. Different images, guarantees and testimonials. But you can also test prices too as well as different form designs.

Is there a limit to how much traffic Really Simple Testing can deal with?

Like any system, Really Simple Testing does have a finite capacity but the way the system works is optimized for performance which means it can deal with a lot! However, there is a limit on page displays for normal members set at 25,000 a month. You can use the system with more but there are extra fees.

What skills do I need to use Really Simple Testing?

Really Simple Testing is designed to be easy to use. All you need to be able to do is cut and paste HTML. You don’t need to be able to code in HTML although the ability to use simple HTML tags is useful. The reason you don’t need to know HTML is that you can use Really Simple Testing to replace blocks of text within your web pages as well as blocks of HTML, or even whole pages.

Is there an affiliate program for Really Simple Testing?

Yes there is. It is by invitation only at the moment but if you think you can sell the Really Simple Testing system to publishers, please get in touch.