Just Enough Research

Split testing can help you optimize your current design until you reach a local maximum, but it can’t tell you how much you can accomplish with a different approach

You always need to answer “Why?” before asking “How?”

If you focus on optimizing what you have rather than also considering larger innovations, who knows what vastly greater heights you might miss (Fig

The standard is ninety-five percent, meaning that there is a ninety-five percent chance that you can rely on the result.

How large a sample do you need? It depends on the sensitivity and how large an improvement you want to see

tests will be faster and more reliable with a larger sample size

Small, incremental changes will have a more significant influence on a high-traffic site

Small, incremental changes will have a more significant influence on a high-traffic site

If you are looking at making a small change, you will need a larger sample to make sure that you can be confident in the result.

Run the experiment until you’ve reached a ninety-five percent confidence level