How to Sequence Growth Tests
Growth Mar 10, 2019
When looking to drive growth for your startup, it’s highly unlikely that you’ll run out of ideas to execute on. If you’ve hit that point then I’d urge you to bring in some outside help — a stranger, perhaps — to look at your product and to offer candid feedback. Solicit people to poke holes in your model — you need to keep innovating to survive.
For the rest of you — you’ll probably hit a point sometime soon where you have too many ideas, and realize that you’ll only be able to execute on 10% of them at best due to various constraints — time, money, or more likely, resources. What’s more — as time passes, what once seemed like a viable test idea may have become irrelevant because of core product changes, company direction changes, changes to the value prop — you name it! So, how do you make sure that the tests that you are running are the right ones to be running at the right time? It’s all about planning around effort and impact.

At Upside, we measured effort and impact of a test on a scale of 1–5. For effort, a 1 is “very easy” and a 5 is “very hard.” For impact, a 1 is “huge potential impact” and 5 is “small impact.” Here’s how we looked at it:
Effort
1 — very easy
2 — easy
3 — medium
4 — hard
5 — very hard
Impact
1 — huge potential impact
2 — large potential impact
3 — good impact
4 — unsure of impact
5 — low impact
In the early days of growth testing, it’s relatively easy to pick off a large number of “1 and 1” efforts to impact tests. This is what we often call “low hanging fruit” — the things that we can almost instantly deploy that should generate a huge impact on your goals. In the early stages, you’ll want to make sure you are running a large number of low or fairly-low effort tests that drive high impact, alongside higher effort tests that also drive high impact. Focus on impact, and brutally prioritize your time and tests that way. Your test plan should look something like this:
Test 1
Effort — 1
Impact — 1
Test 2
Effort — 1
Impact — 1
Test 3
Effort — 2
Impact — 1
Test 4
Effort — 2
Impact — 1
As your product matures, you’ll typically have to work increasingly harder to produce high effort low impact tests. It’s just the nature of optimization — you hope that your early tests move the needle and as a result, you make it harder to reach the same impact with low effort. This isn’t a bad thing! It means you’ve probably got product/market fit, and that the growth flywheel is spinning! Congratulations!
At Upside, when we had a test idea we’d look at it through the lens of goal/hypothesis as well as effort/impact. The general ratio we look for is somewhere around 75% medium or high impact, low effort tests and 25% higher impact, higher effort tests. In one low effort, high impact test we simply wanted to see if we could get business travelers to sign up for a special offer by popping up an email capture modal. That test took 15 mins to bake and resulted in a point-level improvement in conversion rate across the board. It’s rare to get such big wins so easily, but you should strive to ideate and launch as many of these “low hanging fruit” tests as possible.
You’ll want to spend some time each week making sure to clean out or de-prioritize test ideas that once seemed like great ones, but are now no longer relevant. We take a few minutes of each week to “prune” the older ideas. Tactically, we also put our test ideas into a sortable list so that we can easily see what our 1 effort 1 impact ideas are, and keep them bubbled up to the top.