Several people have asked in the last few weeks about how we’re pacing our beta at Hashable – and specifically, what milestones are we using to decide when to bring more people in. This is a fun topic that deserves lots of discussion. Here are some quick thoughts.
First, all beta’s are different – but probably the same general thinking drives the pacing:
1. Have we gained a strong understanding of who our target adopters are?
2. Do we understand exactly what those users care about and why they’ll adopt?
3. Does our product powerfully deliver to the promises of those value props?
4. Sum of the above: If we drove more people to the site right now, would they convert?
5. Does it intuitively feel like a good time to increase user volume? (i.e. riding the diminishing-returns curve of exclusivity)
The general idea of the beta is to collaborate with your target users to identify and perfect your killer use case – and ultimately, to achieve Product/Market Fit (PMF). Awesome PMF is what brings user conversion, engagement & virality.
So a simple answer here is that you bring more users in as your confidence in your product grows.
Here’s an internal graphic mapping out the Product/Market Fit process (sanitized of user target numbers and a few other details but happy to discuss some of that in person):
It took us ~ 21 weeks to get from our initial concept to the point where we felt confident enough in our live product to start inviting a heavier volume of users into the system. But that alpha period can last significantly longer – Aardvark is a great example of extended alpha to nail PMF (this video is sick, amazing, awesome!).
Here are some quick notes from our concepting/alpha stage that might be useful:
1. The concepting phase lasted about 4 weeks and involved no development or prototyping outside of basic wireframes. We were debating on all fronts: likely early adopters, value props, allowable friction, minimum viable product, narrow entry points, lean start-up principles, adoption strategies, game mechanics, what are we gonna call the thing …
2. One caveat we stuck to in all debates was that nobody knows the right answer except the users. Your concepting and alpha stages are all about making brave dumb guesses. (see again that Aardvark video)
3. I’m not a fan of being in stealth mode during concepting. Our conversations were always active externally too – even some very casual conversations during the concepting stage greatly improve our decision making.
4. Our first dev cycle was 4-6 weeks. We were determined not to stay behind the curtain, developing without user collaboration, for too long. We initially invited maybe 30 people in to try it out. We pushed out releases at least weekly from there and invited a new set of people in each week for fresh perspectives. This continued for at least a month. (we still rev weekly btw)
5. Much of the alpha stage involved further narrowing of the idea and product. The more focused we got, the more excited people got about the product.
6. Once we saw people reacting positively to our revs, we pressed on the gas a little with invites and in-person demo’s. Again, this ended up being about 21 weeks into our process.
7. Our next (first?) big awareness push will be timed according to a couple more key product milestones we want to hit before investing in major outreach.
8. I think learning and reacting are the major characteristics of alpha and early beta – but as you progress along the beta curve towards product confidence your big-block learning turns more into optimizing and tweaking.
9. But also keep in mind that establishing Product/Market Fit is an ongoing process – once you nail it for your early adopters, you’ll likely need to refigure things as you want to branch out to new target communities.
None of this is rocket science, and we're so early on - it's premature to offer any major conclusions or advice. But I think it’s helpful to benchmark with each other on this stuff as it's kind of a dark art. I’d *love* to hear other peoples’ experiences – and also happy to discuss more in person with anyone if that’s helpful.
And lastly – a huge, major, mammoth shout out to the NYC tech community. Much more on this topic to come!