Engineering the Status Network for Growth - Contributor metrics, activation ladders and first steps to mass adoption
Introduction
The goal of Status is mass adoption of Ethereum. How do we know if we are successful and moving in the right direction? A useful tool for this is to use growth metrics. This means success and failure is more visible, and it brings clarity of action. For more on the use of growth metrics, see Paul Graham's essay on Startup=Growth.
This doesn't mean we should do anything just in order to get numbers up.There are many things that we can't easily capture using metrics, such as: are we even building the right thing or more experimental things that we can't measure except on a very long time scale. For these we can let our principle guide us, as well as intuition. Additionally, there are certain constraints on the Ethereum network (that we are part of it) that makes things like paid user acquisition undesirable right now.
It's also worth pointing out that having metrics doesn't require any app changes like Mixpanel, we can do a lot on the network level. And once we get rid of our cluster, or get shielded transactions, we might lose a source of data. And that's completely fine, because our principles are more important than metrics as a tool.
Growth metrics
I'd like to argue there are still very important metrics that we should be tracking regularly and having growth goals for. This is something we do a bit here and there but it's generally rather ad hoc, and not part of a cohesive whole. The goal of this project is to give us a shared understanding of how we are doing with the Status Network overall.
In a Town Hall a while ago the following slide was presented by Jarrad:
Our job: Convert SNT Holders to SNT Users to Active Contributors
This has been rephrased in many contexts but I don't feel like we've taken this mindset very seriously. I'd like to change this.
I'd like to propose two example high impact growth metric goals that I think we should be tracking regularly and work towards.
1. Be the biggest crypto OSS project as measured by monthly active contributors on Github.
We should be proud that we have the most developer activity of any ERC20-based project, but this is just the start. We can easily be the biggest crypto OSS project, and if we are successful there's no reason we can't be the biggest OSS project, period. Having a strong network of contributors is at the core of what we are doing here and there's so much we can do here.
In terms of activation ladder, these are many contributing metrics to this that are also useful to track.
2. 80% of Status core contributors use Status on a daily basis.
While we are after mass adoption, I believe we must first reach intense micro adoption. If the vast majority of us aren't using Status on a regular basis, what are we doing here? How can we expect other people to use us? A small group of people must love us and need our product before mass adoption becomes relevant. A close second to this is the core Ethereum community, such as Devcon people.
A word on haplessness: one critique of this type of metric is that we can't track it because of X and Y. I think that's a poor excuse and reflects lack of resourcefulness. We can easily use surveys of a subset of people and basic stats to get a Good Enough metric here. It doesn't require anything more than this.
Activation ladders
The idea behind an activation ladder is that people invest more resources and take on more responsibility as they get deeper into something.
Example: Awareness -> Consuming -> Discussion -> Code/contributions -> Swarm leads etc
I would expect to see very roughly a x10 (~x3-30) factor for each of these. So if we have 100k impressions, 10k reads, 1k discussions, 100 contributors, 10 team/swarm leads. Of course, if we can increase concentration at later stages of funnel that's even better. For example, empowering people to be accountable for larger important outcomes and leading swarms etc, or making it easier for people to join the conversation.
These are things we can be tracking, and seeing where in the funnel people get stuck.
Additional metrics
In addition to this, tracking things like number of nodes running Status, or ENS registrations, or Whisper volume, or public chat traffic etc, are all great too, and they all capture various nuances that are useful.
These can be bottom-up and added to an analytics dashboard as desired. More on this below in the implementation plan.
Implementation plan aka doodle time
Pardon my french drawing.
The key points here are:
- Analytics collected in one place to show the health of the Status Network.
- Work in progress is not only OK, it is encouraged! If we want to track something but haven't implemented it yet, we can link to a problem description / Gitcoin bounty.
- Filterable by time, i.e. live updated (ideally, if there are some semi-manual steps we can figure out ways to still show it)
- Individual units of metrics, i.e. people can put there what they think make sense (say, a new contract interaction or some ongoing UXR studies)
- Optionally/later divide into sections/tabs to tell a story, or make an activation ladder explicit
Conclusion and next steps
If we want to take growth of the Status Network seriously, let's be more rigorous and explicit about what success and failure looks like, on a reasonable time horizon where we can course correct appropriately.
See references below for a whole lot of links and images that give us a snapshot of what we currently have. Please let me know your thoughts in this thread. I'd like to kick of a swarm on this soon-ish, so please let me know if this something you are excited about working on.
Also see Discuss thread to join the conversation.
References part A - general links
1. Startup = Growth
Judging yourself by weekly growth doesn't mean you can look no more than a week ahead. Once you experience the pain of missing your target one week (it was the only thing that mattered, and you failed at it), you become interested in anything that could spare you such pain in the future. So you'll be willing for example to hire another programmer, who won't contribute to this week's growth but perhaps in a month will have implemented some new feature that will get you more users. But only if (a) the distraction of hiring someone won't make you miss your numbers in the short term, and (b) you're sufficiently worried about whether you can keep hitting your numbers without hiring someone new.
http://www.paulgraham.com/growth.html
2. Existing dashboards
- https://itsjustmagic.status.im/
- https://analytics.status.im/ (not updated live)
3. Jarrad's letter to Status
We can be listening to the market, our users, the Status Network, we should let them be active participants in the projects creation, whether that's testing unfinished features or requesting new ones. We're slowly getting better at this now, but we have a long way to go. If the users are skilled in fixing non-critical features, they'll probably help out and we will thank them for it.
https://our.status.im/a-letter-to-status-context-strategy/
4. Town Hall 180702, "Ascending into the blockchain"
Community is synonymous with Organization
No External vs Internal
Active Inclusivity = Network Growth
5. Town Hall 18052, "the value we generate is in the network"
the value we generate is in the network
Build with, not for (everyone is a potential contributor)
Our job is to: Convert SNT Holders to SNT Users to Active Contributors
6. Activation ladders
When somebody joins a swarm with a particular mission, they don’t go immediately from first hearing of the swarm to being its leader. There are many, many steps in between. This is obvious, but for being so obvious, surprisingly few organizations respond to it. We call this the activation ladder, and the swarm must understand each step on the ladder and make it as easy as possible for everybody to climb it.
I wrote about how the swarm only grows on its edges. The activation ladder is equally important to understanding recruitment: the edges of the swarm are not sharp, but quite fuzzy, and it’s hard to define the moment when somebody decides to activate for the first time. Is it when they hear about the swarm? When they visit its web pages? When they first contact a human being in the swarm? I would argue that all three of these are steps on the activation ladder.
The key insight here is that from the center, where the people leading the swarm are located, the swarm looks like a flat mesa (with just one steep step to climb), but from the outside, it looks like a rounded hill (with many small steps). This is key to making it easy for people to move to the highly-active center of the swarm: wanting to activate people in the swarm, it’s important to understand that this is a gradual process with many steps on the activation ladder.
https://falkvinge.net/2011/03/22/the-swarms-activation-ladder/
7. Retention for growth
Relevant where we can track it, such as for contributors. The math is brutal if you have a leaky bucket.
https://avc.com/2015/07/growth-vs-retention/
8. Removal of Mixpanel thread brainstorm
https://discuss.status.im/t/replacing-mixpanel-a-short-term-brainstorm/149
9. Privacy metrics
- https://beta.z.cash/metrics/
- https://metrics.torproject.org/
- https://zcha.in/statistics/usage
- https://en.wikipedia.org/wiki/Differential_Privacy
10. Organizational design doc
Status should design its goals around 3 metrics:
Monthly Active Contributors
Daily Transacting Users [3]
SNT value. [4]
With this in mind, we must first and foremost develop the infrastructure to build and sustain organisation. Monthly Active Contributors is by far the most important metric as it will allow us to create a sustainable and autonomous organisation.
https://wiki.status.im/Status_Organisational_Design#Goals_of_Organisation
References part B - ad hoc metrics we are collecting
1. Social media
Generally biggest audience, passive / hear about us.
2. our.status.im
Read (passive) and write (active).
3. Discuss
Read (passive) and write (active). More specific conversations. Closest to development aside from GH/Slack.
4. Slack activity
Similar can be gathered from Status public chats.
5. Github
Code and commits. Additionally, we can measure GH events like issue comments etc.
6. Github: Monthly Active Contributors and retention
7. Swarms
Swarms, active projects being successfully lead by (core contributors) right now.
8. Nodes
<Missing analytics for nodes in network etc, can be discovered by crawling network>
9. Weekly downloads
10. Whisper volume
Proxy for chat activity.
11. SNT contract
12. ENS contract
13. Dogfooding
14. SNT holders
Initial distribution.