The Problem With Metrics
How Graphite tries to balance the internally important with the externally important.
We rely on metrics for everything. Go to the doctor, check your heart rate and your blood pressure and your weight and on and on. Go to the mechanic, check your fluid levels and your air filter and your shocks and so on. Yet, in software and often business in general, the metrics we track are very different than those real-life metrics. Software metrics are malleable while real-life metrics are (usually) not malleable—your weight is your weight and your car has oil in it or it doesn’t. The malleability of software and business metrics is where the problem lies.
Yet we all rely on them for some of the most important decisions.
Let me give you a non-software example from my past life. I once worked in a call center, and then I subsequently managed people in that same call center. When you’re on a call, you’re expected to document the conversation as you go. This is not because documenting while you’re talking to someone is proven to be a more accurate reflection of the conversation, but because in doing so, you can be immediately available for the next call. But not everything can always be documented on the call or in a natural flow, so there was a button on the phones that would prevent another call from coming in. This button would trigger a “Call Work” timer.
So far, no big deal. You finish a call, you go into “Call Work” and document it. However, “Call Work” was a metric that was tracked heavily in this call center. The goal was to keep it as close to zero as possible. So, managers would get on a representative who was using “Call Work” and representatives would find ways to keep that “Call Work” time down. Do you think the methods to keep this metric suppressed were beneficial to the customer?
Of course not.
Representatives would artificially delay the end of a call, keeping a customer on the phone while they “looked into the account” or “checked on one final thing.” All they were doing was shifting that “Call Work” time to on-the-call time. The metric—Call Work—looked great when people did this. Oh, Susan averaged 3 seconds of Call Work this week? That’s fantastic! All the while, Susan’s customers spent minutes longer than they needed to on each call.
If you don’t get the point of the story, it’s this:
Metrics that are malleable invariably lead to tricks and gimmicks to help keep those metrics sparkling. Let’s shift back to software metrics now where we see this all the time but may not realize we’re seeing in.
In the traditional web and the decentralized web, user acquisition is a huge concern. No users equals no app equals no business. So, close to 100% of software applications try to keep track of total users. But most take that a step further. They want to know what their Daily Active User counts are. They want to know about Weekly Active Users and Monthly Active Users. Is a user still a user if they are not consistently active? I argue yes, but most of the rest of the software world argues no.
This idea that active user counts as a metric is the measure to end all other measures largely grew out of the freemium/advertising model—the model Google and Facebook pioneered. Facebook, Google, Twitter, et al, care about daily active users because their advertisers care about daily active users. Advertisers don’t want to advertise without guarantees. Daily active user counts guarantee the number of possible eyes on an ad.
Or does it?
This is where the malleability of the daily active users metric comes in. Have you ever received an email from an app you use saying something like “Carla just posted for the first time in a while”? Or maybe you’ve received a push notification saying something like “It’s been a while since you logged in. Log in now to check out what’s new.” Sometimes the messages are more urgent, but most of the time they play off our FOMO (fear of missing out). They get a quick login, track you as an active user, and move on. Even if you only logged in and didn’t do anything else, you are very likely now part of their inflated daily active users pool.
Despite the fact that tracking daily active users as a metric mainly applies apps with advertising-based revenue models, the obsession has spilled over to almost ALL apps, including decentralized apps.
If you visit State of the Dapps rankings page, you’ll see rankings of Ethereum-based decentralized applications.* The rankings are based on active user counts as defined by Ethereum transaction volume (since most Ethereum apps require some sort of transaction on-chain to use them). But do those metrics tell a good story?
[*]Pet peeve alert: Ethereum apps are not the only decentralized apps
Not usually. They tell a story of apps that have people spending ETH. But why are those people spending ETH? Are they doing it to use the app because they want to? Are they doing it because of a marketing gimmick where the app airdropped a bunch of coins with the stipulation that users had to spend X% on transactions in-app?
If the decentralized app is relying on ad-revenue, then sure, track daily active users. Track everything about your users. Copy the old traditional app model. But if your business model is different, if you rely on a SaaS model with no ad revenue, if you are running an eCommerce (traditional or crypto) site, if you are doing anything but ad revenue business, then use less malleable metrics.
For Graphite, I am more than comfortable not knowing much (if anything) about my users. I am comfortable tracking total user counts. I’m comfortable tracking week over week user growth. Graphite is not a social app. It’s not a FOMO app. It’s a productivity app. If you feel like working on something (personally or for actual work) one day but not another day, that’s cool. You’re still a user. I don’t care if you use the app every day or not. I just care that people use Graphite and that people are happy with Graphite.
Are there other metrics that would be beneficial? Sure. And Graphite may track other things (anonymously and without storing ANY user data, of course). But if and when those things are tracked, they will be because they are meaningful to Graphite’s business, not because the rest of the world says to track those things.
And if you’re interested, Graphite just passed 3,000 total users and is seeing an average of just under 9% week-over-week growth. I’m pretty damn proud of that, and I am happy to work towards sustaining that growth and getting more people using something that is the antithesis of Google and Facebook.
Feel like using an app that just cares if you’re satisfied not that you’re addicted to it? Give Graphite a shot:
