Recently, iCharts CEO Seymour Duncker participated in a panel of experts on analytics. The panel included Seymour, along with Eileen Tobias, VP of Business Strategy at NetSuite and Greg Kleiner, VP of Investor Relations and Treasurer at Twilio. The focus of the conversation was how members of the panel choose what metrics they use to measure the health of their organizations.
For SaaS organizations, certain metrics have emerged as the de facto standard. They serve as benchmarks, and they’re frequently used to compare different companies. Here are the five most common SaaS metrics…
- ARR or CARR: Committed Annual Recurring Revenue
- CAC Payback Period: How long it will take to recoup the customer acquisition cost (CAC)
- Churn: How many customers you’re losing, and at what rate.
- CLTV/CAC: Customer lifetime value divided by customer acquisition cost
- Cash Flow: How much money is coming in?
With these metrics, one can gain a fairly solid understanding of the health of a SaaS business. If SaaS companies were baseball players, these would be the numbers on the back of their cards. No metric is perfect, but these provide a nice summary. “I easily watch forty to fifty metrics, but I focus on the ones that matter,” said Seymour. These five metrics essentially represent the “bottom line.” They may not show you what to work on, but they can show you something is off.
Here are 5 of the key takeaways from the panel…
1. The difference between 1st order, 2nd order, and 3rd order metrics
We use metrics for different reasons. We’ve covered first order metrics above. They’re the metrics you’d want to see if you were an investor, for example. They answer questions about the health of the business, and provide an overall sense of where the company is.
Second order metrics serve as indicators. We’ll get more into this later, but the basic idea with second order metrics is that they offer actionable information. While first order metrics can tell you something is wrong, second order metrics show what you can do about it. These are the kinds of metrics you could use if you were a CEO or an executive.
Third order metrics are a lot like second order metrics, but on a much smaller scale. They’re the kind of metrics you’d want to see if you were a regular worker. A guy on an assembly line probably doesn’t have much use for a second order metric like regional sales numbers, but if you told him a shipment of bolts was going to be late, it might affect his whole day.
2. When you cherry pick metrics, you end up fooling yourself
It’s natural to look for a silver lining to bad numbers, but it can be a slippery slope. Every data analyst knows a trick or two to make numbers look better than they really are, but these deceptions hurt you in the long run. Even if one is never caught goosing the numbers, the danger is in covering up real issues. “I’ve seen it in other companies, a tendency to throw out certain metrics. When you stick with metrics, you can really see what’s going on,” said Tobias. One can argue the entire point of analytics is to find objective truths in data. By stepping on the scales, you lose that objectivity.
Coverups also tend to raise questions among stakeholders. “If you change something, people will assume something has changed and now you want to hide it,” said Kleiner. “Be very careful with what you share externally. You’ll have to live with it for a long time.” Instead of using analytics to inform next steps, people start to use analytics to hide past mistakes. If it gets bad enough, the benefits of analytics begin to disappear.
3. The metrics you use will evolve depending on the stage of your company
One of the benefits of consistent metrics is the ability to measure performance over time, however, your KPI needs will evolve over time. For example, churn is usually the most important metric to an early-stage SaaS business. However, by the time they’ve had some success, the organization may be more worried about CAC payback periods. As Seymour put it, “the metrics you use depend on what you’re focusing on.” Eileen had this to add, “When I first joined NetSuite in 2005 we were all about gaining new customers. We didn’t focus on secondary metrics. Over time, we became more sophisticated and got better about retention and upsell.” When you start with analytics, it’s tempting to try to build a “perfect” dashboard that you can live with for years, but needs change. A better approach is to focus on metric you know you’ll use, and add more as you need them.
4. Find the metrics that serve as indicators to bigger issues
Where there’s smoke, there’s fire, but most metrics only tell you something burned down. That said, every fire leaves clues to what caused it. “Don’t steer away from a metric set because you don’t like the results. When metrics aren’t looking how you want, try to figure out why,” said Tobias. For metrics you can really use, you’ll need to do some digging and you may uncover something ugly. That said, an ugly truth is more useful that a pretty lie. Take failures as an opportunity to figure out what went wrong and how you could have spotted it earlier. Search for correlations between certain metrics. For example, one can often draw a direct correlation between the number of sales calls and the number of deals closed. If all you know is how many deals you’ve made, you wouldn’t know you could get more by focusing on your call strategy.
5. Gain the ability to take action in real time, or it may be too late
The ability to make real-time decisions is increasingly important. As Eileen put it, “we’re moving towards a real-time world, and you need the ability to react in real time.” It’s a statement echoed by anyone who follows the analytics industry. However, people often misinterpret the message. When one pictures someone taking action in real time based on analytics, they tend to picture an executive sitting in a captain’s chair, but the real value to real-time action occurs at lower levels of the company.
These third-order metrics may not reveal any sort of truth about the organization, but they make people’s jobs easier day to day. For example, a distribution executive might learn that there’s a delay on an order of stock. For the executive, this is important information, but it isn’t exactly a real-time issue. However, for a worker on the floor, this kind of information can completely change their day. If the worker can learn about an order delay ahead of time, they can adapt in real time and focus on other duties. When you extrapolate this scenario to your entire workforce, the effect can be tremendous.
For more helpful tips for choosing KPIs, check out our resources page.