Part 1 of this post outlined the three steps—and the three corresponding decision trees—that every analytics professional in the B2B SaaS 1 space should use when evaluating a project proposed by other business stakeholders. In many cases, those simple but effective tools reveal that the stakeholder’s proposed project is not worth doing. But, what happens when the three decision trees all signal the project might be worth doing, as shown below?

  • Step 1: Is it obvious what the right decision is, even without this analysis / model / investigation?
    • Answer 1: No. → Decision 1: The project might be worth doing.
  • Step 2: Will this analysis / model / investigation actually be used to make a decision?
    • Answer 2: Yes. → Decision 2: The project might be worth doing.
  • Step 3: If instead of doing this analysis / model / investigation, you did a “back-of-the-envelope” calculation, would you get ~80% of the insight you need?
    • Answer 3: No. → Decision 3: The project might be worth doing.

Then, it’s time for the fourth step: calculating the benefit-cost ratio of the proposed project. It’s one way to do what analytics (and product) professionals call opportunity sizing, though it might be confusing if we use the term opportunity sizing because, in this context, the decision is guided by the (financial) bottom line of resourcing full-time employees. That is, if staffed on the proposed project, will the financial cost of the analytics professionals’ salaries be offset by future financial gains for the business from completing this project? It’s really about the financial viability of the project from a resourcing perspective.

Step 4: Estimate the benefit-cost ratio of the proposed project and make a final call

4a: The benefit-cost ratio

The benefit-cost ratio can be used to estimate the cost-effectiveness of a proposed business project. It’s useful only if both the value of the benefit and the value of the cost are expressed in the same unit, which, in the context of evaluating business projects, should be a financial unit, like USD ($).

Analytics professionals who are evaluating proposed business projects can use these definitions of benefit and cost:

  • Benefit ($): amount of labor cost decrease or net new revenue this project would help generate (summed if both available)
  • Cost ($): amount of money the business will spend on the analytics professionals to do this project
Benefit-cost ratio formula, and key value to monitor when deciding whether a project is worth doing. It is calculated as a fraction: benefit divided by cost, where both values are measured in some currency, for example USD.

A project is considered cost-effective (in other words, worth doing) if it’s greater than 1 because the benefit is higher than the cost. That’s the textbook definition of cost-effectiveness, but in many real-life business scenarios, this threshold will need to be higher, like 5 or 10. This is especially true if the business is cash-strapped and needs to invest in only the most valuable projects.

Step 4 takeaway: The benefit-cost ratio can be used to quantitatively assess if a proposed project is worth doing. It’s a dimensionless ratio calculated by dividing the estimated benefit of the project by the estimated cost of the project. The textbook definition says the projects worth doing always have a benefit-cost ratio greater than 1, but, in reality, the threshold is usually higher than that.

4b: How to estimate the benefit ($)

Calculating the true benefit ($) of a project in B2B SaaS space can be complicated. Do you consider impact on the overall sales motion? Or just new-business deals? Renewals? Expansions? Upsells? How do you know much credit to give to the analytics professionals; after all, they are not the ones facing the customers?

For this particular purpose, it’s okay to estimate and make assumptions, because the analytics professional has to make a quick decision on whether to say yes to a project. Let’s use the definition provided earlier and assume that the benefit ($) of a proposed business project can be estimated as the sum of estimated labor cost decrease and estimated net new revenue.

Formula for estimating the benefit value in the benefit-cost ratio. It is calculated as the sum of estimated labor decrease and the estimated net new revenue that the proposed project would generate.

To estimate the labor decrease of the project, we need to provide best estimates for the following four values:

  • approximate number of business stakeholders trying to generate these insights manually (the assumption here is that the business stakeholder wants help from the Analytics team because the business stakeholder can’t easily generate these insights independently)
  • approximate number of hours per week a single business stakeholder spends doing this manually
  • (composite) hourly salary of the business stakeholder(s)
  • number of working weeks in a year that the business stakeholder(s) spend(s) doing this task

Once we have the best estimates, we multiply the four values as shown in the image below:

Formula for estimating the labor cost decrease for the benefit value in the benefit-cost ratio. It is calculated as approximate number of business stakeholders generating insights manually multiplied by approximate number of hours per week each stakeholder spends doing this manually multiplied by composite hourly salary of the business stakeholder multiplied by the number of working weeks in a year that the business stakeholder spends doing this task.

The idea behind this calculation is that it represents how many business stakeholder hours (and therefore, how much of the business stakeholder salary) would be saved and invested elsewhere if the analytics professional did this project. For instance, if a business stakeholder is spending six hours each week visualizing data in Google Sheets, an analyst could build a dashboard in a BI tool like Looker or Tableau, which would free up the business stakeholder’s time because the visualizations are now automated in the dashboard.

The second part of the benefit equation is the net new revenue this proposed project would bring. Again, we could make this a complex equation by accounting for all nuances (do we think in terms of customer accounts or individual deals? how do you account for multi-year deals with renewals? do you account separately for new-business deals versus expansion deals? do you handle each type of expansions separately?), but for quick decision-making, we can provide best estimates for these three values:

  • estimated number of net new accounts this project would help bring in through sales
  • estimated number of accounts this project would help prevent from churning (objectively, these accounts are not net new, but let’s ignore that technicality to make the calculation easier)
  • average annual (customer) contract value ($)

Once we have the best estimates, we multiply the ACV ($) with both the estimated number of net new accounts and the estimated number of accounts saved from churning as show in the image below:

Formula for estimating the net new revenue for the benefit value in the benefit-cost ratio. It is calculated as estimated number of net new accounts this project would help bring in through sales multiplied by the account's average contract value plus estimated number of accounts this project would help prevent from churning multiplied by the account's average contract value.

Notice that I am glossing over some of the mathematical imprecision, like using contract-level average annual values and multiplying them with the number of customer accounts, so it’s unclear which level the values are averaged across (over all contract values? over accounts’ average contract values?). Because it’s a quick calculation, let’s assume they are the same.

The idea behind this calculation is that it considers whether the proposed project could help generate revenue. It’s best to be conservative in this case because it is highly unlikely that the analytics professional’s project on its own would generate revenue. Instead, the project would make the business stakeholder (like the customer success manager or the sales rep) more effective at their job, which would presumably make it easier for them to impact the go-to-market motion. Two ways to make it conservative: (i) either use a low number of accounts, or, more intuitively, (ii) multiply the final value by a percentage (%) that represents how much credit the analytics professional can take in generating net new revenue.

Step 4b takeaway: To estimate the benefit of the proposed project, the analytics professional needs to estimate the labor cost decrease and the net new revenue that the proposed project would generate. Some projects will generate one or the other, and some projects will generate both. Automation of manual work (for instance, making a dashboard or a “click and run” Python notebook script) is what usually drives labor cost decrease. New tools and capabilities (like predictive models that inform go-to-market or product strategy in a data-driven way) are usually what drive net new revenue.

4c: How to estimate the cost ($)

Calculating the cost ($) of the proposed project is a bit easier.

Formula for estimating the cost of working on this project for the benefit value in the benefit-cost ratio. It is calculated as estimated number of hours that will be spent on the project (by analytics professionals) multiplied by the composite hourly salary of the analytics professionals working on the project.

We need to provide best estimates for the following two values:

  • (composite) hourly salary of the analytics professional(s)
  • estimated number of hours that the analytics professional(s) would spend on this project

If the analytics professional evaluating the project is the only full-time employee who would work on this proposed project, then finding the hourly salary is straightforward (divide annual salary by the number of working weeks in a year and by the number of working hours in a working week). If there are other full-time analytics employees involved, then it’s best to use a composite hourly salary of all the analytics professionals, and account for all of their hours. The composite hourly salary can be estimated by finding the median annual salary across these roles using a tool like Glassdoor or levels.fyi.

Step 4c takeaway: Calculating the estimated cost of the project is usually easier than calculating the benefit. The analytics professional needs to know the projected number of hours the Analytics team would spend on the project and the (composite) hourly salary of the teammates working on the project. Hours spent on project maintenance, stakeholder enablement, as well as any post-mortem work should also be included in the projections.

4d: The units of benefit and cost

This step is purely informative, in case you are like me and like to map units to each metric when running calculations.

For metrics behind the benefit calculation, we have:

  • approximate number of business stakeholders trying to generate these insights manually [person]
  • approximate number of hours per week a single business stakeholder spends doing this manually [hr / week / person]
  • (composite) hourly salary of the business stakeholder(s) [USD ($) / hr]
  • number of working weeks in a year that the business stakeholder(s) spend(s) doing this task [week]
  • estimated number of net new accounts this project would help bring in through sales [account]
  • estimated number of accounts this project would help prevent from churning [account]
  • average annual (customer) contract value ($) [USD ($) / account]

For metrics behind the cost calculation, we have:

  • (composite) hourly salary of the analytics professional(s) [USD ($) / hr]
  • estimated number of hours that the analytics professional(s) would spend on this project [hr]

The image below shows that these units cancel each other out and that we end up with a dimensionless benefit-cost ratio.

Formula for combining estimated labor cost decrease, net new revenue, and cost of working on project with unit metrics. The image shows the benefit-cost ratio is ultimately a dimensionless metric.

Let’s demonstrate why benefit-cost ratio works. We can do that by stress-testing it with two extreme scenarios, for which it would not even be necessary to calculate the BCR: (i) a project that is obviously not worth doing and (ii) a project that is obviously worth doing.

4e: Stress-testing the benefit-cost ratio: Extreme scenarios

In the first scenario, a product manager asks if someone from the Analytics team could build data warehouse pipelines (net new data tables using ETL tools) and then a net new business-intelligence dashboard that would help the product manager track test data in preproduction environment produced by engineers as part of a larger year-long initiative to revamp how clickstream events are instrumented. According to the product manager’s roadmap, the product manager will spend close to 2 weeks every quarter, and about 2 to 3 hours per week, exploring and testing the preproduction data, and it would be great if the product manager could access the preproduction data easily in a business intelligence tool.

Why is this obviously not worth doing? Well, it is understandably an inconvenience for the product manager to have to query the test data in the preproduction environment each quarter, having to potentially rewrite code or adjust it, download the data, and do visualizations in a tool like Google Sheets. At the same time, it is preproduction test data—it’s used for just this project, just to quality-check the data for potential issues, and it will be used for at most two weeks each quarter, which means for at most eight weeks in the entire year. In addition to all that, preproduction data tends to be patchy and sparse. To invest in production-like data architecture for this proposed project would clearly be a poor decision.

The benefit-cost ratio is not even necessary in this scenario, but it can be used to illustrate why it would not be wise to work on this project. Let’s use the following best estimates (for hourly salary, we will use a simple placeholder of $50/hr for all full-time employees and for average annual contract value, we will also use a placeholder of $500K/account]—

Values needed for the numerator (benefit):

  • approximate number of business stakeholders trying to generate these insights manually [1 person]
  • approximate number of hours per week a single business stakeholders spends doing this manually [3 hrs / week / person]
  • (composite) hourly salary of the business stakeholder(s) [$50 / hr]
  • number of working weeks in a year the business stakeholder spends trying to generate these insights manually [8 weeks]
  • estimated number of net new accounts this project would help bring in through sales [0 accounts]
  • estimated number of accounts this project would help prevent from churning [0 accounts]
  • average annual (customer) contract value ($) [$500K / account]

Values needed for the denominator (cost):

  • (composite) hourly salary of the analytics professional(s) working on this project [$50 / hr]
  • estimated number of hours that the analytics professional(s) would spend on this project [~1 working week = 40 hours]
Benefit-cost ratio calculation using numbers for a project that is definitely not worth the effort for an analytics team. This project has a benefit-cost ratio of 0.6.

With a benefit-cost ratio of 0.6, the project would produce net loss for the business, and is therefore not worth taking on.

For the second scenario, let’s imagine the Sales team asks if the Analytics team can build a predictive model that scores new-business accounts in the sales pipeline for their likelihood of closing. The team wants to use the data to allocate full-time employee resources strategically; right now, seemingly promising deals fall through at the last moment, and the team is pulling data from business-intelligence tools every week to no avail—the descriptive data is clearly not helping them. According to the team, about 30 sales reps are trying to make sense of the descriptive data each week, and each rep spends about 8 hours per week collecting the data from business intelligence tools, stitching different metrics, and trying to make sense of what the data is saying. Put differently, it’s crisis mode for the Sales team.

Why is this obviously worth doing? First, it’s every analytics professional’s dream opportunity: to build a predictive model that will be used continuously by the business to make strategic decisions and that will provide unique intelligence to the business that otherwise could not be obtained. Just imagine if the predictive model was built and implemented successfully: so many hours of unnecessary work would be saved for sales reps and appropriate resourcing around the pipeline accounts would ensure more deals are closed. It’s most certainly a hefty investment for the Analytics team because it would require exploratory analysis, model building, testing, and then enablement. But, that’s exactly what the team is uniquely positioned to do.

The benefit-cost ratio is also not necessary in this scenario, but it can be used to illustrate why this project should definitely be prioritized by the Analytics team. Once again, let’s use the following best estimates (for hourly salary, we will use a simple placeholder of $50/hr for all full-time employees and for average annual contract value, we will also use a placeholder of $500K/account]—

Values needed for the numerator (benefit):

  • approximate number of business stakeholders trying to generate these insights manually [30 persons]
  • approximate number of hours per week a single business stakeholders spends doing this manually [8 hrs / week / person]
  • (composite) hourly salary of the business stakeholder(s) [$50 / hr]
  • number of working weeks in a year the business stakeholder spends trying to generate these insights manually [~A full working year = 48 weeks]
  • estimated number of net new accounts this project would help bring in through sales [~5 accounts]
  • estimated number of accounts this project would help prevent from churning [0 accounts]
  • average annual (customer) contract value ($) [$500K / account]

Values needed for the denominator (cost):

  • (composite) hourly salary of the analytics professional(s) working on this project [$50 / hr]
  • estimated number of hours that the analytics professional(s) would spend on this project [~A full quarter = 12 working weeks = 480 hours]
Benefit-cost ratio calculation using numbers for a project that is definitely worth the effort for an analytics team. This project has a benefit-cost ratio of 128.2.

With a benefit-cost ratio of 128.2, which is hyperbolic because this is an extreme scenario, the project would produce notable net gain for the business, and is therefore worth taking on. Most projects will never have a benefit-cost ratio this high, but in this example, the ratio illustrates the magnitude of the proposed project’s benefit to the business.

Step 4e takeaway: If you are calculating a benefit-cost ratio, it’s very likely that it is not obvious if the project is worth doing. Very few projects will have BCR ratios as low or as high as the two illustrated extreme scenarios; instead, most BCRs will hover between 1 and 10, or potentially between 10 and 20. If the calculations are consistently producing high BCRs, it’s likely that the benefit estimates are too liberal, and this is most often a result of overestimating net new revenue.

5: The full “When to say no” decision tree

We now have a full decision tree for when to say no to a proposed analytics project in the B2B SaaS space (the thresholds for Step 4 will vary depending on the context of the business):

  1. Step 1: Is it obvious what the right decision is, even without this analysis / model / investigation?
    • Answer 1: No. → Decision 1: The project might be worth doing.
  2. Step 2: Will this analysis / model / investigation actually be used to make a decision?
    • Answer 2: Yes. → Decision 2: The project might be worth doing.
  3. Step 3: If instead of doing this analysis / model / investigation, you did a “back-of-the-envelope” calculation, would you get ~80% of the insight you need?
    • Answer 3: No. → Decision 3: The project might be worth doing.
  4. Step 4: Run a benefit-cost ratio calculation for the proposed project.
    • Answer 4: BCR less than 1.0 → Decision 4: The project is not worth doing.
    • Answer 4: BCR equal to or greater than 1.0 and less than 10.0 → Decision 4: The project is worth doing, but should not be a top-priority project.
    • Answer 4: BCR equal to or greater than 10.0 → Decision 4: The project is worth doing and should be a top-priority project.

At first, this might seem like a tedious process , but after a few trial runs, it starts to feel natural and, most importantly, it reduces guesswork.

  1. Hyper-condensed acronym for “business-to-business (B2B) software-as-a-service (SaaS).” ↩︎

Leave a Reply