Build versus Buy

September 27th, 2017

The decision to build versus buy recently came across my desk. People like to say engineers bias towards building, so I wanted to apply an purely economics lens to the decision. In my analysis, I treat vendors as “negative customers”. What happens if we apply the same SaaS economics to vendor purchasing?

In this case, we’re looking to do some simple A/B testing at Sentry. We compared building against buying Optimizely, which seems to be the leader in this segment (though many, many people seem to build, including Facebook, Dropbox, Intuit, etc).

Costs

I’m valuing engineering time at $1,500 per day ($375,000 a year). At most small to medium sized company, an experienced engineer salary will be $150,000 and $100,000 in equity. Increasing head count doesn’t come without costs though, which is why I include an additional $125,000. Management, recruiting, human resources. The bureacracy is expanding to meet the needs of the expanding bureaucracy.

Optimizely quoted us $50,000 - $70,000 per year. I’m going to use the lower price and I’m also assuming a week’s worth of engineering time to initially integrate it (coming from someone who just spent 4 weeks integrating Hubspot, this is optimistic), bringing the cost to a low-ball of $57,500. I’m treating this as “negative CAC”.

Assuming our business has 90% margins and a CAC payback period of 1.5 years, what would the business pay to build $57,500 in customer revenue? Afterward, what should the business pay for ongoing maintanence?

$86,250 in CAC equivalent for initial development $5,000 in COGS equivalent for infrastructure and maintanence (the same COGS we’d be willing to sink into revenue should be sunk into the double negative revenue for paying vendors).

Infrastructure is laughably cheap with IaaS (almost no ops time is needed). An analytics database would end up cost in the low hundreds of dollars a year. For Optimizely’s prices, so long as building takes us less than 2.5 months for development and 3 days for maintanence a year, we should build.

There’s a couple things that make this decision uniquely easy.

1. For API-only products, it’s considerably easier and cheaper to build. You don’t require product management or design time, slimming the team to a single developer and (maybe) a few hours of ops time. If your team requires a UI and WYSIWYG chart editors, you should basically double all your build costs.
2. Initially, you only need a subset of functionality, with your usage expanding over time. This means you don’t need to drop everything and spend 3.5 months of development to have feature parity. In our case, an MVP should take less than 2 weeks to build.
3. A/B testing is well defined. There is good literature and open source implementations of A/B testing that we can pull from. This makes the costs of building simpler and more predictable.
4. Compliance with 3rd party infrastructure tools takes a bit more work.

My estimate is that building our own will take roughly 4 weeks (spread out over the course of the next year). Reversing the calculation, that would mean we should be willing to pay up to $20,000 per year.