Numeracy's acquisition

August 11th, 2021

Two years in

Snowflake bought Numeracy to replace their UI and bring BI capabilities to their customers. Two and a half years later, despite 10x the number of engineers working on Numeracy¹, the old UI is still being used by customers and the new UI is still not “full GA”. The acquisition wasn’t a failure, but it definitely wasn’t a resounding success. When we were bought, leadership assumed the integration might take a year, maybe a year and a half. After all, Numeracy literally worked with Snowflake! Wasn’t it just a matter of swapping out the user login system? What happened?²

Any choice which involves forcing dozens or hundreds of people to study abstruse details about a system which they have no fundamental interest is doomed to failure—most of the group will just not bother, and will end up choosing essentially randomly, with a slight bias to whatever is most familar, assuming that familiar choices will be less likely to be really bad. -Keith Packard on his selection of git for X.org

One of the things I admired about Stripe (from afar) was their written tradition. Thoughtful mailing lists discussions. Writing is such a powerful art. So many people understand a small piece of the puzzle, but a large organization can never arrive at consensus through dozens of meetings or hundreds of 1:1s. You end up with n! perspectives, a lot of “planning” at the center and disconnection at the edges. Importantly, Jira is no replacement for planning. You need prose. Writing whose whole is greater than its breakdown into individual sub-tasks. You might think that pandemic would have promoted this behavior, but discussions still centered in meetings and Jira became the source of truth. Stripe created this writing structure when it seemed most impossible to have a writing culture: when it was a small team working out of the same office. Writing must be a practice.

Onto the specific issues!

The first and constant issue at Snowflake was always security. While you might sell Snowflake to CIOs and (sometimes) CEOs, but buying Snowflake is different from adoptiong Numeracy, specifically because the old UI still existed and its very limitations were a strength. The inability to share meant data security by default. So, the blockers to Numeracy adoption are the security teams and Snowflake admins. We had initially thought that, like our adoption at DoorDash, if we had enough users at a company, the admins would eventually see the light. We should have known DoorDash was the exception, not the rule, because literally in our attempts to market and sell Numeracy, they were our one and only exceptional customer.

I suppose some truths are hard to see when you’re a hammer. Snowflake had bought us for our great product, so our product must be great.³ But just because they were wrong, doesn’t mean that we were right. Sure, aircraft passengers shouldn’t be confronted with complexity, but aircraft cockpits are complicated for a reason.

In particular, security had a funny way of intermingling with cache invalidation. Part of building a performant database UI is caching. When searching for database objects, like columns that start with “org”, caching is vital. Querying the database on keystroke can literally take seconds, when you need results in (ideally), <30ms. We would aggressively cache not only data objects, but also users and roles, to determine if a user still had access to some data. When Numeracy was a BI tool, it was okay if we were a little laggy with updating our view on the database. As a BI tool, you accept it as a layer on top of the database. However, when you’re the database UI, the user expects things to be reasonably up-to-date (at least on page refresh) and the admin’s expect revocation of privileges to be enacted immediately. However, Snowflake didn’t have hooks or push notifications of changes to the security model, so we just had to poll. And Numeracy didn’t have a notification system for our UI, so the UI code just had to…try? Suddenly, you cannot trust any data because at any time, it could be declared stale or inaccessible.

Eventually, we came to realize that caching on the Numeracy backend was a fool’s errand. We were literally loading data from the Snowflake database into a Postgres database so we could serve it to the client when requested. We cut out the middleman, built a SQL passthrough and a query builder in the UI code itself, but this unfortunately took almost two years to realize.

In retrospect, one of the biggest misses we celebrated was standing up a on-premise Numeracy server within the first month of our arrival. Yes, it meant that internal users could try out Numeracy, but it pinned our thinking to the current product. Granted, Snowflake’s leadership thought that’s what they wanted, but DBs and BIs are vastly different products.

We were building a BI at a company that was busy selling a database and whose long-term vision was for a data marketplace, not BI. The emporer had no clothes, but we were his tailors. God, when Google bought Looker, we got a new shot in the arm, because by god, if Google was going into BI, so were we. No matter that Looker would remain effectively separate from BigQuery and BigQuery would retain its own UI.

What should have happened?

To measure success, you need to ask, What’s the win condition? Success as a startup is vastly different than success as an acquisition. Namely, Numeracy’s job at Snowflake was not to increasing customer acqusition or contract size: Snowflake itself was doing that just fine. If Snowflake were a car, the database itself was the engine and Numeracy the body. Yes, the body should be aerodynamic, but fundamentally, the body’s job is to showcase the car, not drive it. Unless Snowflake was willing to allow people to use Numeracy without using Snowflake and the Numeracy product be responsible for its own P&L, Numeracy should never have thought of itself as competing in the BI market. We were a database UI. We should build a database UI. It might sound obvious, but deep within the bowels of a company, it’s sometimes hard to know which way is up.

This meant from day 1, we should have abandon dashboards, parameters, and maybe even worksheets. It’s hard to imagine, because the core Numeracy experience was a very Google Docs-like “I should be able to write a query, create a chart, share it with someone else, who can then modify it and send it back to me.” All of this clashed with Snowflake’s security model, but we (including Snowflake leadership) considered Numeracy a product being built on top of Snowflake’s platform.

But we weren’t building a product. You couldn’t use Numeracy without Snowflake. Snowflake was the product. We weren’t even building Snowflake’s primary interface. SQL was the user interface. We were building a pixel interface on top of a SQL interface.

Numeracy still should have its own data models, but we should have stripped away any non-essential features. We would have shelved dashboards and parameters much sooner and potentially arrived at a much simplified sharing model. We should have even deprecated my favorite feature: server-generated column insights, which generated column statistics based on our assessment of the column’s data type. Once again, we were a pixel interface on top of a SQL interface. Any features involving data processing, job scheduling or data security (this is especially true for sharing) should have been shelved and moved into Snowflake database.

Finally, we should have immediately focused on building the admin UI and in particular, UI for DDL. So much of Snowflake UI’s needs were building a UI for DDL statements. But any DDL statement was more options and arguments that any designer should rationally want to represent in a UI. In the beginning, we built (and later auto-generated) endpoints per statement type, replete with lots of query parameters. This was insane, as if something were inherently wrong with SQL. Yes, SQL has its warts, but so does JSON and REST. And going back to it, Numeracy was not meant to be a product but a pixel interface to Snowflake’s SQL interface. Because Javascript is the language of the browser, building a layer that constructs SQL with JavaScript seems so obvious in retrospect. Making sure that layer covered the 80% use case, instead of being feature complete, seems doubly obvious…you shouldn’t be trying to achieve pixel pariety with a god damned programming language.

What I learned

I personally wish I had been bolder. The acquisition was a relief but also disheartening. I felt I had failed to find a go-to-market for Numeracy. I should have considered how Brian (the founder) felt, having “given up” a company he worked on for more than four years. I wish I had trusted that I was still the same person, that a bad market beats a good team anyday, we’d moved the team from a arid BI landscape to a more fertile database one. This team still had to “get to market” and I should have faced the new problem of how to bring Numeracy to Snowflake’s customers and even what Numeracy was in the context of Snowflake. Winning should not have been “integrating Numeracy into Snowflake” (as it was termed in our contracts), but delivering the UI the customers needed with Numeracy’s unique perspective.

I should have picked up the pen and wrote. Wrote my perspective, my learnings, my misgivings. I still remembering at Dropbox, I would write a weekly letter about noteworthy apps on the “Dropbox platform”. People later told me that it was the only thing that made them remember Dropbox still had a platform that people used and loved (this was in the heyday of Carousel). It felt like such a menial task at time, but…writing has tremendous power, not only to influence others but to shape narrative, force discussion, and clarify shared goals. And the funny thing is that you don’t need to be good at writing to do it, you just need to keep writing. People will learn your voice…including yourself.

So to my future self, write boldly, more.