xi editor and modularization

July 4th, 2020

In the 🔗 xi editor retrospective, Raph touches on modularization’s costs that I have experienced myself.

Looking back, I see much of the promise of modular software as addressing goals related to project management, not technical excellence. Ideally, once you’ve defined an inter-module architecture, then smaller teams can be responsible for their own module, and the cost of coordination goes down. I think this type of project management structure is especially appealing to large companies, who otherwise find it difficult to manage larger projects. And the tax of greater overall complexity is often manageable, as these big companies tend to have more resources.

…monolithic architectures, perhaps ironically, are better for community. Everybody takes some responsibility for the quality of the whole.

My experience at Snowflake supports this assessment. Building modules or microservices means you’re committing to supporting stable interfaces and migration paths. While good in theory, it’s nearly impossible to anticipate how new features or bugfixes will touch the codebase and anytime a change spans two services, you have to be much more thoughtful around how it’s deployed and rolled back.

I think one of the strongest positive models was the database / business logic split, which is arguably the most successful example of process separation. In this model, the database is responsible for performance and integrity, and the business logic is in a separate process, so it can safely do things like crash and hang.

Even in the case of application/database, maintaining and migrating this interface isn’t trivial. Ask anyone who’s done ops and they’ll tell you database migrations must be performed carefully to maintain stability, performance, and uptime. Modularization is useful because they allow a separation of concerns. This can be achieved by better organizing and documenting your codebase, but modularization can enforce it so long as you’re willing to pay its cost.