Continuous customer acceptance testing

April 29, 2026



For customer contracts, legalese is worse than code. Code is more precise, consistent, and enforceable. Just like we test functions with unit tests and APIs with integration tests, customers should define acceptance tests — not uptime percentages — for vendors they rely on.

Software has leaned on vendors for decades: every npm install is effectively adding a new vendor. Yet in package management, dependency hell is managed by version pinning and testing. The frustrating part of hosted vendors is the impossibility of version pinning and the futility of integration testing (unless specifically offered by the vendor, like Stripe Sandboxes). Continuous deployment has meant continuous change, but when you build on infrastructure, continuous instability.

For Kernel, browser infrastructure adds another layer of instability: the target websites themselves. The DOM, captchas, memory consumption — all changing constantly, and impossible to sandbox1. When an automation fails, it’s a three-way blame game: your code, the browser infra, or the target website? A workflow automating a UnitedHealthcare site might break because the site shipped a new version, because Kernel’s stealth proxies started failing, or because the customer’s Playwright script has a bug. It’s one thing to know your automations might break. It’s another to find out because your customer is reporting it.

The typical response is SLAs and escalations — but escalations mean you’re waiting for things to break. Then the back-and-forth inherently slows things down: isolating the problem, writing the fix, deploying the fix, validating the fix. Each human step takes hours. And each time the fix fails to resolve the issue increases frustration.

Customers should define their own acceptance tests as part of their contract. Run them post-deploy — no individual customer should be able to block deploys — but alert both the customer and Kernel when an automation might fail. Then, when engineering begins to work on a fix, they just have to make the test pass. Test-driven development, applied to customer workflows2.

Footnotes

  1. Unlike Stripe Sandboxes, which work because Stripe controls both sides of the API, Kernel can’t offer a “Kernel Sandbox.” Browser automations run on the real web — that’s the whole point. A browser with a proxy that blocked all POST, PUT, and DELETE requests would likely break websites, even if the automation script itself was meant to be idempotent.

  2. This would need to be limited to enterprise contracts, but it’s akin to “a dedicated engineer” or “uptime SLA” that many infra companies already provide — just codified in acceptance tests instead of legalese.