Towards Granular Compute

Runtimes, infrastructure, and APIs tend to get more granular. Maybe we're just in the unbundling phase (implying a future bundling phase), or maybe it's a byproduct of moving functionality to the edge (the network is the bottleneck),  or perhaps this is just a general form of progress (breaking things down into abstractions).

At a basic level, granularity lets us bin-pack workloads. Different workloads have varying levels of safety when it comes to resource sharing and allocation. Isolation happens at the operating system level (virtual machines), process level (containers), and thread level (v8 isolates). Hardware security module (HSM) workloads and edge functions need different security guarantees.

But it's also changed the way that we write code. Virtual machine managers might be smart enough to overcommit memory, but it's difficult to have the granular knowledge to know when it's safe to preempt. Doing it at the process level (containers) can make it easier to handle interrupts and restarts. Once compute gets granular enough, it might not even matter.

Fault-tolerant infrastructure creates less error handling code at the application level. Noisy neighbor applications that misbehave and suck up CPU and RAM might crash other processes. If you deploy on something like AWS Lambda, the process will still crash, but you can limit the blast radius.

Smaller APIs are often easier to integrate into existing infrastructure. For example, carved-out authentication or authorization APIs are much easier to add to your application than Active Directory (in theory).

Granularity won't solve everything (there's no silver bullet). But it gives us more building blocks.