Cutting Out the Middleman

TypeScript for infrastructure (as code) isn't just a technical decision. It's a customer-driven movement. And the customer isn't your traditional IT manager, it's a JavaScript developer.

Many developers laud Heroku as one of the best developer experiences of all time (and are frequently trying to recreate it). Heroku had many problems (and still does as part of Salesforce), but it proved the hypothesis that developers could deploy their own applications. How much of the DX was simply that you could provision a load balancer and certificate without getting anyone else involved?

I started working on Kubernetes back in 2016 on because of my own frustration with the rift between engineering and IT (The GitLab Upgrade). Kubernetes might have missed the mark (so far) for being a true developer platform (my belief in Kubernetes Maximalism), but again, it shows a shift of responsibility from IT to developers. You can see this with the trend of platform teams, which I'm a big fan of. Even if Kubernetes is mostly locked down to developer teams, custom resources can expose much more functionality than you'd get with a more traditional PaaS.

While the role of software engineering is unbundling, some responsibilities will be shifted. You can see this at a high level with no-code or low-code (why have engineers if you can write it yourself?), but also at a lower level (why have IT provision resources if developers could do it themselves?). Not all attempts at encapsulation and unbundling will work.

It will be interesting to see this play out in the modern data stack that's evolving. Data and analytics teams are adopting more backend/frontend software engineering best practices – like version control and CI/CD. Many are mostly proficient in SQL but these new skills are helping them demand higher salaries in software engineering positions. Will the new stack require new roles (e.g., data engineers) or will someone figure out how to cut out the middleman somehow? And, who is the middleman? The engineering team or data team!