Templating: When to Move Along the Heptagon

Templating sits between a configuration file and domain-specific language on the Heptagon of Configuration.

When should you use templating and when should you move along the Heptagon of Configuration?

Templating is the first step in turning configuration as data into configuration as code. Using string manipulation, templating engines can remove duplication and introduce basic control flow into our otherwise static data – all without any understanding of the underlying data. That is, you can use the same templating engine to produce HTML, YAML, SQL, or any other text data. That's what makes it so powerful.

The "first generation" of DevOps tooling (Ansible, Salt)  leaned heavily on templates. Puppet, another early DevOps tool, used a special templating DSL, which I'll put halfway on the Heptagon of Configuration.

Templates even found their way into the Kubernetes world – they are heavily used in Helm charts to provide extensibility in deploying variations of the same application on a cluster.

When should you ditch templating?

It was time to move along the Heptagon of Configuration where there became significantly more templating logic than actual data. This made Helm charts difficult to grok, and impossible to test for correctness. The lack of understanding of the underlying data types becomes a liability as templating increases.

There are tools like kustomize and ytt that perform transformations on certain types of data to achieve similar results to a template. They aren't universal – kustomize and ytt won't work on non-YAML or arbitrary data. This might include different kinds of layering or patching.

I listed DSL and Bash as the next two steps, although I'd like to replace Bash with a more generic Code as the ecosystem of configuration as code has evolved.

An early DSL in the Kubernetes community was called ksonnet, which was based on jsonnet, which ultimately looked a lot like borgcfg which Google uses to configure it's internal cluster management tool.   DSLs can be highly expressive, but require specific knowledge that can soon become outdated as the requirements or underlying products change.

Languages like CUE bill themselves as a data validation language and "inference engine". It is more expressive than a templating language because it requires more assumptions about the data is it modifying, but less expressive than a general-purpose programming language.

Eventually, even data configuration languages become too burdensome for complex use cases. If the same service behaves radically differently with two different and verbose configuration schemes, it might be a good opportunity to split those use cases out since the internal code paths might be just as divergent.