The End-to-End Principle in System Design

The end-to-end principle is a design pattern used in the early Internet that suggests that

Specific application-level functions usually cannot, and preferably should not, be built into the lower levels of the system – the core of the network.

It was formalized in a 1984 paper, End-to-End Arguments in System Design, by Saltzer, Reed, and Clark. The paper uses an example of securing a file transfer between two computers. There are many steps during the transfer where the file could get corrupted or lost. Should the network be responsible for error checking, de-duplication, ordering, and crash recovery? The end-to-end solution solves this problem at the end node – a simple checksum at the source and destination.  

David Clark wrote a follow-up paper in 2000 examining how the Internet had changed. Namely, he recognized that the Internet was full of users that might not have others' best interests at heart   – spammers, the government interests, users who don't trust each other, users who don't trust the software they're using, etc.

Clark touches on different ways of approaching this problem: firewalls, NAT, trusted-third parties, public-key cryptography, and non-technical solutions.

He ends with a paragraph that's still relevant 22 years later,

We have painted two pictures of the constraints that technology imposes on the future Internet. One is that technological solutions are fixed and rigid. They implement some given function, and do so uniformly independent of local needs and requirements. They create a black and-white outcome in the choice of alternatives. Either an anonymizing service exists, or it does not. On the other hand, we observe in practice that there is a continuing tussle between those who would impose controls and those who would evade them. There is a tussle between spammers and those who would control them, between merchants who need to know who the buyers are and buyers who use untraceable e-mail addresses, and between those who want to limit access to certain content and those who try to reach it. This pattern suggests that the balance of power among the players is not a winner-take-all outcome, but an evolving balance. It suggests that the outcome is not fixed by specific technical alternatives, but the interplay of the many features and attributes of this very complex system. And it suggests that it is premature to predict the final form. What we can do now is push in ways that tend toward certain outcomes. We argue that the open, general nature of the Net, which derived from the end to end arguments, is a valuable characteristic that encourages innovation, and this flexibility should be preserved.