Programmer Humor

The crash early, crash often approach of Erlang has made for some amazingly resilient systems.

One time on a project I was working on, some horribly broken code was merged (nobody in the team had even heard of reviewing code). As soon as a specific call was made, it was executed once and then the thread crashed. The only way we noticed was that response times increased with load. All data and behavior was still correct. Whole nodes could go down and all you notice is a dip in performance until it comes back online.

Of course it requires special care in designing. Everything runs in stateless server threads with supervisors restarting them as needed. This in turn requires some language support, like lightweight threads. Our application would happily run tens of thousands of threads on an ancient sparkstation.