Watching the symphony last night: the conductor provides common ground for the joint activity of playing the music. She is a single point of coordination, that canonical clock that everyone can see at the same time. There is no information delay from the conductor to any member of the orchestra. They are playing from the same script, they have situational awareness of each other, plus they have a common source of timing.
Maybe a jazz band, playing improvisationally, is more like a distributed system without a central coordination point. All listening to each other, all sharing enough knowledge of the music and each other to respond and build the song. They have common snippets of scripts, and lots of awareness of each other.
For distributed systems in computing, imagine a jazz band with information delay. They’re trying to improvise but they can only hear what the other musicians played a few measures ago. Worse: usually it’s one measure ago but sometimes it’s several measures ago and no one knows which. They have to decide what to play in realtime based on out-of-date information. Everything they play will affect their peers, but they don’t know when.
Yeesh.
Add a central point of coordination, and a strict script, and we’re back to being in an orchestra – except with information delay, they can’t play in sync. If it takes the conductor’s movements a different amount of time to reach each musician, they aren’t playing together anymore.
We want our software systems to sing. But we can’t expect them to play a perfect song, not with central coordination, not with peer-to-peer. I don’t know what the solution is, but more and more I appreciate that this is hard.