Things that are out of our control: complex systems and unintended consequences

May 6, 2010 by · Leave a Comment
Filed under: Uncategorized 

A lot of the talk about privacy nowadays revolves around the ways that the bodies with whom we entrust so much information about our lives might misuse that information.

However, there is another aspect that is to do with the increasing complexity of the tools we use to manage our day to day lives. Sometimes, these tools get so complex that no one really understands exactly how the entire system works. They are so complex that changes to one part of the system are liable to have completely unintended consequences for other parts of the system. And the point is that the system is so big and complex that these consequences cannot all be checked in advance.

Facebook has become such a system. We know this because sometimes it does things that the people in charge of it are completely unaware of. For instance, for a while yesterday there was a security flaw in Facebook that could

expose personal information by enabling your Facebook friends to see both your live chats, as well as your pending friend requests.

Facebook was quick to resolve the flaw, but the point is that it was there, and it had been discovered by people from outside Facebook.

What does this mean? Well, it reminds us of another way in which personal information about ourselves that we entrust to a commercial entity might be accessible by strangers. This is not because of malintent on the part of Facebook, and I’m not even sure I’d say it was because of their negligence, but rather it is a consequence of the complexity of the systems in which we spend so much of our lives.