Every once in a while, I read something that makes my stomach drop, my blood run cold, and gives me a strong desire to scream. Nick Bostrom struck all of these notes in me, with The Unilateralist’s Curse. In this paper, Nick provides a proof that with as the number of agents who are each capable of taking an action that affects all of the others rises, the likelihood of someone taking the action approaches certainty, regardless of whether the results of that action are objectively or collectively desirable.
Furthermore, the most likely actor for a given situation is the one who is most optimistic and least well informed. Scared yet? You should be. While Nick points in a few solutionlike directions, he acknowledges that at this point the solutions laid out are imperfect, situationally limited, and require buy-in from all parties; tricky to get among the sort of people who are likely to act unilaterally, especially if they don’t know about the curse.
How insidious is this? Well, as I was reading it and getting skin crawls from yet another reason for things to go horribly wrong, I thought to myself, “Well, I’d better get out ahead of the curve on FAI and – OH GOD DAMMIT I’M DOING IT!”
If you care about things, if you want the world to get better, and you wish everyone would just listen to you for like five minutes, so that we can get things done, you’re vulnerable too. The apathetic and uninvolved need not concern themselves, but if you feel heroic responsibility and the need to *act*, this is quite likely to come up in your life at least once.
I’m not telling you not to act on your own – there are times when the consensus is wrong, and you have to do something if you want to stop something that should not be. That having been said, if you find yourself thinking, “Why is no one doing the obvious thing? I had better, because clearly they’re incompetent!” I urge you to think again, to look more deeply into the facts, and then to consider again. Why are no other actors acting? Do they know something you don’t?
If you think you, or someone you know, might be subject to the Unilaterist’s Curse, urge caution, urge a deep inspection of the facts, and if they do support you… Then act. We can’t stop the world burning if we’re all too afraid to pick up a fire extinguisher, but for the sake of that which is valuable, be certifiably correct. Not just sure, or certain, or however you have the feeling of being right tagged in your mind. Feeling right is not being right, and as the most likely actor is also the most misinformed one… If you find yourself at that point, you have to at least consider that it’s you.
Best of skill out there. Try not to burn the commons in order to save the commons.