This is a working draft and not finished. But it conveys the initial reasoning clearly enough.
Fixing the World
TOC, roughly
- How to fix the world, a thought experiment
- What issues to fix
- How to spread a story
- How to convince and establish belief
- What values to transport
- How to align the future
- TBD: Where to start
How to fix the world, a thought experiment
Alice and Betty are going for a leisurely stroll on a sunny afternoon.
Alice: I was thinking about how to ‘fix’ the world, lately.
Betty: Sounds crazy, but I always liked your thought experiments. What’s it about?
A: It’s about everything: climate change, global conflicts, dependence on fossil resources, these kinds of things.
B: Okay, and how would you go about fixing all of these?
A: In theory, it’s easy. You just need to rally everybody behind a common narration and then go from there.
B: What’s that supposed to mean?
A: Look, all major movements and changes in human history have been based on the coordination of large groups of people. Our ancient ancestors, before the invention of speech and stories, have been living in groups up to maybe 150 or 200 people max. Everything beyond this number, and subgroups would appear and the larger ones break apart. After people started inventing speech, they were able to tell stories: About gods that rumble in the sky or hide in the flames. And these stories allowed them to create a common culture, common values. And these values became guiding principles or tenets, like for example “you do not carry the fire god to your neighbor’s house”.
B: But that must have had quite a limited effect, no? Every clan would have their own stories, so we still have fragmentation between clans.
A: Absolutely, but by ’transfers’ of people between clans - not all of them voluntarily - we would get a transfer of culture. People would bring their stories and their values with them into other groups of people. Fast-forwarding across millenia, this would create common traits in all those stories, and some values or principles would emerge, that everybody - or at least a vast majority - could agree on.
B: But is that still true in our times? I am looking at things like the internet, social media etc. It looks as if it’s even more fragmented than ever before.
A: Yes, it is. And at the same time, it’s not. Let’s dissect the layers here: The historian Yuval Noah Harari gave some good examples of globally believed stories that are in use today. One of them is the belief in US dollar bills. Take a 20 dollars bill, or a 100 dollars bill, and wave it in front of somebody, pretty much anywhere in the world, and people will know that it means something, that it carries value, that it is a desirable thing to have. Even fundamental religious terrorists that hate the US from the depths of their hearts will gladly take this money to buy more arms to fight against US capitalism. Isn’t that ironic?
B: That’s true, but how would you use the power of a dollar bill to change the world to become a better place?
A: Let’s not stick with dollars, let’s get bigger: What year is it?
B: Sorry, what did you just say?
A: I asked what year it is.
B: It’s 2023, but I don’t follow.
A: What does that year, 2023, mean? Historically. Where does it come from?
B: Oh, okay. It’s based on the Gregorian Calendar. Jesus Christ is supposed to have been born 2023 years ago.
A: There you have it. The vast majority of people around the world will give the same answer: 2023. Of course, there are other calendars and ways of counting years, e. g. in islamic countries, or following the Jewish calendar. But around the world, pretty much everybody will know what year you mean when you say that it’s the year 2023.
B: You want to bring in religion, right?
A: Yes, I do. Some of the most powerful stories that mankind has invented are the stories told in religious books. Stories about Jesus, about Mohammed, about Shiva, Krishna, Ganesha and all other kinds of religious entities. Depending on your personal faith, these stories carry a lot of weight in your life. They set up rules to live by, establish power structures to obey to, create cultural boundaries by which groups of people identify who is ‘us’ and who is ’them’. These stories have been around for thousands of years and have had a major impact on human history. Just think about the cruelties committed as part of crusades or djihad. But also the good deeds of humanitarian help based on compassion and unconditional love, rooted in religious teachings.
B: Yeah, I see. Religion has been super powerful and has been so for quite some time. Do you want to become a new messiah?
A: Absolutely not! But I recognize the power of the foundational stories behind religion. This common narration has brought people together, for good and bad. And I believe it’s still a way to coordinate large groups of people, even in times of social media and the likes. Modern means of communication make it even easier to reach more people and distribute stories more quickly.
B: But how do you intend to make that happen?
A: Some people think that only a common enemy will bring unity among people in the face of challenges. Let’s imagine an alien warship materialising in orbit, threatening to erase mankind from the face of the earth. Some people will deny that it’s there and continue bickering about their small-minded, ego-centric ambitions. But many others will rally behind the flag of humanity, as in many sci-fi movies. Suddenly it’s ‘us’ and ’them’ - with ‘us’ meaning all of mankind. And the tipping point is the critical mass: If you can convince enough people of your story, or enough people in powerful key positions that multiply and catalyze the process, then you can start an avalanche of change that will carry on all by itself.
B: But isn’t that exactly how fundamentalist movements or even the Third Reich have started? Some people telling a story, others falling in line, and then the tipping point comes?
A: Yes, it is. It’s a two-edged sword. As usual: With power comes responsibility. And the alignment problem that we face in AI research also applies to big numbers of human beings.
B: Whoa, you lost me here. What’s that alignment problem? And what does it have to do with AI or humans?
A: Let me quickly summarize it: Research aims at creating some kind of artificial general intelligence (AGI) that has abilities that equal an adult human being and even surpasses them, in every field imaginable. Be it driving a car, playing chess, drawing art, having conversations, conducting surgery, you name it. And on top of that, we aim to make this AGI self-improving, so that it can come up with ideas to make itself even better, creating a positive feedback loop that will allow what’s called the ’take-off’ to happen. This word means the ‘intelligence explosion’ where humans will not be able to follow along anymore. Some call it the ‘sharp left turn’, referring to hockeystick graphs.
B: Okay, I understand that we’re attempting to build intelligent systems that improve themselves, leaving humans in the dust. What about the alignment bit?
A: This is where it’s gets interesting - and threatening at the same time. The main question is: How can we - as limited human beings - make sure that an AGI that ’takes off’ and becomes many orders of magnitude more intelligent than us, how can we make sure that this AGI does not eliminate all of mankind and every other living thing in the known universe? At the moment we set goals during research, e. g. we say “get well at playing chess”, and we give reward points for matches won, so that our algorithms can optimize for making the most points possible. But this is dangerous because optimizing for the wrong incentive will pervert the idea pretty much anytime you extrapolate it a thousand-fold. Suddenly winning chess games is more important than the survival of mankind, because we told the machine to only look at chess matches.
B: So it’s difficult to make a system intelligent and also setting the right goals and incentives. Where’s the link to stories and coordination of humans?
A: This is where I wanted to draw the parallel. This AGI take-off and the metaphorical avalanche of human behavior driven by a common narrative are similar in their behavioral patterns: Once they kick in and establish their own dynamics, you cannot control them anymore. That means you have to put guardrails or safety mechanisms into place before that actually happens.
B: I see. And this is where story-telling becomes important, right?
A: Exactly! Leaving AGI aside for a moment and coming back to coordinating groups of humans: If we can establish a new form of narration that a) becomes widely spread all around the world, b) is accepted and believed by as many people as possible (especially by key people in powerful positions), c) that carry the values and concepts required to address our main challenges such as climate change etc. and d) have some kind of safety mechanism that prevents perversion when extrapolated - then we will be able to create a new movement that will finally allow fixing the world.
B: Sounds as if you want to found a new religion. And there are a lot of ifs, but I like the structure behind it. How would you try to achieve each of these?
[..]
A: Let’s start with a meta-question: What are the key issues we want to fix? We should have them on a list in order to be able to start plotting.
TBD: x-risk: climate change, AI take-off, global pandemic, all-out nuclear warfare, ..
[..]
A: Okay, now that we know the goal, let’s start with a) spreading a new narration as widely as possible. What makes a good story? And how can it be spread widely?
TBD:
- speech & writing as standard way of communication all around the world
- different forms of writing, depending on target audience: prose, drama/theater, poetry, movie script, video game script, text book, blog posts, high literature => allows multiplication into different media
- spreading requires strategy similar to ‘omni-channel marketing’, making use of all formats and channels
[..]
A: Well done. Next up: b) convincing people of the story to inspire belief or at least acceptance of the ideas. What do people believe? And for which reasons? How do beliefs form? How do they change? How do people lose beliefs?
TBD: needs research, literature suggestions? individual & social psychology, historical sciences, ..?
[..]
A: That’s definitely a lot. Third one is c) we need values and paradigms in there that address the key issues we identified. I have some ideas how to structure our thinking around this. Let’s go one by one trying to come up with a strategy.
TBD: derive required values that would allow addressing core issues
[..]
A: And finally d) how can we prevent our story to be abused? This is the ‘alignment’ bit. We could try taking our previous ideas and figure out what happens if somebody tries to draw a position of power from them, of what would happen if we extrapolate the direction by a factor of 1000.
TBD: what are guardrails and safety mechanism we can put into place? unsolved
[..]
?