on the value of reading science fiction (and classics)

Many disagreements between people are caused by differences in how changeable they believe a system is. How easy is this system to change? To what extent can it change, and in what ways?

Here are some examples of disagreements due to changeability:

The Limits of Knowledge, and the Importance of Intuition

Sometimes a disagreement is due to differences in knowledge. In our political example, perhaps person 1 believes person 3 could vote ‘yes’ on a bill, whereas person 2 has spoken to person 3 and knows that they will vote ‘no’. Person 1 thinks the system is changeable, whereas person 2 knows that it is not.

But in most debates, disagreements about changeability occur with both parties having incomplete information. We don’t know how the market will react to a new entrant with better widgets. We don’t know whether scaling laws will continue. We may have arguments to justify our views, but must ultimately fall back onto what seems true, likely, or possible to us. In other words, we must rely on our intuitions.

This limitation of knowledge and reliance on intuition is either because we’re in a complex domain or at the edge of our collective understanding; we don’t know what the future will look like.

This sense, this intuition, about how a system could change in the future is shaped by what we’ve been exposed to. It’s shaped by the arguments, authorities, events, places, and things we’ve interacted with. For example, a person who grew up in a geopolitically unstable region may have a stronger intuition about the possibility of war than the typical person born in a more stable region. But a war historian in that stable region may also have built a strong intuition about the possibility of war. Similarly, a person who works on LLMs may have a stronger intuition for the possibility of AGI than the average person.

This is a common pattern in human thought. While we have reasons for what we believe, it is often our intuition, based on what we’ve been exposed to, which shapes what seems true, possible, or likely to us.

I’ve noticed this in myself as well. When I was studying deep learning in college, AGI felt much more likely and imminent to me than it does today working in crypto, despite none of my beliefs actually changing. I just spend less time exposed to it, making my feeling less visceral.

To repeat what we’ve said so far:

• Many disagreements are due to differing beliefs about a system’s changeability

• We often have incomplete knowledge about how changeable a system actually is, forcing us to fall back on our intuitions

• Our intuitions are largely shaped by our experiences and what we’ve been exposed to

The Value of Science Fiction

“The largest and furthest-reaching political changes of all time have invariably been the effect of technological progress.”

Scott Alexander, We Wrestle Not With Flesh and Blood, But Against Powers and Principalities

At a macro-level, most of the largest changes to the world are due to changes in technology and its downstream effects (I’ll merely assert this for brevity, but the curious reader can check out Scott’s argument linked above for a justification). Therefore, many of the most important debates are about the changeability of technology and the systems around it.

Despite its importance, I’ve found that most people severely underestimate how much technology can change. This is primarily due to intuition: things like AGI, interstellar travel, and nanotechnology just seem unrealistic. These technologies don’t exist, and they’ve never spent much time thinking about them, making them hard to imagine. Suffering from incredulity, they dismiss propositions about the world’s changeability. This is one category of what Arthur C. Clarke called “failure of the imagination”, which he considered one of the most common reasons for mispredictions about the future.

This finally gets us to the main point: the primary value of reading science fiction is in building intuition about how changeable the world actually is.

Science fiction is not about building knowledge. It is not for saying what technology is possible, when it will exist, or what its effects will actually be. Science fiction is about building intuition for how much technology, and therefore the world, can change.

Our intuitions are shaped by our past experiences. Yet the future will not look like the past. Therefore, it’s difficult to build intuition about the future, especially the long-term future, without exposure to science fiction.

Further Calibration

But there’s also an equal and opposite danger of reading science fiction. Building intuition about radical futures, the science fiction reader may begin to overestimate what is possible, how quickly it will be possible, or how positive (or negative) a technology will be. The town crier who believes the singularity is tomorrow after overdosing on Ray Kurzweil may have a different kind of “failure of the imagination”.

If one’s goal is to not just have intuition about future technology, but to be well-calibrated about the future — that is, to believe what they should believe to the extent that they should believe it — then it helps to understand the counterbalances to radical change.

In other words, as one builds intuition for the world’s changeability, they should also develop an understanding of what actually has low changeability, what systems are hard to change and why they’re hard to change.

Perhaps the best counterbalancing force to science fiction is the great works of literature, which among other things cultivate our understanding for what human nature is like, and how stubbornly difficult it can be to change.

This is one reason why many of the best intellectuals today are well read in both science fiction and classics. They have not only a long-view of history, but are also well-calibrated about how the world can change over long periods of time.

The 2 are often directly complementary. For example, I recently read Permutation City and The Epic of Gilgamesh which, despite being written 4,000 years apart, both deal with the same topic: the human desire to live forever.

Yet they have very different things to say on that topic. The Epic of Gilgamesh tells of the ultimate failure of the heroic Gilgamesh, who after acquiring the flower that grants immortality, has it stolen by a snake and must return home empty-handed, where he ultimately dies. The theft by the snake can be interpreted as a symbol that the only thing that never dies is human sin, and that the human desire to be immortalized (which is also addressed in other parts of the story), is hopeless.

Permutation City has a different message. The main character in the book does solve immortality (sort of) through brain emulation, but this immortality leaves open problems, like the question of personal identity and the desire to know the true metaphysic.

What then can we learn from these pair of stories? One reveals how the desire for immortality has existed since the dawn of man — The Epic of Gilgamesh is literally the oldest story in world literature that we have. The other is about one way in which death may actually be solvable thousands of years later, but would not be the end to our problems.

History, particularly the history of technology, sits in a middle ground between science fiction and classics. If science fiction reveals how the world could radically change and classics show how it stays the same, history reveals ways in which the world has already changed.

History thereby helps us fill in the barbell between classics and science fiction, providing further calibration on the world’s changeability.