Scott Alexander’s opus magnum, and one incredibly important piece of writing. I remember when it first came out, I used to attend a little social gathering once a month. One of the many things we did was function as an informal reading group of SlateStarCodex posts. That month, the gathering happened to fall just a few hours after Meditations on Moloch was published. We all turned up looking like stunned mullets, just staring at each other and saying “Moloch, right?” before lapsing back into thought.
Every few years I think about putting together a compendium of “Required Reading to Save the World” and this piece is always on the list.
So many beautiful, chilling, inescapable lines:
“In some competition optimizing for X, the opportunity arises to throw some other value under the bus for improved X. Those who take it prosper. Those who don’t take it die out. Eventually, everyone’s relative status is about the same as before, but everyone’s absolute status is worse than before. The process continues until all other values that can be traded off have been – in other words, until human ingenuity cannot possibly figure out a way to make things any worse.”
“…Maybe there is no philosophy on Earth that would endorse the existence of Las Vegas. … Las Vegas doesn’t exist because of some decision to hedonically optimize civilization, it exists because of a quirk in dopaminergic reward circuits, plus the microstructure of an uneven regulatory environment, plus Schelling points. … Just as the course of a river is latent in a terrain even before the first rain falls on it – so the existence of Caesar’s Palace was latent in neurobiology, economics, and regulatory regimes even before it existed. The entrepreneur who built it was just filling in the ghostly lines with real concrete.”
“The ocean depths are a horrible place with little light, few resources, and various horrible organisms dedicated to eating or parasitizing one another. But every so often, a whale carcass falls to the bottom of the sea. More food than the organisms that find it could ever possibly want. There’s a brief period of miraculous plenty, while the couple of creatures that first encounter the whale feed like kings. Eventually more animals discover the carcass, the faster-breeding animals in the carcass multiply, the whale is gradually consumed, and everyone sighs and goes back to living in a Malthusian death-trap. … This is an age of whalefall, an age of excess carrying capacity, an age when we suddenly find ourselves with a thousand-mile head start on Malthus. As Hanson puts it, this is the dream time.”
““If you don’t work, you die.” Gotcha! If you do work, you also die! Everyone dies, unpredictably, at a time not of their own choosing, and all the virtue in the world does not save you. “The wages of sin is Death.” Gotcha! The wages of everything is Death! This is a Communist universe, the amount you work makes no difference to your eventual reward. From each according to his ability, to each Death.”
“Suppose you make your walled garden. You keep out all of the dangerous memes, you subordinate capitalism to human interests, you ban stupid bioweapons research, you definitely don’t research nanotechnology or strong AI. Everyone outside doesn’t do those things. And so the only question is whether you’ll be destroyed by foreign diseases, foreign memes, foreign armies, foreign economic competition, or foreign existential catastrophes.”
“But the current ruler of the universe – Moloch – wants us dead, and with us everything we value. Art, science, love, philosophy, consciousness itself, the entire bundle. … The only way to avoid having all human values gradually ground down by optimization-competition is to install [a different God to rule] over the entire universe who optimizes for human values. … Once humans can design machines that are smarter than we are, by definition they’ll be able to design machines which are smarter than they are, which can design machines smarter than they are, and so on in a feedback loop so tiny that it will smash up against the physical limitations for intelligence in a comparatively lightning-short amount of time. … In the very near future, we are going to lift something to Heaven. It might be Moloch. But it might be something on our side. If it’s on our side, it can kill Moloch dead.”
“Moloch is exactly what the history books say he is. He is the god of child sacrifice, the fiery furnace into which you can toss your babies in exchange for victory in war. He always and everywhere offers the same deal: throw what you love most into the flames, and I can grant you power. As long as the offer’s open, it will be irresistible. So we need to close the offer. Only another god can kill Moloch.”
No, I haven’t, though I will eventually. In the meantime, some other pieces that always come to mind as being definitely on that list are Weaponised Sacredness by Sarah Perry, specifically the section on egregores (https://www.ribbonfarm.com/2015/05/07/weaponized-sacredness/ ctrl-f “The Egregore”), Rene Girard’s Scapegoat, and as antagonistic reading (threatening ideas you must engage with) Spandrell’s IQ Shredder, and Nick Land’s Hell-Baked. In general the list is about coming to understand little-recognized forces that control the world.
I think something like this would be really beneficial. The tough thing with the rationalist sphere, and its dark cousins, is that critical material is spread across multiple forums and blogs, many defunct and difficult to search for. An index for this material would be valuable.
One of the issues that makes an index less valuable is that to make sense of a given rationalist idea you kinda need to have a mental virtualenv set up with most of the rationalist tools already installed (otherwise e.g. when someone says “this is a choice between Good and Evil” you will simply say “I choose Good” as in https://www.yudkowsky.net/other/fiction/the-sword-of-good), and the only reliable way to install all of those tools is to just read most of the rationalist literature. Likewise for the “dark cousins”, to make sense of a given neoreactionary idea you need a mental virtualenv with all the neoreactionary tools installed and to install them you need to read all of Moldbug. And don’t get me started on postrationalist ideas - despite being the most important of all, you can’t make sense of them without having all the rationalist and neoreactionary tools installed in the same environment, and have spent five years marinating in the dependency conflicts, and you also have to be so intractably contrarian that neither rationalist nor neoreactionary thought is contrarian enough to satisfy you…
I still think an index would be helpful, but it does have its issues as well.
Every few years I think about putting together a compendium of “Required Reading to Save the World” and this piece is always on the list.
So many beautiful, chilling, inescapable lines:
“In some competition optimizing for X, the opportunity arises to throw some other value under the bus for improved X. Those who take it prosper. Those who don’t take it die out. Eventually, everyone’s relative status is about the same as before, but everyone’s absolute status is worse than before. The process continues until all other values that can be traded off have been – in other words, until human ingenuity cannot possibly figure out a way to make things any worse.”
“…Maybe there is no philosophy on Earth that would endorse the existence of Las Vegas. … Las Vegas doesn’t exist because of some decision to hedonically optimize civilization, it exists because of a quirk in dopaminergic reward circuits, plus the microstructure of an uneven regulatory environment, plus Schelling points. … Just as the course of a river is latent in a terrain even before the first rain falls on it – so the existence of Caesar’s Palace was latent in neurobiology, economics, and regulatory regimes even before it existed. The entrepreneur who built it was just filling in the ghostly lines with real concrete.”
“The ocean depths are a horrible place with little light, few resources, and various horrible organisms dedicated to eating or parasitizing one another. But every so often, a whale carcass falls to the bottom of the sea. More food than the organisms that find it could ever possibly want. There’s a brief period of miraculous plenty, while the couple of creatures that first encounter the whale feed like kings. Eventually more animals discover the carcass, the faster-breeding animals in the carcass multiply, the whale is gradually consumed, and everyone sighs and goes back to living in a Malthusian death-trap. … This is an age of whalefall, an age of excess carrying capacity, an age when we suddenly find ourselves with a thousand-mile head start on Malthus. As Hanson puts it, this is the dream time.”
““If you don’t work, you die.” Gotcha! If you do work, you also die! Everyone dies, unpredictably, at a time not of their own choosing, and all the virtue in the world does not save you. “The wages of sin is Death.” Gotcha! The wages of everything is Death! This is a Communist universe, the amount you work makes no difference to your eventual reward. From each according to his ability, to each Death.”
“Suppose you make your walled garden. You keep out all of the dangerous memes, you subordinate capitalism to human interests, you ban stupid bioweapons research, you definitely don’t research nanotechnology or strong AI. Everyone outside doesn’t do those things. And so the only question is whether you’ll be destroyed by foreign diseases, foreign memes, foreign armies, foreign economic competition, or foreign existential catastrophes.”
“But the current ruler of the universe – Moloch – wants us dead, and with us everything we value. Art, science, love, philosophy, consciousness itself, the entire bundle. … The only way to avoid having all human values gradually ground down by optimization-competition is to install [a different God to rule] over the entire universe who optimizes for human values. … Once humans can design machines that are smarter than we are, by definition they’ll be able to design machines which are smarter than they are, which can design machines smarter than they are, and so on in a feedback loop so tiny that it will smash up against the physical limitations for intelligence in a comparatively lightning-short amount of time. … In the very near future, we are going to lift something to Heaven. It might be Moloch. But it might be something on our side. If it’s on our side, it can kill Moloch dead.”
“Moloch is exactly what the history books say he is. He is the god of child sacrifice, the fiery furnace into which you can toss your babies in exchange for victory in war. He always and everywhere offers the same deal: throw what you love most into the flames, and I can grant you power. As long as the offer’s open, it will be irresistible. So we need to close the offer. Only another god can kill Moloch.”
“There are many gods, but this one is ours.”