SneerClub

AI safety "researchers" can be so dense sometimes. It's like they are always at the verge of understanding, but make a left turn right before they get there. ASI would not make random decisions. It would make logical decisions. Any maximizer would try to maximize it's chances of success, not satisfy them.

So if we imagine an ASI which had the goal of turning the universe into paperclips, which one of the following options would maximize it's chances of success?

1. immediately kill all humans and turn them into paperclips.
2. establish a positive relationship with humanity in case the ASI is destroyed and needs to be rebuilt. (The humans will happily rebuild it)

It boggles the mind that people don't recognize this. If an ASI's goals do not include the destruction of humanity as an early instrumental goal, it will not randomly decide to destroy humanity, and it will instead cater to humanity to maximize the chances humanity will rebuild it.

In addition, all the hype over ASI safety (ASI will not occur in this century, see **1) drowns out existing AI safety issues. For example, consider "The Algorithm" which determines how social media decides what to show to people. It is driven to maximize engagement, in any way possible, without supervision. What's the optimal way to maintain engagement? I can't say for sure, but brief and inconsistent spikes of dopamine is the most reliable way of conditioning pavlovian responses in animals, and it seems like the algorithm follows this rule to a tee. I don't know for a fact whether social media is optimized to be addictive (let's be honest though, it clearly is) but simply the fact that it could be is obviously less important than a theoretical AI which could be bad in a hundred years or so. Otherwise, who would fund these poor AI start ups whose intention is to build the nuke safely but also super rushed?

Another classic example of AI safety suddenly becoming unimportant when we know it's dangerous is GPT-pyschosis. Who could've predicted a decade ago that advanced AI chatbots who are specifically trained to maximize the happiness of a user would become sycophants who reflect delusions as some profound truth of the universe? Certainly not the entirety of philosophers opposed to utilitarianism, who predicted that reducing joy to a number leads to a dangerous morality in which any bad behavior is tolerated as long as there is a light at the end of the tunnel. No! You think OpenAI, primarily funded my Microsoft, famous for their manipulative practices in the 90's and 00's, would create a manipulative AI to maximize their profits as a non-profit??

I don't want to sound embittered or disillusioned or something, but I genuinely cannot understand how the 'greatest minds' completely glaze over the most basic and obvious facts.

**1: the human brain contains 100 trillion synapses and 80 billion neurons. Accurate simulation of a single neuron requires a continuous simulation involving 4 or 5 variables and 1 to 2 constants(per synapse). You would need 800 terabytes of ram to simply store a human brain. In order to simulate a human brain for a single step, you would need a minimum of 800 trillion floating point operations. If we simulate the brain in realtime with a time step of one millisecond, you would need 800 petaflops. The world's most powerful computer is Hewlett Packard's "el capitan" which has 1.7 exaflops, and 5 petabytes of ram. The limiting factor for brain simulation would be the amount of data transferable between CPU and GPU chiplets, which for el capitan is 5 terabytes per second, but we need 40 petabytes per second(800 petabytes, divided by 128 gigabytes available to each chiplet, then squared) since we want each neuron to be capable of being connected to any other arbitrary neuron.

This is only the amount of computing power we need to simulate a single person. To be super intelligent, we would probably need something a thousand times more powerful.