Why Artificial Superintelligence May Be Fundamentally Impossible

Since the debut of ChatGPT, the idea of “artificial superintelligence" has suddenly become a mainstream talking point. Just browse YouTube, and you’ll find countless podcast episodes with “AI experts” saying that superintelligence is inevitable, and when it comes, we’re all doomed.

But something about the concept of superintelligence has always bothered me. I don’t mean the highly disruptive nature of superintelligence and what it could do to society and economics. But more so, I question the very idea of whether a superintelligence can even exist in the form of a computerized system built by humans.

However, that’s different than humans building general intelligence that has an incredible amount of computational horsepower, meaning it can do a lifetime’s worth of thinking and research in the span of a few minutes.

To me, those are two different things. Superintelligence suggests the model will inherently know things it hasn’t seen or had the opportunity to test in the real world. Essentially, it creates accurate knowledge out of thin air by simulating all possible counterfactual possibilities and then 100% accurately choosing the right one.

Supercharged general intelligence, on the other hand, is similar to having millions of the smartest people within a given field working 24 hours a day, 7 days a week. It’s basically just automating traditional intelligence and leveraging its scale.

But that’s where my problem with superintelligence starts to emerge. Proponents who argue superintelligence is just a few years away argue that superintelligence will simply “know things” that we can’t currently understand.

I have no problem with that concept at the surface level. But if we dive deeper into what that concept actually means, we see that it’s essentially saying that superintelligence will know everything, all at once.

If superintelligence can create accurate knowledge completely internally without any real-world experimentation or feedback, that means it will essentially have infinite knowledge the instant it comes online.

Infinite knowledge requires infinite computational scaling, which we know is impossible.

An Example

Let’s take the example of advancements in material sciences.

In material science, new materials are based on theoretical knowledge, accidental real-world discoveries, simulation, and then finally, real-world testing and application to further refine the idea.

The real-world testing is essential because in virtually all types of engineering, the real world doesn’t behave exactly as the calculations or the simulations. Some aspect or property was unaccounted for and only became apparent when applied to the real world.

However, with superintelligence, people are suggesting that real-world experimentation will no longer be needed. The model will be so accurate with its ability to simulate the real world that it will simply spit out a working formula for a new material on the first try.

The problem is that this means the superintelligent model must have a 100% accurate simulation inside of it. It can simulate everything in the known universe to exacting precision. It must have that if we are to believe it can forgo real-world experimentation and trial and error.

But as I said earlier, that would require infinite scaling of computational power for it to work.

Here is why you couldn’t limit the superintelligence to non-infinite computation and still get perfect results.

If you told the superintelligence to only use a finite amount of computational power, the results would not be perfect. It’s entirely possible that the knowledge it needed to have 100% accuracy for a given task was somewhere in the theoretical computational power that you forced it not to use.

At that point, it just becomes supercharged general intelligence, which I explained earlier. It’s better than a single human due to its computational advantage, but it’s still not perfect, and any output will need to be tested before it's trusted.

This is why I argue that AI of any kind cannot solve humanity's problems as many proponents suggest.

Take global climate change as an example. Many AI proponents, such as Google’s former CEO, Eric Schmidt, argue that AI will solve global climate change, so we shouldn’t worry about building data centers that spew greenhouse gases.

But how exactly will AI solve a problem like that?

I assume Eric Schmidt thinks that AI will spit out a perfect answer one day to solve the problem. But that answer will just be a series of ideas and proposals. Humans will still have to approve and implement those ideas over a long period of time.

If the model is not 100% perfect, then there will be debate over the ideas and whether or not they will work, which is exactly what we have now when facing any public dilemma.

As I stated earlier, for the model to be 100% accurate and therefore trusted without question, it must have infinite knowledge, and infinite knowledge requires infinite computational power.

It’s that paradox that causes me to believe superintelligence is not possible.

There will be supercharged general intelligence, which I believe is not far away. But superintelligence is fundamentally impossible. In essence, superintelligence has a fundamental problem, not a practical problem.

In closing, I don’t believe the impossibility of superintelligence necessarily protects society from the very real possibility of AI disrupting virtually all aspects of life as we know it.

Supercharged general intelligence is likely enough to make all economic theory useless, while creating all kinds of new problems and challenges on the geopolitical front.

The problem with thinking of superintelligence as the only serious AI threat is that it allows us to push out the inevitable disruption that AI will cause to some imaginary time, when in reality, the disruption is well underway with the technology and systems we already have access to.

—

James La Forte is an IT professional & freelance writer focusing on technology, business, and finance. His work has been featured in U.S. News & World Report, TechRepublic, MarketWatch, Gartner Insights, and other publications.