What happens if AI just keeps getting smarter?

It isn't actually smart, or thinking. It's just statistics.
- right? AI didn't pass the Turing test, Humans fucking failed it.
  
  Not that the turing test is meaningful or scientific at all...
- Soon to be built upon sarcastics.
- I'm not sure why that's a relevant distinction to make here. A statistical model is just as capable of designing (for instance) an atom bomb as a human mind is. If anything, I would much rather the machines destined to supplant me actually could think and have internal worlds of their own, that is far less depressing.
  
  It's relevant in the sense of its capability of actually becoming smarter. The way these models are set up at the moment puts a mathematical upper limit to what they can achieve. We don't quite know exactly where, but we know that each step taken will take significantly more effort and data than the last.
  Without some kind of breakthrough w.r.t. how we model these things (so something other than LLMs), we're not going to see AI intelligence skyrocket.
  
  A statistical model is just as capable of designing (for instance) an atom bomb as a human mind is.
  No. A statistical model is designed by a human mind. It doesn't design anything on its own.
  
  If it got smarter it could tell you step by step how an AI would take control over the world, but wouldn't have the conscience to actually do it.
  Humans are the dangerous part of the equation in this case.
- a meat brain is also a stastistical inference engine.
  
  Nah, biology has a ton of systems that all interconnect. Pain feedback itself is a tiny fraction of what makes a real brain tick, and "AI" doesn't have afraction of an equivalent. Of one solitary system.
  No, brains are far, far more than statistical inference. Not that they cannot be reproduced, but they are far, far more than math machines.

We need to have AI before we start worrying about what happens if it gets smarter than us.
We do not have AI.
- We have had AI for about 75 years. What we don't have is AGI.
  
  We've had goalposts for 75 years. What we don't have is a way to stop grifters from redefining them.
  
  We have had AI for about 75 years. What we don’t have is AGI.
  I can't believe people are downvoting this statement. You can get textbooks and journals titled "Artificial Intelligence", accredited universities teach the subject, and researchers meet at conferences to discuss the latest research, but apparently that isn't real because... other people use the term differently?
  I dislike OpenAI and LLMs as much as anyone else, but we can still be clear about our terminology.
  
  We can't even agree on a definition for "intelligence" so it's pretty obvious we haven't got an artificial version of it yet.
  Can't make what you can't even define, after all. "Artificial intelligence" is about as meaningful a term as "artificial geflugelschnitz".

There's a kind of scam that some in the AI industry have foisted on others. The scam is "this is so good, it's going to destroy us. Therefore, we need regulation to prevent Roko's Basilisk or Skynet." LLMs have not gotten better in any significant degree. We have the illusion that they have because LLM companies keep adding shims that, for example, use Python libraries to correctly solve math problems or use an actual search engine on your behalf.
LLM development of new abilities has stagnated. Instead, the innovation seems to be in making models that don't require absurd power draws to run. (Deep Seek being a very notable, very recent example.)
I watched this video all the way through hoping they would turn things around but it's just the same fluff for a new audience.