Leading AI Scientists Warn AI Could Escape Control at Any Moment
Leading AI Scientists Warn AI Could Escape Control at Any Moment
During an international summit, some of the world's foremost AI experts came together to write a definitive dispatch on its dangers.
"Experts agree these AI systems are likely to be developed in the coming decades, with many of them believing they will arrive imminently," the IDAIS statement continues. "Loss of human control or malicious use of these AI systems could lead to catastrophic outcomes for all of humanity."
You're viewing a single thread.
Oh no, they'll write really average essays! What ever shall we do!!!
Or maybe they'll produce janky videos that don't make any sense so have to be shorter than 10 seconds to cover up the jank!!!
Language models aren't intelligent. They have no will of their own, they don't "understand" anything they write. There's no internal thought space for comprehension. They're not learning. They're "trained" to mimick statistically average results within a search space.
They're mimicks, and can't grow beyond or outdo what they've been given to mimick. They can string lots of information together but that doesn't mean they know what they're saying, or how to get anything done.
in the coming decades
Given that in the past 15 years we went from "solving regression problems a little bit better than linear models some of the time" to what we have now, it's not unfounded to think 15 years from now people could be giving LLMs access to code execution environments
*20 years, not 15.
That's not really machine learning though. If you wanted to go way back, AI research goes back to implementations of hebbian learning in computer science back in the 1950s as a way of emulating human neurons. I was merely pointing out that AI was a computer science "dead end" until restricted Boltzmann machines were revisited by Hinton et al back in 2008 or so, and that 99% of the growth in the field has happened since the early 2010s when we reached a turning point where deep learning models could actually outperform classical statistical models like regression and random forests
People have been doing that already, checkout Devin.
You’re wrong in so many ways.
And by the time you realize it. It’ll be too late. There’s are dozens more those links available.
You're wrong, and silly. The post you're linking to is from the personal blog of a data "manager" whose focus is how decisions are made. The post is about what an "AI" might interpret as meaning.... but it completely overlooks that ALL OUTPUTS are trained....
The posters method? Talking to ChatGPT. So you may as well be invoking Blake Lemoine (the spiritualist at Google who believed language models had souls because he talked to them and they seemed to) - the post you're linking is making that same mistake. From the post:
"However, the responses that I got from ChatGPT seemed more coherent than “haphazardly stitched together” sequences of forms it has observed in the past. It seems like it is working off some kind of conceptual model." [emphasis added]
It's trained on humans who write as if they have conceptual models THAT'S THE ENTIRE TRICK. That's why it "seems to have intelligence" in the responses, because it's mimicking the intelligence that went into writing all that training data - OUR HUMAN INTELLIGENCE. We wrote the data it trains on, WE have intelligence, it has a fancy probabilistic form of regurgitation.
The probabilities are done by the "shape" of language, but that's not understanding. That's not having an internal sense of the world or what's being said. It's "locked in a mode" (at training time, and limited to the training data and on screen memory/text).
But yeah dude, posting that link as "proof" of intelligence is silly. Just because something can pretend or "seem to" have reasoning, or dreams, or decision making, doesn't mean that those things are being done. LLMs only respond when prompted - they're not sitting there thinking when they're silent. Likewise, they're not learning outside of the text on the screen, and their training data. They won't "think" about any conversations they've had in the past, they won't think about anything after they've done their output.
It's an echo of the training data.... some of which is discussions about meaning, or discussions that appear to show a conceptual framework, or talk about the experience of reasoning, or dreaming, or having a sense of meaning. So the LLM can write about those things as if it has them, or has done them.... but it hasn't. Those outputs are from the HUMANS who had the EXPERIENCES. The LLM, doesn't do any of that, it just writes as if it does. It writes as if it has intelligence, because intelligent data went into it, and so some people mistake that as intelligence.
People who mistake an image, for substance, may as well be claiming paintings of food are food, maps of places ARE the places, or that there's a "mirror world" in your bathroom mirror. It's cute like a child's fantasy is cute. But to suggest such in this domain - as an adult - shows either idiocy in its highest form, or simply a complete lack of understanding of the technology. Of it's nature. Of what's going on.
You're being tricked into seeing intelligence where there isn't any, because it's reflected in the training data WE (intelligent beings) wrote. You've adopted the intended illusion, rather than questioned it. An LLM has told you something, and you've believed it - much like that blog post, much like Blake Lemoine. Go try and walk into a mirror, you won't get in, it's flat, the image isn't really there - it's just a piece of glass with a black background, reflecting the world outside of it as if it's inside of it.
You're right. They're more than stochastic parrots. And some people here don't realize that. They can do a lot of things. But as is, they lack any substancial internal state hence things like consciousness, the ability to learn while in operation and a body. So while AI content can harm people and society, we're still far away from the robot apocalypse.
They’re more than stochastic parrots. And some people here don’t realize that. They can do a lot of things.
They can only do what they're trained to do. There's been no proven new functions that aren't already present in the training data. Much of the "novel functions" such as finding they can speak in other languages is because that data was online already. It was already in the scraped information they were trained on.
So whilst no doubt they're a technology that will be applied to many data sets, they will always rely on those data sets to produce content/outputs. Otherwise they would no longer be LLMs, they'd be augmented. So far no augmentation written in their code produces intelligence....
....to go further - we have never had a means of "coding" something into sentience, and likely never will. Sentience from Semantics is a pipe dream (akin to sigils, or magic enchanted rituals/symbols). We need more than semantic models/theories.
Some people just wish to argue from faith in future possibilities, rather than what's currently possible/happening.
You can only do what you’re trained to do as well. The only difference is you get to continue to exist after you’ve completed whatever task you were assigned at the moment. I still remember people incapable of seeing any future to the web. That is the kind of mentality that is pervading this space. But as with most things in tech and programming in particular. Garbage in garbage out.
Nope, I can re-train for small tasks at a moments notice. I can learn as I go and retain that information longterm, then make choices much later based on what I've learned.
I think and have an internal world which chugs along constantly because I am autonomous.
These are all characteristics a human intelligence has, that language models don't.
These are the hurdles.
...but also, obviously this is a huge field with huge possibilities (no one is denying that), but it's not intelligence yet.
Potential doesn't equate to reality - and it's only potential until it does. Then it's reality.
Right now in reality, there's no intelligence there. Regardless of whether there might be one day.
Sure. What I'm referring to is that they just don't generate any random garbage. But actually store knowledge and have the ability to combine and apply that. Sure they get trained on some datasets. That's what AI and machine learning is all about. But it has complex implications and consequences. LLMs work very unlike "intelligent" living creatures. However that doesn't mean they can't generate "intelligent" text. They do it a different way. There are some severe limitations as of now and I didn't find good use for my real-world tasks yet as they're just not intelligent enough to do anything useful. Except translation and role-play games. That works very well and I'm glad I have something outperforming google translate by quite some degree. Intelligence isn't well defined. And it's not set in stone that you need human-like intelligence for lots of tasks... And I mean even a human can only do things they've learned before. Or infer things from other things they've learned. So fundamentally it's not that different. For example I'm not a lawyer. If I wanted to write some legal document, I'd need to read a lot of stuff and study that matter. An LLM would need to do exactly the same to be enabled to generate text that sounds like a legal document. And the "intelligence" part we're talking about is finally understanding the subject and be able to connect things, so to speak. Infer, and apply learned knowledge to new things. And we have some evidence that AI can do exactly that. So... It's a bit crude, and not there yet. But it's more than a stochastic parrot. The fundamental parts to a subarea of intelligence is there. And not by accident. Machine learning was invented to infer patterns from some datasets.
And I'm not sure about the sentience part either. Sure it's completely impossible with the current approach. But is there a fundamental barrier? Didn't nature already "code" it into existence with the structure of our brains? And we found out it's just physics? A bit of chemistry and electricity in a complex structure of interconnected cells? It's utter sci-fi, but why wouldn't we be able to the same with silicone chips? I know people regularly deny the possibility. But I've never seen a good argument or a scientific paper ruling it out. I think it's still debated whether there are fundamental barriers. Or what makes sentience in the first place. Just stating some uninformed opinion on that doesn't proove anything. And for a positive proof we're missing a good idea, research and even any hardware that'd be remotely capable of doing the calculations, lots of money and energy. So we're far away from even thinking about it. So maybe we'll know in 100 years. Or you give me some mathematical proof that rules it out?!
I'm not really here to debate the possible, just state the reality: It's not intelligent. Regardless of the fact it produces "intelligent text"... which I take it is short for "intelligent sounding text".... which of course it does - that's what it was trained on.
Fair enough. Yeah, the article was about the future and hypothetical advancements in science in the next decades to come. But I'd agree. As of now I wouldn't call it intelligent. I tried letting ChatGPT write my emails and despite everyone hyping AI to no end and calling the newest one on a PhD level student... I don't see that at all.