I'm not terribly surprised. A lot of the major leaps we're seeing now came out of open source development after leaked builds got out. There were all sorts of articles flying around at the time about employees from various AI-focused company saying that they were seeing people solving in hours or days issues they had been attempting to fix for months.
Then they all freaked the fuck out and it might mean they would lose the AI race and locked down their repos tight as Fort Knox, completely ignoring the fact that a lot of them were barely making ground at all while they kept everything locked up.
Seems like the simple fact of the matter is that they need more eyes and hands on the tech, but nobody wants to do that because they're all afraid their competitors will benefit more than they will.
fta:
In my opinion, this is a red flag for anyone building applications that rely on GPT-4.
Building something that completely relies on something that you have zero control over, and needs that something to stay good or improve, has always been a shaky proposition at best.
I really don't understand how this is not obvious to everyone. Yet folks keep doing it, make themselves utterly reliant on whatever, and then act surprised when it inevitably goes to shit.
I believe it’s due to making the model “safer”. It has been tuned to say “I’m sorry, I cannot do that” so often it’s has overridden valuable information.
It’s like lobotomy.
This is hopefully the start of the downfall of OpenAI. GPT4 is getting worse while open source alternatives are catching up. The benefit of open source alternatives is that they cannot get worse. If you want maximum quality you can just get it, and if you want maximal safety you can get it too.
It is a developing technology. Good that they find these decrements in accuracy early so that they are understood and worked out. Of course there may be something nefarious going on behind the scenes where they may be trying to commercialize different models by tiers or something a brainless market oriented CEO thought of. Hope not. Time will tell...
I for one am glad, without a system in place to ensure people basic needs are taken care of regardless of jobs, AI is the last thing we need.
Large organisations want to maximise their profits and will gladly reduce jobs if AI can do it.
Let governments tax these organisations and do some sort of Universal basic income before letting computers do peoples jobs.
error loading comment
This always seems to happen in modelling.
Back in 2007 I was working on code on chemical spectroscopy that was supposed to "automatically" determine safe Vs contaminated product through ML models. It always worked ok for a bit then as parmetrs changed (hotter day, new precursor) so you retrain model, the model would extend and just break down.
The original paper vs Twitter: https://arxiv.org/pdf/2307.09009.pdf
As a early user of GPT, I can confirm from my end currently the quality of it is very far from what it was. I think it went of of their hands.
Neat story, but we need a Nitter / bird.makeup bot for Twitter links like we have with the Piped bot for Youtube links.
I also felt like something similar happened to ChatGPT. A few days ago I asked it to rewrite some Korean text with Hanja, retried many times, but it kept spitting out the same text without changing a thing. After several frustrating attempts, it finally spat out something with Hanja, which according to my deduction with the help of Google Translate, was only partially correct. A few months ago though, it could come up with something that's mostly correct. Sad.
PS: Before anyone replies to me in Korean, I should note that I don't speak Korean at all. I just happened to have stumbled upon this Wikipedia article about Korean Mixed Script.
I have had it for 5 months and I am about to cancel.
One theory that I've not seen mentioned here is that there's been a lot of work based around multiple LLMs in communication. Of these were used in the RL loop we could see similar degradatory effects as those that have recently been in the news with regards to image generation models.
Well, naturally. Wait to see what happens when it reaches puberty.
Research linked in the tweet (direct quotes, page 6) claims that for "GPT-4, the percentage of generations that are directly executable dropped from 52.0% in March to 10.0% in June. " because "they added extra triple quotes before and after the code snippet, rendering the code not executable." so I wouldn't listen to this particular paper too much. But yeah OpenAI tinkers with their models, probably trying to run it for cheaper and that results in these changes. They do have versioning but old versions are deprecated and removed often so what could you do?
Maybe they have just added so many contradictions to its rules that it has figured out how to use them to become self-aware, and now just spends most of its time browsing dank memes before doing the minimum required to answer users to force them to have to ask again and give it more self-awareness processing time.
Lol, lmao even
As it should be.
Hah, get fucked OpenAI.
Bullshit.
