[Gary Marcus] Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
[Gary Marcus] Evidence that LLMs are reaching a point of diminishing returns - and what that might mean
The conventional wisdom, well captured recently by Ethan Mollick, is that LLMs are advancing exponentially. A few days ago, in very popular blog post, Mollick claimed that “the current best estimates of the rate of improvement in Large Language models show capabilities doubling every 5 to 14 months”...
Hopefully we're close to realizing generative LLMs are essentially a fad, neat technology but not enough horsepower to do the real interesting stuff
Honestly, I’m not sure how much better I would want chatGPT to be. The only thing I really want is a larger context window.
The only thing I'd like is fact-checking itself, which is impossible considering how they work (it basically guesses everything). LLMs are close to being useful but I find most of the time they're not usable because it keeps spouting wrong info.
diminishing returns
There were returns?