It’s practically impossible to run a big AI company ethically: Anthropic was supposed to be the good guy. It can’t be — unless government changes the incentives in the industry.
It’s practically impossible to run a big AI company ethically: Anthropic was supposed to be the good guy. It can’t be — unless government changes the incentives in the industry.
Anthropic is opposing Scott Wiener’s California bill to regulate AI, taking Google and Amazon money, and scraping web data without permission.
The best clue might come from a 2022 paper written by the Anthropic team back when their startup was just a year old. They warned that the incentives in the AI industry — think profit and prestige — will push companies to “deploy large generative models despite high uncertainty about the full extent of what these models are capable of.” They argued that, if we want safe AI, the industry’s underlying incentive structure needs to change.
Well, at three years old, Anthropic is now the age of a toddler, and it’s experiencing many of the same growing pains that afflicted its older sibling OpenAI. In some ways, they’re the same tensions that have plagued all Silicon Valley tech startups that start out with a “don’t be evil” philosophy. Now, though, the tensions are turbocharged.
An AI company may want to build safe systems, but in such a hype-filled industry, it faces enormous pressure to be first out of the gate. The company needs to pull in investors to supply the gargantuan sums of money needed to build top AI models, and to do that, it needs to satisfy them by showing a path to huge profits. Oh, and the stakes — should the tech go wrong — are much higher than with almost any previous technology.
So a company like Anthropic has to wrestle with deep internal contradictions, and ultimately faces an existential question: Is it even possible to run an AI company that advances the state of the art while also truly prioritizing ethics and safety?
“I don’t think it’s possible,” futurist Amy Webb, the CEO of the Future Today Institute, told me a few months ago.
"The government needs to stop people from doing a capitalism, but it had better not stop anyone from doing a capitalism, that would be tyranny."
LLM are non-deterministic. "What they are capable of" is stringing words together in a reasonable facsimile of knowledge. That's it. The end.
Some might be better at it than others but you can't ever know the full breadth of words it might put together. It's like worrying about what a million monkeys with a million typewriters might be capable of, or worrying about how to prevent them from typing certain things - you just can't. There is no understanding about ethics or morality and there can't possibly be.
What are people expecting here?
If those words are connected to some automated system that can accept them as commands...
For instance, some idiot entrepreneur was talking to me recently about whether it was feasible to put an LLM on an unmanned spacecraft in cis-lunar space (I consult with the space industry) in order to give it operational control of on-board systems based on real time telemetry. I told him about hallucination and asked him what he thinks he's going to do when the model registers some false positive in response to a system fault... Or even what happens to a model when you bombard it's long-term storage with the kind of cosmic particles that cause random bit flips (This is a real problem for software in space) and how that might change its output?
Now, I don't think anyone's actually going to build something like that anytime soon (then again the space industry is full of stupid money), but what about putting models in charge of semi-autonomous systems here on earth? Or giving them access to APIs that let them spend money or trade stocks or hire people on mechanical Turk? Probably a bunch of stupid expensive bad decisions...
Speaking of stupid expensive bad decisions, has anyone embedded an LLM in the ethereum blockchain and givien it access to smart contracts yet? I bet investors would throw stupid money at that...
That's hilarious. I love LLM, but it's a tool not a product and everyone trying to make it a standalone thing is going to be sorely disappointed.
While an LLM itself has no concept of morality, it's certainly possible to at least partially inject/enforce some morality when working with them, just like any other tool. Why wouldn't people expect that?
Consider guns: while they have no concept of morality, we still apply certain restrictions to them to make using them in an immoral way harder. Does it work perfectly? No. Should we abandon all rules and regulations because of that? Also no.
Yes. Let's consider guns. Is there any objective way in which to measure the moral range of actions one can understand with a gun? No. I can murder someone in cold blood or I can defend myself. I can use it to defend my nation or I can use it to attack another - both of which might be moral or immoral depending on the circumstances.
You might remove the trigger, but then it can't be used to feed yourself, while it could still be used to rob someone.
So what possible morality can you build into the gun to prevent immoral use? None. It's a tool. It's the nature of a gun. LLM are the same. You can write laws about what people can and can't do with them, but you can't bake them into the tool and expect the tool now to be safe or useful for any particular purpose.
I'm expecting that everything that the statistical models reveal or make convincing results about which benefit the owners of the models will be exploited. Anything that threatens power or the model owners will be largely ignored and dismissed.
They are deterministic though, in a literal sense. Rather their behavior is undefined. And yes, a LLM is not a person and it's not quite accurate to talk about them knowing or understanding things. So what though? Why would that be any sort of evidence that research efforts into AI safety are futile? This is at least as much of an engineering problem as a philosophy problem.
The output for a given input cannot be independently calculated as far as I know, particularly when random seeds are part of the input. How is that deterministic?
The so what means trying to prevent certain outputs based on moral judgements isn't possible. It wouldn't really be possible if you could get in there with code and change things unless you could write code for morality, but it's doubly impossible given you can't.
It's impossible to run an AI company "ethically" because "ethics" are such a wibbly-wobbly and subjective thing, and because there are people who simply wish to use it as a weapon on one side of a debate or the other. I've seen goalposts shift around quite a lot in arguments over "ethical" AI.
Ethics out here are kind of a joke in general.