cross-posted from: https://feddit.online/c/[email protected]/p/2029056/406-scientists-demand-ai-models-be-cut-off-from-the-internet-immediately-and-demand-a-m
How about somebody prosecutes the criminal acts of these companies instead?
(Disclaimer: I’m just memeing and reserve the right to change my position if I learn of bad unintended consequences.)
This is stupid. Open-weight models that can largely do what the OpenAI/Anthropic models did are already out in the wild.
Do these people not understand that you can’t put the cat back in the bag?
It’s not at all stupid, because there’s no such thing as a “rogue agent,” only rogue users of AI. With proper airgapping, there’s no such thing as a “rogue agent.”
But isn’t that the problem? You can’t force a rogue user to do anything; or they wouldn’t be rogue. So why should they apply air gapping to their agents?
I must be missing something, because you’re getting a bunch of upvotes. I don’t disagree with anything you said, but it feels like a total non-sequitur to what I was saying.
Yes - proper air-gapping is a good idea. But asking the big AI labs to air gap is kind of pointless. Liability already gives them legal incentives to avoid incidents. If that didn’t stop them, a letter signed by a bunch of academics sure won’t. More importantly, open-weight models capable of doing all the things that we’re concerned about are already in the hands of bad actors around the world.
This letter is asking us to lock the front door of the house when the back wall is missing.
Liability already gives them legal incentives to avoid incidents.
That’s quite optimistic, especially since there’s almost no enforcement of liability.
… open-weight models capable of doing all the things that we’re concerned about are already in the hands of bad actors around the world.
Computational resources are the limiting factor. AI companies have resources which no independent actor could ever dream of, which makes them far more dangerous to the general public. A chihuahua with an attitude problem can only do so much damage. A Great Dane with an attitude problem is a threat no one can afford to ignore.
You think scammers in Africa/India are capable of running a local open weight model equivalent to the performance of a cloud frontier model?
The ones in Russia definitely are. They don’t have to be local models - you run them in the cloud like anyone else.
The idea that the frontier labs have LLMs with special capabilities that make them dramatically better than the top open-weight models is largely marketing.
The reality is that most systems are poorly secured, and LLMs make exploiting vulnerabilities fast, cheap, and easy. What would have taken a trained security researcher weeks before can be done by a much less skilled attacker with an LLM in a few hours.
This is correct. We can thrash all we want about slop and hallucinations, but from a defensive posture position, the models only need to be right some of the time and they still constitute a threat.
Knowing it sucks and pretending it doesn’t exist are two different things.
the models only need to be right some of the time and they still constitute a threat.
Right!?!
And with how cheap and fast they are to run, they don’t have to be right that often as long as a separate run can verify/validate claims. In a lot of ways what the LLMs do should probably be thought of as just a very sophisticated sort of brute force approach. You’re effectively just searching the space of combinations of known vulnerabilities. It doesn’t require particular cleverness, just an encyclopedic knowledge, a reasonable way to prune the search space, and the ability to put in the compute time.
I think this is suggesting that all AI models have restrictions. Yes, some people will ignore those restrictions, but at least there will be some recourse as there will be actual rules put in place. If a company or an individual does something reckless with AI that affects another company or individual negatively the offender may be able to currently say “it wasn’t me it was the AI/I didn’t know”. They may not have technically broken any rules since the current law is not able to account for AI agents acting somewhat autonomously. If there are rules in place then at least we can point to that and say “it doesn’t matter that you did not explicitly instruct it to do X, you should have had the following things in place to prevent that”.
The fact that some people may be able to run local models and may never be caught does not mean that we shouldn’t have the infrastructure in place to stop the harm where and when we can.
Technology is famously hard for the law to keep up with, but I don’t think that’s what’s going on here. We have laws for when a person uses a technical tool to obtain unauthorized access, exfiltrate data, or do other malicious things. These don’t necessarily require any specific intent. If you’ve set up an AI tool and pointed it in the direction of hacking without proper airgapping, you’ve probably violated the CFAA if you’re in the US when it’s successful. Other countries generally have similar laws. Prosecutors don’t need to try someone if it’s really ambiguous or unclear what they actually did. Is this really unreasonable under the legal system?
To the extent there’s a gap, it’s the system around the law. Lawyers and judges aren’t technical experts, and may not recognize “it wasn’t me, it was the ai” as the same argument as “it wasn’t me who shot him, it was my gun”, in that it’s usually foolish and unserious, but there can be a convoluted scenario in which it’s a good argument.
