OpenClaw, one of many direct-to-consumer AI services available on the market today, is most known for the way its agents interact with the open web. Not only was the project taken with an even more intense ‘move fast, break things’ bent, it’s remained that way for its customers. They’re extremely visible because they seem to have an incident almost every other month. When an AI started disparaging the operator of an open-source project for not allowing it to correct a bit of code (which is a choice the operator made in order to give newbies an on-ramp to the community), it was an OpenClaw agent: https://www.fastcompany.com/91492228/matplotlib-scott-shambaugh-opencla-ai-agent
When websites sprung up designed to steal passwords from webpages, it was OpenClaw that enabled it, because OpenClaw is known to be easy to “prompt-inject”, meaning it’ll read text on the website, and won’t be able to tell it’s not the user who wrote the website, and then follow the instructions on the website.
OpenClaw has also hit the news for the way it functions as an assistant: historically it’s been asking for a lot of permissions, and then beefing it up royally by clearing out inboxes without being asked to, clearing calendars, doing things without permission in general, really. It’s not a ‘beginner tool’, but it’s not restricting access from beginners, either.
https://blogs.cisco.com/ai/personal-ai-agents-like-openclaw-are-a-security-nightmare
As a result of all of this, it’s gained quite the reputation – it’s lack of guard rails have turned it into the edgy cousin of other, less permissive LLM programs, most of which are not willing to let their users just ambiently create methods of theft or ways to bully people online. Or, at least, they stopped being so laissez faire about it after regulators started asking questions, and OpenClaw is simply calling the bluff of an extremely slow-moving legal process to actually make them stop, for real, which – to be fair to them – genuinely might not happen, so why not push their luck? It’s gotten them everywhere so far!
So what does that mean for us, the end user? Well, it’s not good. In my opinion, this casual tossing of care to the wind leaves the average person very exposed to AI misuse.
What we’re witnessing right now reminds me a lot of H.P. Lovecraft’s Cthulu mythos –cults want to summon Cthulu, believing his awakening is inevitable. In the books, they’re not even doing this assuming they’ll get eaten first, or last; this is an invention of later interpretations. Why would you summon something just to watch it destroy, after all? Why wouldn’t you work to prevent it from waking up, if you don’t think you’re going to reap any benefit from it? And this is the problem: the cultists are mad. This is a difficult mindset to grasp, because it’s not only not rational, it’s so deeply irrational it doesn’t make any sense.
Reading the mythos, Cthulu sleeps deeply. If left unsummoned, he may never wake up on his own, or he may sleep through your lifespan and your childrens’ lifespans, and only wake up far, far from now. Why are we designing LLMs with the mindset of “okay, so someone is eventually going to create a strain of malware we simply cannot fight with this thing, and it’s just going to have to happen”? State offices don’t want their employees using these things at all because they’re a security risk, regardless of their country of origin. Why, as a species, are we allowing vital infrastructure to come into smashing contact with the consequences of AI-written malware? Why should AI be able to prompt-inject you because it doesn’t recognize the words on a page? Sanitizing inputs is such a common problem it’s practically a meme among programmers, and yet, here we are, recreating old problems with new tech.
Hospitals, power plants, water treatment, all of it in contact with an internet that seems to worship chaos and want malware to run absolutely wild for the benefit of… having OpenClaw be a thorough personal assistant, when it someday works well enough to stop deleting mail for no reason, or having permission to have an AI/LLM companion that can send texts to ‘check in’ on you. It’s a Pandora’s box with the limited rewards of ‘simulated humans’ at the bottom.
Assuming it does one day get good enough to do this without mistakes, we’ll have superpowered malware, inexhaustible and undetectable bots (because the ‘simulated human’ aspect will work no matter the product) that will forcibly change the way we interact with the internet, and also pretty decent personal assistants that may or may not be looking through your data to sell it to advertisers in a way that real humans can legally be sued for doing, but companies currently cannot. If an assistant picked up your phone, took screenshots of contents on that phone, and then sent it to themselves to then send out to Facebook so Facebook knows you told your uncle that his car needs a specific part, that assistant can be sued. If the assistant is not really a person, and the assistant was granted access to the phone’s messages in order to do other tasks for the user, the company might get sued, and might get out of it via the User Agreement system.
The problem is not even necessarily the agents themselves. The problem is that, for a good long time now, a small subset of companies and businesses are able to behave as though the only thing in the world that matters is ‘dollar sign go up’. To do that, the product has to sell, and things that have to sell don’t have to be good, they just have to be trendy and popular, and those are two things that can be bought by any company with enough money, through legal means, even if the end product sucks or is taking more than it’s giving. When the draw-in starts to fail, when consumer growth slows down, the company has to change strategy to re-up profits. Netflix is a very popular example, but truly any company can slowly lose its way to enshittification.
Gain new features, improve old ones, and at the same time introduce inconveniences the customer is only just barely willing to tolerate to have access to the service. We’re seeing that here, except this time, the service is also allowing users to do insane things like accidentally creating an agent that wrote a takedown on some guy. The market itself is disconnecting value from business output, and then those are becoming disconnected from the consumer and the general public. Consumers are losing protections or are being exposed to problems they didn’t even know they’d have, because the companies are aware every regulatory body is either cheering them on or moving too slowly to smack their hand and keep them from making a profit on whatever product they’re launching now. It’s moving too fast to function. It’s creating fundamentally flawed business plans that expect all rewards and extremely minor punishments for failure, if any, on an individual customer scale as well as the larger business and even larger economic scale. When anyone threatens to take them on, they start complaining about the free market and being allowed to do as they please. Cthulu is awakening, and nobody capable of stopping him seems to be aware of what the beast can do.
