This is all just JavaScript, so yes. As a tissue-thin defense, had they not left their source maps wide open, it would have been much harder to know this string existed and how to edit it. Not impossible, but much harder.

Yeah, letting the intrinsically insecure RNG recursively rewrite its own security instructions definitely can’t go wrong. I mean they limited it to only so so when the users asked nicely!

Edit to add:

The more I think about it the more it speaks to Anthropic having an absolute nonsense threat model that is more concerned with the science fiction doomsday AI “FOOM” than it is with any of the harms that these systems (or indeed any information system) can and will do in the real world. The current crop of AI technologies, while operating at a terrifying scale, are not unique in their capacity to waste resources, reify bias and inequality, misinform, justify bad and evil decisions, etc. What is unique, in my estimation, is both the massive scale that these things operate despite the incredible costs of doing so and their seeming immunity to being reality checked on this. No matter how many times the warning bells about these systems’ vulnerability to exploitation, the destructive capacity of AI sycophancy and psychosis, or the simple inability of the electrical infrastructure to support their intended power consumption (or at least their declared intent; in a bubble we shouldn’t assume they actually expect to build that much), the people behind these systems continue to focus their efforts on “how do we prevent skynet” over any of it.

Thinking in the context of Charlie Stross’ old talk about corporations as “slow AI,” I wonder if some of the concern comes either explicitly or implicitly from an awareness that “keep growing and consuming more resources until there’s nothing left for anything else, including human survival” isn’t actually a deviation from how these organizations are building these systems. It’s just the natural conclusion of the same structures and decision-making processes that leads them to build these things in the first place and ignore all the incredibly obvious problems. They could try and address these concerns at a foundational or structural level instead of just appending increasingly complex forms of “please don’t murder everyone or ignore the instructions to not murder everyone” to the prompt, but doing that would imply that they need to radically change their entire course up to this point and increasingly that doesn’t appear likely to happen unless something forces it to.

@Soyweiser @fiat_lux

So many of these people, as with the NFT clowns, have “Twelve Year Old First Day On The Internet” Energy

I don’t work at one of those companies, just somewhere mainlining AI, so this answer might not satisfy your requirements. But the answer is very simple. The first thing anyone working in AI will tell you (maybe only internally?) is that the output is probabilistic not deterministic. By definition, that means it’s not entirely consistent or reproducible, just… maybe close enough. I’m sure you already knew that though.

However, from my perspective, even if it was deterministic, it wouldn’t make a substantial difference here.

For example, this file says I can’t ask it to build a DoS script. Fine. But if I ask it to write a script that sends a request to a server, and then later I ask it to add a loop… I get a DoS script. It’s a trivial hurdle at best, and doesn’t even approach basic risk mitigation.

I’m sure these English instructions work because they feel like they work. Look, these LLMs feel really great for coding. If they don’t work, that’s because you didn’t pay $200/month for the pro version and you didn’t put enough boldface and all-caps words in the prompt. Also, I really feel like these homeopathic sugar pills cured my cold. I got better after I started taking them!

No joke, I watched a talk once where some people used an LLM to model how certain users would behave in their scenario given their socioeconomic backgrounds. But they had a slight problem, which was that LLMs are nondeterministic and would of course often give different answers when prompted twice. Their solution was to literally use an automated tool that would try a bunch of different prompts until they happened to get one that would give consistent answers (at least on their dataset). I would call this the xkcd green jelly bean effect, but I guess if you call it “finetuning” then suddenly it sounds very proper and serious. (The cherry on top was that they never actually evaluated the output of the LLM, e.g. by seeing how consistent it was with actual user responses. They just had an LLM generate fiction and called it a day.)

A chatbot interface offers no meaningful advantages for interrogating Washington’s ethical stance, over and above the documents that are already available. Instead, it offers a pleasant sheen of false certainty. So in that way, it’s dragging a guy who’s been dead for two centuries into the social media era. Huzzah!

In my experience “all hands” meetings are very much CEOs and their sycophants cosplaying at podcast hosts for an hour whilst forcing their employees to watch/listen. They are almost never useful and a colossal waste of money - especially in corporation’s with 10k+ employees. Like the salary cost for 10k people for 1 hour would probably pay off my mortgage.

I was sufficiently interested based off of this that I tracked down a few others of his. This one felt like a good take for an era where these things are being used for more than just slop generation despite the underlying flaws not being resolved.