Because LLMs are really bad at telling stories.

…Look, I’m an advocate for LLMs here. I was using 6B models to mess around with roleplaying and storywriting in like 2022, with GPT-J finetunes. But that’s with human guidance, using it as a tool or “sounding board.”

They’re fascinating writing aids.

But by themself, unattended, off-the-shelf LLMs have a horrendous architecture for writing long form stories, basically. They go off the rails at the drop of a hat, and cannot self correct. They lose track of the longer context, and write things that don’t make sense. They have weird, very inhuman patterns of using the same character names and ideas, or mimicking prose from earlier in the context where it’s not applicable at all. Their prose is terrible in an eerie, repetitive way.

So for children, there’s a risk of:

• Not-safe-for-family content popping up literally randomly, in the middle of the story, which a parent might miss skimming.

• Weird, nonsensical storylines I would classify as “brainrot”

• Very repetitive themes and ideas, basically a lack of variety.

• Instilling strange linguistic habits.

• Emotions and tone that don’t line up with the content, especially when TTS is used (as most TTS models aren’t tightly integrated into the main LLM; they’re tiny, dumb models reading a script blind, with no context).

…Have you seen any horror movies where something is supposed to be human, or acting like it, but it isn’t quite right? Because it clearly doesn’t understand, its just mimicking?

That’s the risk here, one you wouldn’t get even with TikTok doomscrolling. And that’s the best case scenario, without even getting into influence ops, engagement maximization techniques, or enshittification Meta is notorious for.

Researchers have built finetunes with frameworks to try and improve autonomous LLM writing, but as someone who’s followed their research, I can tell you there’s a zero percent chance Meta implemented it.

Yes but if ai raises your kids for you, you can work longer hours in the mines!

Stonks go up!