- cross-posted to:
- technology@lemmit.online
- cross-posted to:
- technology@lemmit.online
Fiction written by artificial intelligence is easy to detect because it struggles with complex story structure and tends to moralize in clunky ways, according to a preprint study from researchers at University of Maryland, College Park and Google DeepMind. They found that AI fiction has tells that go beyond stereotypical overuse of em-dashes and other obvious AI tropes and have more to do with the formulaic nature of the text itself.
“AI stories over-explain themes and favor tidy, single-track plots while human stories frame protagonists’ choices as more morally ambiguous and have increased temporal complexity,” the study, which looked at more than 50,000 AI-generated short stories, found. “Claude produces notably flat event escalation, GPT over-indexes on dream sequences, and Gemini defaults to external character description. We find that AI-generated stories cluster in a shared region of narrative space, while human-authored stories exhibit greater diversity. More broadly, these results suggest that differences in underlying narrative construction, not just writing style, can be used to separate human-written original works from AI-generated fiction.”
Basically, AI-generated fiction sucks and at the moment is easy to detect. The typical method of detection involves looking for stylistic markers such as an abundance of em-dashes, the overuse of the word “delve,” or an obsession with goblins, but this project tried something different. “The idea for this project came because we are hoping to eventually move past plain text detection, into some sort of space where we can separate human ideas from AI-generated ideas,” Jenna Russell, a University of Maryland researcher and one of the study’s authors, told 404 Media. Russell is also an intern at the AI-detection company Pangram.
You must log in or # to comment.
I was recently talking to an independent bookshop owner, looking at some of the submissions for self-published books. It was depressing as fuck. An absolute slop fest. The ‘illustrated’ childrens books were by far the worst offenders.
LMMs are just mediocrity machines.
You input one great book, and 25 dogshit fanfictions, you’re not going to output a great book.
It’s like teaching someone how to cook Chateaubriand, but also 19 recipes for hot dogs, and then expecting them to make braised lamb or some shit.
You are wrong, even a LLM fed with 26 great books couldn’t write a good one.
Oh, I fully fucking agree.
I’m just putting this in ‘crayon eater’ terms for them to try and understand.
AI stories give me whiplash with how borderline personality disorder the characters seem after any significant length.
I mean yeah they mostly learned from fanfic authors.
peak mediocrity
I remember the stuff they added to Duolingo after the leadership went “AI first”.
Duolingo has these story exercises. Old stories were all kind of fun and distinctive, and the characters are fun and memorable. Then they got the bright idea to AI generate more stories.
“Oh, don’t worry, the new content will all be reviewed by humans”, they said.
Were they?
The new stories were… I don’t even remember any more. Best I can describe them is “there’s stuff and it happens and the characters are just kinda there”. It went in one ear, out of the other. I fucking can’t. I don’t think I was learning stuff from that.
And these were tiny stories, like couple of hundred words tops. Anyone uses LLMs to write novels is out of their rocker.
there’s stuff and it happens and the characters are just kinda there
You’re welcome to ignore this, but I feel I have to point out, not to you, but the universe in general (and just coincidentally as a reply to your comment), how well this describes almost everything about the most recent two episodes of a beloved TV series featuring a blue box that travels in time and space (which I like the usual amount). I’m going to go cry now. 🟦
You’re right and you should say it and I’m also going to cry now
Ugh you’re right about Duolingo stories. They’re so formulaic nowadays. Half of them are about Oscar brown-nosing some journalist.
Lots of fiction is mediocre so this tracks.
Also the human part, the general idea of what the story is about is unoriginal because of the AI user himself
Claude produces notably flat event escalation, GPT over-indexes on dream sequences, and Gemini defaults to external character description.
Which are all ways of creating more text without saying anything. Repetition is another classic.
I feel like that’s a problem with all low-effort, AI-generated texts (which is the vast majority of them): You give it a prompt with maybe ten pieces of information and expect it to generate a text that’s a hundred times as long, while also not straying too far from the prompt.
Of course, it’s going to take every opportunity to not say anything that would advance the plot. Because advancing the plot means either using up the little input you gave it, or to invent new information which might contradict the prompt…
Yeah, AI sucks for writing, but I had a back and forth going where ChatGPT wpukd produce like, 3-4 paragraphs and dialog and I would tell it where to go next and it was producing some pretty good stuff. This was back on like, 2 or 3 model versions ago too. I had to stop because it updated the model and suddenly it was all “Wait, you had me writing a sexy story!!! I can’t do that, I am programmed to be a puritan!!!”
You are not going to get a whole book ornshort stoty with one prompt.
50 shades of grey was also stupid and bad and it very much was not written by AI.
50 shades of grey and fanfics like my immortal are fun to read because a person was weird enough to write them. Llm slop is just spam.
I learned recently that it was actually Twilight fanfiction to start, and the author had to tweak the world to make the vampires rich guys.
Which I guess explains it a bit more.
To be fair, it was heavily implied that the Cullens were rich. At least in the films.
If you are a Vampire, especially one of the type where you “mostly act nornal and sometimes suck blood” and not some bat man horror monsyer, you would have a hard time NOT being rich really.
You could potentially work like 5 jobs, you have very few expense needs like food or even proper shelter if you want to save up. Even just a basic investment of any kind would grow enormously over a hundred years.
This just makes me realize that the old people were going to high school in twilight…
I googled it. Edward is over 100.
What the fuck? Just… What the fuck? Bella, was fucking 17.
Jesus Christ, no wonder the presidents a pedophile and nobody cares.
The book explains it by saying every few decades they go back to school to relearn what all has changed, so that part at least kind of makes sense. The falling in love with high school students is hella gross tho.
I stopped being interested in high-schoolers as friends or romantic interests by the time I graduated from university. Our frames of reference were too different for meaningful connection.
Somewhere around 30, I became actively disgusted by the idea of someone my age socializing with teenagers outside of a familial, educational, or mentoring role. (That one guy who graduated from high school 12 years ago and still gets invited to students’ parties because he is willing to bring beer? He’s gross. And so is the 35-year-old teacher who ran off with one of his students the day after she graduated.)
I’m in my mid-40s now and my feelings haven’t changed. I don’t expect that another 60 years of life experience is going to bring me around to the other side.
And that is why I will never understand why Arwen Undómiel would give up her immortality for a man nearly 2,700 years her junior. Even if that man was the legendary Aragorn, son of Arathorn, the most heroic hero I’ve encountered in my literary life.
Thank you for coming to my TED Talk.
Which is pretty telling of how bad llm fiction is.
the overuse of the word “delve” is hilarious
How am I supposed to write a story about goblin miners now?
Not explore, not search, but delve!
Excuse me while I wash my hands after typing that.
Don’t you mean delve your hands after delving that
Papa Smurf? Is everything okay?
Oh no. Did they have to actually read 50,000 slop books? Such masochism…
They let an AI do the evaluation.
Other tells I’ve noticed: lots of purple prose, sentence fragments, and the good ol “not X—Y” framing. Another tell is jarring tone shifts from chapter to chapter or even scene to scene i.e. a character might come across as sympathetic in one part of the story and an absolute psychopath in another for no reason.
It’s a problem in fanfic right now, but I think this’ll pass when prices innevitably go up. It’s not like fanfic writers are really making that much from donations.
Rule of three is a other one. Although, it does make the output nicely structured.
Load bearing
The more we criticise AI, the more it will take notes and improve. And before we know it, we won’t be able to tell if a fictional story is man-made or AI-made.
Edit: I hadn’t realise that this is FuckAI initially. I don’t love AI, I hate what it would do to arts and even the concept of jobs and livelihood, but as much as those who hate AI either don’t realise or don’t want to admit, the potential is there and the wilful ignorance and denial is really just the unspoken feeling of uncanny valley surfacing. AI is here to stay and will improve upon, unfortunately. I just wish it had been open source instead.
Also, the overwhelming majority of people already can’t tell if a music is AI or not. So AI imitating art is already underway as we speak: https://news.sky.com/story/a-third-of-daily-music-uploads-are-ai-generated-and-97-of-people-cant-tell-the-difference-says-report-13469818
the wilful ignorance and denial is really just the unspoken feeling of uncanny valley surfacing
This article is discussing a research paper that looked into AI’s hilarious tendency to “delve,” among other things, and one of the many conclusions they came to is exactly what we all already suspected: it sucks at fiction. That is the furthest thing from being willfully ignorant.
I’m saying that it is not like it won’t improve. I edited my comment to show a news link reporting that the majority of people already can’t tell if a music is AI. The same would happen on written literature.
Except that hasn’t happened.
You underestimate our ability to discover patterns. It’s this trait that has given us language, art, science, etc. That very same ability is why we’re able to figure out small little nuances like the above mentioned delving phenomenon. And the fact that the companies behind these models are so dead set on preventing people from writing erotic fiction in favour of bland literature with forced morals only adds to said predictability.
And if there is one thing people hate more than bad literature, it’s a boring one.
I see your point. But again, I don’t see any reason why AI would not learn to think and behave the same as humans at some point-- if there is no corporate input. I see the current models of AI as still quite literally in infancy.
It’s been in its infancy for a long, long time.
“AI,” as we know it today, is just marketing speak that refers to LLMs or image generators. That isn’t to say there isn’t grain of truth to the usage of the word. Machine learning is the underlining method used to “train” these models with whatever information needed to full fill a task. So, yes, it is “learning” per se but not in any way similar to humans. It’s just a computer doing what we tell it to do at the end of the day.
And that’s fine if it weren’t for the fact that the training data is based on our art and that it consumes even more power and water than even blockchain! Unlike blockchain, there is no known way to undo that. Like, I’ll at least give credit to devs behind Ethereum. They acknowledged it was power hungry and switched to an efficient algorithm. Still doesn’t addresses the laundry list of issues but credit where credit is due. What’s the solution for AI?
The more we criticise muppets, the more they will take notes and improve. And before we know it, we won’t be able to tell if a movie is human acted or muppet acted.
You’re being sarcastic, but you aren’t wrong.
“It will take notes”? AI is just autocorrect on steroids.
AI is still on training. It is either naive or wilful ignorance to think they are not being improved upon.
I’ve read several articles by people in the field who claim that no, the quirks AI has now, cannot be improved on much. Not with the LLM technology that is used now, it would be necessary to start over with a better method.
The only sensible comment here.
Yeah there are limitations with the current AI models, but it doesn’t mean there won’t be breakthrough at some stage. The renewable energy technologies also had limits ten years ago but those limits had been broken through to make it increasingly more efficient than before.
Well, but that’s not what I wrote.
I’m saying in general that detractors of AI think it won’t ever improve and hit its limit, but so were lots of technology before such as renewable energy technology.
Probably, but it’s not going to happen with LLMs. That’s quite a bit different from renewable energy where it was a question of refining existing technology.
It’s naive to think that you can take something and feed it shit. Then, when it regurgitates out shit to you, it’ll give you caviar when you only continuously feed it shit.
They can’t even reliably get image generators to stop generating people with six fingers.
It’s a matter of time. Not if but when.
Anthropic or OpenAI should hire you as a marketer. You sure hit all their nonsense empty promises.
The technology is fundamentally limited. It will never be „intelligent“ because it is founded on flawed premise - that language leads to intelligence and not vice versa.
I’m sure as hell people said the same about horse and buggy being the only mode of personal transport because cars were slow: empty promises.
Only in Lemmy people have so much complete dismissal. Outside of Lemmy, even detractors of AI recognise the potential albeit admit the current model is overrated. Again, I’m certain the feeling of uncanny valley in Lemmy and most don’t recognise it.
LLMs are a dead end technology. You are in a cult.
It’s not that I love LLM, but its potential is there try, as you might deny. I know this is FuckAI, but i find Lemmy in general is experiencing uncanny valley and either don’t realise it or don’t want to admit it and thereby refusing its potential. That’s like saying automobiles in 1800s is a deadend technology and horse and buggy is the only way because the maximum speed of cars was 2 mph.
Love it or hate and try as one might, AI is a Pandora’s box opened. I just wish it’s open source instead.
LLMs need an ever increasing amount of energy and compute for diminishing returns, they all operate at a loss to generate mediocre results and can only function as long as they’re getting unlimited money from investors to burn.
Not every technology just gets exponentially better forever. Some hit a wall, and then that’s it.
Literally tech bros said the same shit about block chain and nfts.
Keep dreaming sweety.
They have nothing to do with AI.
Again, love it or hate, denying AI is like denying cars aren’t better than horse and buggy.
What a ridiculous take.
And yes, bringing up tech bros forcing their delusional nonsense on the general population when, time and time again the last decade you people have been catastrophically wrong just further proves that LMMs are fucking dumb and a massive waste of resources.
🤩That’s not true! 🚀 You people are so mean. 😤
You’re absolutely right!
/s
