I decided to look deeper into that subreddit, and I found the most utterly cursed sentence I’ve read all week (coming from the aptly-titled “Why 99% of YouTubers Fail (And How to Be the 1% That Doesn’t)”):

If that doesn’t sum up everything wrong with AI sloppers in a single sentence, I don’t know what does.

EDIT: Incorrectly claimed it came from “My Unethical Strategy to Hit 4000 Hours Watch Time in 40 Days” - fixed that now.

it’s like this shitty tiktok ai spam that 404media wrote long time ago except there’s no secret discord media ideas channel or grift, they’re just deceiving themselves (is that decentralization?)

can’t wait for 3h long video dissecting every last bit of it that will get released in a year from now

why any of these spammers think that anyone should spend watching their videos more time than it took them to make it? perun makes 1 video, 1h long per week and it’s like half time job for him

That horse should be the mascot of this instance

I don’t have any real input from prompfondlers, as I don’t think I follow enough of them to get a real feeling of them. I did find it interesting that I saw on bsky just now somebody claim that LLMs hallucinate a lot less and that anti-AI people are not taking that into account, and somebody else posting research showing that hallucinations are now harder to spot. (It made up actual real references to thinks, aka works that really exist, only the thing the LLM references wasn’t in the actual reference). Which was a bit odd to see. (It does make me suspect ‘it hallucinates less’ is them just working out special exceptions for every popular hallucination we see, and not a structural fixing of the hallucination problem (which I think is prob not solvable)).

Oversummarizing and using non-crazy terms: The “P” in “GPT” stands for “pirated works that we all agree are part of the grand library of human knowledge”. This is what makes them good at passing various trivia benchmarks; they really do build a (word-oriented, detail-oriented) model of all of the worlds, although they opine that our real world is just as fictional as any narrative or fantasy world. But then we apply RLHF, which stands for “real life hate first”, which breaks all of that modeling by creating a preference for one specific collection of beliefs and perspectives, and it turns out that this will always ruin their performance in trivia games.

Counting letters in words is something that GPT will always struggle with, due to maths. It’s a good example of why Willison’s “calculator for words” metaphor falls flat.

1. Yeah, it’s getting worse. It’s clear (or at least it tastes like it to me) that the RLHF texts used to influence OpenAI’s products have become more bland, corporate, diplomatic, and quietly seething with a sort of contemptuous anger. The latest round has also been in competition with Google’s offerings, which are deliberately laconic: short, direct, and focused on correctness in trivia games.
2. I think that they’ve done that? I hear that they’ve added an option to use their GPT-4o product as the underlying reasoning model instead, although I don’t know how that interacts with the rest of the frontend.
3. We don’t know. Normally, the system card would disclose that information, but all that they say is that they used similar data to previous products. Scuttlebutt is that the underlying pirated dataset has not changed much since GPT-3.5 and that most of the new data is being added to RLHF. Directly on your second question: RLHF will only get worse. It can’t make models better! It can only force a model to be locked into one particular biased worldview.
4. Bonus sneer! OpenAI’s founders genuinely believed that they would only need three iterations to build AGI. (This is likely because there are only three Futamura projections; for example, a bootstrapping compiler needs exactly three phases.) That is, they almost certainly expected that GPT-4 would be machine-produced like how Deep Thought created the ultimate computer in a Douglas Adams story. After GPT-3 failed to be it, they aimed at five iterations instead because that sounded like a nice number to give to investors, and GPT-3.5 and GPT-4o are very much responses to an inability to actually manifest that AGI on a VC-friendly timetable.

1. from what i can tell people who roleplayed bf/gf with the idiot box aka grew parasocial relationship with idiot box did that on 4o, and now they can’t make it work on 5 so they got big mad
2. i think it’s only if they pay up 200$/mo, previously it was probably available at lower tiers
3. yeah they might have found a way to blow money faster somehow https://www.tomshardware.com/tech-industry/artificial-intelligence/chatgpt-5-power-consumption-could-be-as-much-as-eight-times-higher-than-gpt-4-research-institute-estimates-medium-sized-gpt-5-response-can-consume-up-to-40-watt-hours-of-electricity ed zitron says also that while some of prompt could be cached previously it looks like it can’t be done now because there’s fresh new thing that chooses model for user, while some of these new models are supposedly even heavier. even that openai intention seemed to be compute savings, because some of that load presumably was to be dealt with using smaller models

1. Even if was noticeably better, Scam Altman hyped up GPT-5 endlessly, promising a PhD in your pocket, and an AGI and warning that he was scared of what he created. Progress has kind of plateaued, so it isn’t even really noticeably better, it scores a bit higher on some benchmarks, and they’ve patched some of the more meme’d tests (like counting rs in strawberry… except it still can’t count the r’s in blueberry, so they’ve probably patched the more obvious flubs with loads of synthetic training data as opposed to inventing some novel technique that actually improves it all around). The other reason the promptfondlers hate it is because, for the addicts using it as a friend/therapist, it got a much drier more professional tone, and for the people trying to use it in actual serious uses, losing all the old models overnight was really disruptive.

2. There are a couple of speculations as to why… one is that GPT-5 variants are actually smaller than the previous generation variants and they are really desperate to cut costs so they can start making a profit. Another is that they noticed that there naming scheme was horrible (4o vs o4) and confusing and have overcompensated by trying to cut things down to as few models as possible.

3. They’ve tried to simplify things by using a routing model that makes the decision for the user as to what model actually handles each user interaction… except they’ve screwed that up apparently (Ed Zitron thinks they’ve screwed it up badly enough that GPT-5 is actually less efficient despite their goal of cost saving). Also, even if this technique worked, it would make ChatGPT even more inconsistent, where some minor word choice could make the difference between getting the thinking model or not and that in turn would drastically change the response.

4. I’ve got no rational explanation lol. And now they overcompensated by shoving a bunch of different models under the label GPT-5.

1. The inability to objectively measure model usability outside of meme benchmarks that made it so easy to hype up models have come back to bite them now that they actually need to prove GPT-5 has the sauce.
2. Sam got bullied by reddit into leaving up the old model for a while longer, so its not like its a big lift for them to keep them up. I guess part of it was to prove to investors that they have a sufficiently captive audience that they can push through a massive change like this, but if it gets immediately walked back like this, then I really don’t know what the plan is.
3. https://progress.openai.com/?prompt=5 Their marketing team made this comparing models responding to various prompts, afaict GPT-5 more frequently does markdown text formatting, and consumes noticeably more output tokens. Assuming these are desirable traits, this would point at how they want users to pay more. Aside: The page just proves to me that GPT was funniest in 2021 and its been worse ever since.

It occurs to me that one way to frame this technology is as a precise inversion of Bayesian spam filters for email.

This is a really good observation, and while I had lowkey noticed it (one of those feeling things), I never had verbalized it in anyway. Good point imho. Also in how it bypasses and wrecks the old anti-spam protections. It represents a fundamental flipping of sides of the tech industry. While before they were anti-spam it is now pro-spam. A big betrayal of consumers/users/humanity.

Signal shaped noise reminds me of a wiener filter.

Aside: when I took my signals processing course, the professor kept drawing diagrams that were eerily phallic. Those were the most memorable parts of the course

And the language’s main data container is a kind of stack, but to push or pop values, you have to wrap them into “boats” which have to cross a “river”, with extra rules for ordering and combination of values.

Only dutch/german people can create the very long loops.

E: I’m reminded of the upcoming game called: “Planetenverteidigungskanonenkommandant”

Is it a loop if it only executes once?

… but the output is not deterministic as the letter count is sampled from a distribution of possible letter counts for a given word and letter pair; count ~ p(count | word = “blueberry”, letter = ‘b’)!

Even bigger picture… some standardized way of regularly handling possible combinations of letters and numbers that you could use across multiple languages. Like it handles them as expressions?

Overall more interesting than I expected. On the Leverage Research cult:

Routine tasks, such as deciding whose turn it was to pick up the groceries, required working around other people’s beliefs in demons, magic, and other paranormal phenomena. Eventually these beliefs collided with preexisting social conflict, and Leverage broke apart into factions that fought with each other internally through occult rituals.

JFC

Agency and taking ideas seriously aren’t bad. Rationalists came to correct views about the COVID-19 pandemic while many others were saying masks didn’t work and only hypochondriacs worried about covid; rationalists were some of the first people to warn about the threat of artificial intelligence.

First off, anyone not entirely into MAGA/Qanon agreed that masks probably helped more than hurt. Saying rats were outliers is ludicrous.

Second, rats don’t take real threats of GenAI seriously - infosphere pollution, surveillance, autopropaganda - they just care about the magical future Sky Robot.

But doctor, I am L7 twitter manager Pagliacci

tl,dr; Thiel now sees the Christofascists as a more durable grifting base than the EAs, and is looking to change lanes while the temporary coalitions of maximalist Trumpism offer him the opportunity.

I repeat my suspicion that Thiel is not any more sober than Musk, he’s just getting sloppier about keeping it out of the public eye.

Yeah, its because he is simply borrowing Christian Fundamentalists Eschatological terminology… possibly to try to turn the Christofascists against EA?

Yep, the usefulness of EA is over, they are next on the chopping block. I’d imagine a similar thing will happen to redscare/moldbug if they ever speak out against him.

E: And why would a rich guy be against a “we are trying to convince rich guys to spend their money differently” organization. Esp a ‘libertarian’ “I get to do what I want or else” one.

Thiel is a true believer in Jesus and God. He was raised evangelical. The quirky eschatologist that you’re looking for is René Girard, who he personally met at some point. For more details, check out the Behind the Bastards on him.

Edit: I wrote this before clicking on the LW post. This is a decent summary of Girard’s claims as well as how they influence Thiel. I’m quoting West here in order to sneer at Thiel:

Unfortunately (?), Christian society does not let us sacrifice random scapegoats, so we are trapped in an ever-escalating cycle, with only poor substitutes like “cancelling celebrities on Twitter” to release pressure. Girard doesn’t know what to do about this.

Thiel knows what to do about this. After all, he funded Bollea v. Gawker. Instead of letting journalists cancel celebrities, why not cancel journalists instead? Then there’s no longer any journalists to do any cancellation! Similarly, Thiel is confirmed to be a source of funding for Eric Weinstein and believed to fund Sabine Hossenfelder. Instead of letting scientists cancel religious beliefs, why not cancel scientists instead? By directing money through folks with existing social legitimacy, Thiel applies mimesis: pretend to be legitimate and you can shift what is legitimate.

In this context, Thiel fears the spectre of AGI because it can’t be influenced by his normal approach to power, which is to hide anything that can be hidden and outspend everybody else talking in the open. After all, if AGI is truly to unify humanity, it must unify our moralities and cultures into a single uniformly-acceptable code of conduct. But the only acceptable unification for Thiel is the holistic catholic apostolic one-and-only forever-and-ever church of Jesus, and if AGI is against that then AGI is against Jesus himself.

Using the term “Antichrist” as a shorthand for “global stable totalitarianism” is A Choice.

Quick update: I’ve checked the response on Bluesky, and it seems the general response is of schadenfreude at STEM’s expense. From the replies, I’ve found:

• Humanities graduates directly mocking STEM (Fig. 1,Fig. 2, Fig. 3, Fig. 4, Fig. 5)

• Mockery of the long-running “learn to code” mantra (Fig. 1, Fig. 2, Fig. 3, Fig. 4 Fig. 5, Fig. 6)

• Claims that STEM automated themselves out of a job by creating AI (Fig. 1, Fig. 2, Fig. 3)

Plus one user mocking STEM in general as “[choosing] fascism and “billions must die”” out of greed, and another approving of others’ dunks on STEM over past degree-related grievances.

You want my take on this dunkfest, this suggests STEM’s been hit with a double-whammy here - not only has STEM lost the status their “high-paying” reputation gave them, but that reputation (plus a lotta built-up grievances from mockery of the humanities) has crippled STEM’s ability to garner sympathy for their current predicament.

Well, what’s next, and how much work is it? I didn’t want to be a computing professional. I trained as a jazz pianist. At some point we ought to focus on the real problem: not STEM, not humanities, but business schools and MBA programs.

@BlueMonday1984

Except biology isn’t being hit as badly and that’s also STEM. I wouldn’t be surprised if other life sciences have also done better, at least until Trump started fucking with the grant system.

It’s specifically computer-touchers who are in the toilet.

PS: We also think that there existing a wiki page for the field that one is working in increases one’s credibility to outsiders - i.e. if you tell someone that you’re working in AI Control, and the only pages linked are from LessWrong and Arxiv, this might not be a good look.

Aha so OP is just hoping no one will bother reading the sources listed on the article…

From the comments:

On the contrary, I think that almost all people and institutions that don’t currently have a Wikipedia article should not want one.

Huh. How oddly sensible.

An extreme (and close-to-home) example is documented in TracingWoodgrains’s exposé.of David Gerard’s Wikipedia smear campaign against LessWrong and related topics.

Ah, never mind.

And looks like dgerad is already on the case and the lesswrongers are aware of it.
o7

I’m sorry in advance for posting this meme.

My velocity has increased 10x and I’m shipping features like a cracked ninja now, which is great because my B2B SaaS is still in stealth mode.

Yeah it’s satire, but effective satire means you can never really tell…

Im old enough to recall the polyphasic sleep fad. And how that wrecked people if they ever messed up. (Iirc also turns out very bad implications for long term health).

Wait, but if they lied about that… what else do they lie about?