- cross-posted to:
- fuck_ai@lemmy.world
- cross-posted to:
- fuck_ai@lemmy.world
We’re about to learn a painful lesson about delayed gratification in software engineering.
New data from China, 26,811 students tracked January 2023 through June 2025. Students using AI for homework saw their scores jump 20 percent. Completion time dropped nearly half. They aced the assignments.
Then exam season came. Those same students scored 20 to 40 percent worse when they couldn’t use the tool.
The homework phase is over. The exam phase is coming.
We’re doing this in software right now. Vibe coding feels incredible. Features ship fast. Nobody’s asking what happens in Month 18 when the original dev has left and nobody understands the codebase.
Commercial pilots fly with autopilot for most of every flight. They’re required to maintain manual flying proficiency regardless. If the system fails mid-air and the pilot can’t take over, people die.
Most teams using AI right now have forgotten how to fly manually. They’ve become passengers in their own systems. The autopilot flies, nobody checks instruments, and the first sign of trouble will be a breach notice or outage.
Three rules:
- Command the mission. Define architecture before prompting. Ambiguity kills in code and in flight. Delegate selectively. Offload mechanical work. Keep design and security reviews human. Verify everything. Audit before production.
- Never trust the automation without checking instruments.
- Quick wins feel good. Sustainable engineering feels boring. Boring keeps systems standing.
Organisations surviving the next two years won’t ship the fastest. They’ll be the ones who remember how to fly without the aids.
people insisting that you actually be skilled, independently of your tools, doesn’t make them Luddites. Rather, being unable to do so makes you a phony.
Compilers would look the same. You give some people a tool that turns fiddly low-level repetition into relatively painless double-checking. Then you yank that tool away, and oh no, they’re not as good at the fiddly low-level repetition.
What matters is - are these students worse-off, with the tool? It’s not going away. Foreverafter, a couple gigs of linear algebra will turn your to-do list into an actual running project. We have to ask if teaching the fiddly low-level repetition is the point… or just a means to an end.
You can scoff that compilers are objectively better, but they weren’t! Sometimes they still aren’t! We were teaching kids to consider inline assembly well into the Windows Vista era. Even now, compilers with decades of active development can have bizarre corner cases, which any hardened bit-banger of machine code would never write. But they’re all good enough that plaintext assembly has become an arcane third-year topic, and hexadecimal machine code is the realm of bare-metal maniacs.
Teaching math, kids have to know what numbers are, and how base ten works, and why the calculator says 11-3=8. But as you go up to college math it matters a lot less whether people can do shit in their heads… or even remember how. Open-book and open-calculator exams are still fuckin’ hard. They’re just hard because you’re trying to apply some obscure proof to a rat’s-nest of swooping symbols, instead of because you’re worried about having enough time to write out long division to umpteen places. The latter can be sufficiently approximated with two wooden sticks. We can assume anyone doing math professionally has two sticks. Continuously! As a given, for their job. Probably at home as well.
Teaching programming, you have to break people a certain way. No gentle and charming syntax can disguise the mechanized nastiness of what code is. Whatever semi-readable text you’re looking at will be squinched through a flake of tortured sand in a bazillionth of a second. But only after getting transformed at least two times in processes you kinda have to trust will be computationally equivalent, unless you want to minor in a branch of mathematics haunted by Stephen Wolfram. Ultimately there is a shockingly small state machine designed to trick you into off-by-one errors.
But once people can knock out fizzbuzz on a paper 6502… they’ve kinda got it. They’ve been inculcated. They will soon apply arcane patterns to a rat’s-nest of obtuse variable names. Ssshould they have to do that by hand? They’re not doing the assembly by hand. They’re not even looking at the assembly. They might never need to think about assembly again. So long as they’re playing spot-the-bollocks for a flurry of pull requests, that’s arguably a more useful skill than writing clean toy code from scratch. Especially in pen and ink.
The main problem with AI is that it’s pretending to understand more than it actually does right now. It can’t do software architecture. It can’t really write maintainable code without a lot of re-prompting and corrections. And yet it will happily make a bunch of those decisions for you when you ask it to write you an app.
Compilers are well-defined tools. “Human-readable” code goes in, “machine-readable” code comes out, according to some spec somewhere. Yes, compilers can be nondeterministic and they do a lot of weird stuff sometimes and they choose the wrong approach often, so some developers still need to understand assembly and CPU cache and SIMD and other features of the hardware our code runs on. At the very least every developer needs to understand the performance tradeoffs they’re making, or we end up with a JS-based start menu in Win11. But at least compilers have a well-defined class of inputs and outputs.
For AI these issues are amplified a lot. They are non-deterministic and unbounded in what layer of software they operate on or produce. I’d argue every developer who uses AI needs to understand software architecture, approaches to software development, and actually read and understand the code that the AI produced, at a minimum. Otherwise the whole industry will become saturated with and built on top of code no-one understands or can maintain, and it will literally collapse in on itself at some point. Carefully reviewed, selected, maintained code has always been precious, it will become a golden nugget in a sea of pig manure.
Which is a skillset being learned. One which users are presumably better at than non-users, since they’re practicing it. If the chatbot-that-codes Just Worked, there’d be nothing to discuss; we could consider human coding successfully completed. The people who don’t move up to proper computer science can get on with whatever they wanted software for. We only teach them all of this because they’ve got shit to do, and up until now, it’s been really really hard. Like if beginner astronomy meant telescope engineering.
Already the key complaint has become “maintainability.” That’s quite a high-level criticism. When I sass specific compilers, it’s for using register-juggling macros that could have been one hardware instruction… not for adding a feature from a one-sentence description, but in a way that will complicate any future refactor. Or would, if refactoring was not also something you can tell the chatbot to do.
Compiler developers are a hardened core of ultranerds. I couldn’t do it above a toy level. If I needed to understand how they work, in detail, then I would be in trouble. Thankfully it is sufficient to abstract all of their work into ‘C goes in ARM comes out.’ Or at least to mumble about intermediate representations and distract people with Godbolt before running away. Sure, ‘add a button above the widget’ is more pluripotent than a K&R-standard Hello World, but even the foundation you celebrate is layer upon layer of Don’t Worry About It.
AI coding right now is a blunt instrument. Those often suffice. And this one can already be used to remove layers. To obviate a fat framework. To translate to a faster language. To golf. Or it could format your hard drive if a website tells it to, IDFK. It is a higher class of footgun. We take those.