• 0 Posts
  • 2 Comments
Joined 2 years ago
cake
Cake day: June 22nd, 2023

help-circle
  • Preface: I have a lot of AI skepticism.

    My company is using Cursor and Windsurf, focusing on agent mode (and whatever Windsurf’s equivalent is). It hallucinates real hard with any open ended task, but when you have ALL of:

    • an app with good preexisting test coverage
    • the ability to run relevant tests quickly (who has time to run an 18 hour CI suite locally for a 1 line change?)
    • a well thought out product use case with edge cases

    Then you can tell the agent to write test cases before writing code, and run all relevant tests when making any code changes. What it produces is often fine, but rarely great. If you get clever with setting up rules (that tell it to do all of the above), you can sometimes just drop in a product requirement and have it implement, making only minor recommendations. It’s as if you are pair programming with an idiot savant, emphasis on idiot.

    But whose app is well covered with tests? (Admittedly, AI can help speed up the boilerplating necessary to backfill test cases, so long as someone knows how the app is supposed to work). Whose app is well-modularized such that it’s easy to select only downstream affected tests for any given code change? (If you know what the modules should be, AI can help… But it’s pretty bad at figuring that out itself). And who writes well thought out product use cases nowadays?

    If we were still in the olde waterfall era, with requirements written by business analysts, then maybe this could unlock the fabled 100x gains per developer. Or 10x gains. Or 1.1x gains, most likely.

    But nowadays it’s more common for AI to write the use cases, hallucinate edge cases that aren’t real, and when coupled with the above, patchwork together an app that no one fully understands, and that only sometimes works.

    Edit: if all of that sounds like TDD, which on its own gives devs a speed boost when they actually use it consistently, and you wonder if CEOs will claim that the boosts are attributable to AI when their devs finally start to TDD like they have been told to for decades now, well, I wonder the same thing.