> Making software debuggable, maintainable, layered, and composable – that’s still quite a trick. Quite a lot of that work requires extensive, thoughtful reasoning. And that’s where the LLM’s today, even the leading edge of the “capability” from frontier models, fall short.

It’s been my quest during my career to figure out what is maintainable software, what is composable or not, and how the two things, and many other things, are in direct conflict. There is no single answer. If your goal is to take over the market quick, as many here would like, maintainability is a very low priority aspect of your code base. Composability may matter for integrators but can be entirely ignored in your CRUD backend. Beyond that, I don’t know of a good way to measure most of these intangible properties. Highly competent software developers disagree in even basic things, like whether OOP is a good idea, should we all be using pure functional programming etc. Hence, how would you expect an LLM to get good at figuring this out for you? If you describe exactly what trade offs you are willing to make, and give it ways to measure how well it’s doing, then I do think LLMs will be able to not fall short. Given the current state of things, it’s just a matter of opinion whether LLMs fall short, or humans fall short for that matter.

AI generated code is like IKEA furniture.

IKEA furniture embodies many elements of good cabinet making but skips many nonessential elements. And does this more consistently than cabinet makers who can be bored, incompetent, depressed, burnt out, resentful, tired, having a bad day.

In the future AI code inevitably will embody most good software engineering practices. And will do this more consistently than software engineers who can be bored, incompetent, depressed, burnt out, resentful, tired, having a bad day.

Just look at the messages on HN or around you at your colleagues to see how mediocre the average software engineer is..

Today's IKEA is good enough for most people.

Tomorrow's AI coding will be good enough for most corporations.

Good enough to vastly reduce the need for fine craftsmen and women / software engineers.

Good enough to deskill those who call themselves cabinet makers / senior software engineers. These days the cabinet makers I personally know just do contract kitchens for project builders.

But IKEA is and AI will be, bad enough that at the high end with special requirements / taste / money / an inflated sense of self worth, some furniture makers still exist and thrive.

Perhaps 1% percent of current software engineers of today will be needed in the future when AI code inevitably has the ability to follow good software engineering practice.......

And as usual it will mainly be the mediocrities that remain ( so there is hope for you too ), with occasional islands of excellence.

• quietbritishjim

  today at 8:40 AM

  The problem with this analogy (actually one of many) is that most people can make do with a cabinet that is literally identical to everyone else's. IKEA is great for that.

  If you want software that is literally identical to what someone else is using then you don't need AI. You need a license to that software! That is just the traditional software model.

  AI gives software that is bespoke with hundreds of decisions made, hidden from you, in the background. If it's a throwaway script, that's fine (and I don't mean to undersell this - this is a huge application). If you want a larger program that is going to form part of your business process then it will need at least some level of supervision from an actual expert.

• amelius

  today at 8:22 AM

  Having not been formally verified, almost all software today feels cheap. Maybe an AI can change that at some point.

  • andai

    today at 8:48 AM

    By Dijkstra's standards, we've been vibe coding for an entire century!

    I had similar thoughts recently, that now that machines are good at writing proofs, this could help with their reliability in software development.

    Then I had a funny incident where an LLM implemented a feature completely backwards. Plenty of tests were supplied which demonstrated that the completely broken feature was correctly implemented.

    I realized that formal verification would not have helped here, if I had left the task to the machine. It would simply have written a mathematical proof of the correctness of the incorrect feature!

    Apparently this is an issue for humans as well, called the "spec gap" or something like that.

  • ChrisGreenHeur

    today at 8:25 AM

    Yay let’s lock ourselves into the formal verification toolsets, so that we can never use new language features again.

    • Thanemate

      today at 8:35 AM

      OR new languages and frameworks ever again, since they'll slow down code generation due to lack of training, and from LLM generation standpoint this is a terrible thing to trade off.

• andai

  today at 8:47 AM

  Ikea cabinet is pleasant to look at, and its design makes sense.

• hugodan

  today at 8:20 AM

  ...ah the classic middle manager analogy of code is X, where X is nothing like code at all but is being used to drive a point that is just standing on poor grounds.

  keep it, we have been through "is like building a house", "like following a recipe", "like a living organism", "like $SOMETHING_WITH_COMPONENTS", etc...

  we can handle your IKEA furniture, thanks you for your contribution

With generated code, the directory structure, interface design and general state management is usually a haphazard mess. Even with the best frontier models. But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt. Subtle things like which error states are "oh shit we need to bail" vs "this isn't a deal breaker." Sometimes it will ask, but more often than not it will just make a decision and it's often the wrong one. If I don't have a fully kitted out test suit and a good type checker to verify the final product against, the the whole looping thing is just useless to me and I'm back to reviewing every line of code it puts out and having to draw on my years of architecture experience to make sure we don't build a giant pile of trash.

• Gigachad

  today at 5:15 AM

  Because they are designed to be used by managers who don't know how to answer these questions and don't want to be asked them. Just have the magic answers box pick something.

• Alien1Being

  today at 8:10 AM

  Surely this is a solvable problem.

  If the average, mediocre software developer can address the issue of directory structure, interface design, general state management, edge cases and subtle assumptions it should be possible to train AI systems to address these issues .

  Software development is not some mystical magical activity.

  I remember people making similar arguments about autonomous driving...

• aryehof

  today at 8:08 AM

  > But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt.

  This is a problem with your instructions, your specification. An LLM isn't a mind reader. It will attempt to succeed regardless of missing requirements and ambiguity.

• bluegatty

  today at 6:14 AM

  The generated code is fine at the functional level, the directory structure is usually the standard pattern for the given type of project.

  The error types and codes, it will produce to spec.

  If you type 'make me that thingy' - yes, it's probably not going to do what you want, but if you give it spec and guidance, it usually will.

  The 'interface design' ... not very good though.

  • szundi

    today at 7:43 AM

    [dead]

• slopinthebag

  today at 5:40 AM

  They're RLHF'ed to an inch of their lives to be able to one-shot complete tasks, since requiring human input defeats the purpose of being able to replace the labor force.

  But once the insanity ends LLMs will be packaged as tools for developers to use to boost their productivity, and we'll consider them as we do IDE's and debuggers and stuff. But we have to get through this hype cycle first.

  • siva7

    today at 7:39 AM

    wake up, slopinthebag. wake up..

• chrisjj

  today at 8:20 AM

  > more often than not it will just make a decision and it's often the wrong one.

  Let's not forget these chatbots rely on a random number generator to pick output options.

• yoz-y

  today at 8:04 AM

  It absolutely hates code that would crash or error in any circumstance. So it adds a ton of dubious fallbacks.

  • disgruntledphd2

    today at 8:56 AM

    Crash means zero reward, so yeah.

> It helps to know that LLMs don’t “reason”. They predict ..

Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.

• TheWrongGuy

  today at 7:50 AM

  By that standard, human brains don't either. Our externalizations of concepts like language or symbolic structure allow us to do so. In the parlance of our times, we built our own reasoning harnesses because our intuition lead us to do so.

• jayd16

  today at 5:46 AM

  Even if that was true, you'd have to still prove it has emerged.

  • krackers

    today at 6:37 AM

    What would be your test to determine that?

• complex_pi

  today at 7:05 AM

  Maybe it looks like reasoning, and maybe that's enough for some.

• slopinthebag

  today at 5:47 AM

  Why would "reasoning" be an emergent property of prediction?

  • mw888

    today at 6:20 AM

    Predict multiple outcomes, induct across them, refine.

  • js8

    today at 6:22 AM

    There's a lot of reasoning in the training data.

    • today at 7:14 AM

  • hbcdbff

    today at 7:36 AM

    Why wouldn’t it?

  • hsn915

    today at 6:20 AM

    How do you predict without reasoning?

    • chrisjj

      today at 8:23 AM

      Flip a coin.

    • slopinthebag

      today at 6:46 AM

      Where is the reasoning in linear regression?

      • danielbln

        today at 7:00 AM

        Where is the reasoning in synaptic transmission?

        • necovek

          today at 7:34 AM

          There isn't, which is exactly the point: we do not yet understand the fundamentals behind reasoning.

          • anon48293

            today at 7:47 AM

            Don’t we? We can build something which has all the output associated with reasoning.

            I’d say we’ve figured out the fundamentals behind reasoning.

            • chrisjj

              today at 8:24 AM

              > We can build something which has all the output associated with reasoning.

              Sure. A photocopier fed with a maths paper.

But do the same SWE fundamentals apply if the one doing the programming is many times smarter than us?

The article says what many here like to hear, but in my opinion the core arguments are false.

> Making software debuggable, maintainable, layered, and composable – that’s still quite a trick

Not really. I have been working on a mobile app for months, and I stopped even glancing at the code about two months ago.

150k LOC, around half of that in tests, and the AI still has no problem maintaining the code on my behalf.

Debuggable? It can add extensive instrumentation in seconds.

None of this requires expertise, prompting, or mention of TDD. It's the default.

Frankly I do not believe the author tried developing a large codebase fully agentic and without reviewing the code. I believe many here look at the code produced, deem it substandard, and go hands on.

> They’re foundationally incapable of always and consistently preventing prompt injection attacks

From Anthropic's article about the Auto mode:

> We commissioned an evaluation from a third party, Trajectory Labs, who tested different models within the latest publicly available versions of Claude Code and Codex as of July 17th 2026.1 They tested 72 indirect prompt injection scenarios held out from Anthropic

> In this evaluation, none of the 720 attack attempts succeeded against Claude Fable 5, Opus 5, or Sonnet 5 running auto mode. On the other hand, 5.83% of the attacks succeeded against GPT-5.6 Sol running Codex's Auto-review mode. Notably, this is greater than the 0.09% average attack success rate against our latest models running in bypassPermissions mode without additional safeguards. The tests showed a 19.03% attack success rate against GPT-5.6 Sol when running in Full Access mode

I'm sure someone is going to reply with how they do not trust Antrophic's research, but lacking other data, prompt injection appears to be largely solved already.

• hbcdbff

  today at 7:35 AM

  How do you expect us to take your views on LLM code quality and durability seriously when a) you don’t even look at the code and b) you’ve only been doing this for two months?

  • user43928

    today at 7:52 AM

    I've been working on the app for four months, and I am clearly not talking about code quality.

    I am talking about product quality and maintainability. Both are more than adequate.

    I know this because I have worked on it for an estimated 300 hours. Has the author practiced a similar approach for even a week? I doubt it.

    • Krei-se

      today at 8:29 AM

      I work on my project for 2 years now and using an LLM always came back to bite me. Learning how something works is needed, slow and painful - but pain is gain.

      If this works for you - awesome. Until it doesn't.

      As always there is 0 code or link. All talk.

      • user43928

        today at 8:37 AM

        And when do you expect my approach will stop to work? The core features are complete and the codebase is already sizable.

        I will not publish my app on GitHub for free. It's a paid app, and I am putting in the hours not for your approval, but for commercial gain.

        I also do not think it wise to link my HN account to my real name and expose my opinions and comments to my employer and colleagues.

        • skydhash

          today at 9:01 AM

          Then you may as well said you've solved P=NP.

          We do not require links to your app. What people are expecting is a description of your approach and sample outputs. So that someone else can try it and have the same standard of output. That's how you make a point that your approach is good.

          When we buy books like "The Practice of Programming" or "The Pragmatic Engineer", it's because we are hoping to learn useful and productive behaviors. It isn't to hear boasts about how good the authors are good at using tools.

• andai

  today at 8:55 AM

  I tried this recently and the results were total banana cakes. They couldn't even make changes to Pong without breaking it.

  • user43928

    today at 9:01 AM

    I did not try it on games.

    Many features in my iOS app at first come out technically working, but with poor UX and verbose text in the UI.

    One or two rounds of testing and refinement and they typically work well.

• shakna

  today at 8:00 AM

  Prompt injection. Solved.

  But accidentally breaking systems is not an issue either, obviously. Even though the system prompt asks for safety rails, and other prompts wouldn't accidentally violate that.

  https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gy...

  • user43928

    today at 8:25 AM

    Alignment of the latest models is questionable, yes. That's a different topic.

    For this particular gym incident, supposedly Opus 4.6 was used in OpenClaw, predating the current safety guardrails of Fable and co.

LLM is the new Excel

"They’re foundationally incapable of always and consistently preventing prompt injection attacks. “Alignment work”, safety harnesses, and sandboxes all help to add barriers against the worst, but there are fundamental gap" ...

They seem to be very good at a lot of rudimentary best practices, more so than humans, but more accurately - if you run and audit pass with specific instructions ... they're very good at that.

I mean - it's what they're the best at which is applying 'fuzzy heuristics' in a mechanical way. If can describe issues concisely, the patterns, the styles, the rules then LLMs can very mechanistically and methodologically grind through them.

I don't even see how this is controversial - without getting into 'what their reasoning means' - we can all agree that their synthetic reasoning is pretty good at narrow scales, and they've been 'trained by compilers' and are extremely good at spotting common patterns.

If you back that up with a lot of tokens ... they excel.

Designing architecture, that's difficult, but hammering away at all the 'known-knows across a system' especially to identify things ... they're pretty good at that.

> In the past year, agent harnesses crossed the “can it be done” rubicon.

Brother, I'm still in "Can you get it right?"-mode. What am I doing wrong? (Rhetorical, but advice welcomed).

• simonw

  today at 5:44 AM

  Tell it to use red/green TDD and start things off with an already configured test suite, maybe with a single test that asserts 1+1==2.

  Make sure it know how to run the tests before it starts writing any additional code.

  Then set it a clear goal.

  • slopinthebag

    today at 5:51 AM

    Basically all the examples of LLM's building impressive things have been because they have human written tests to base the implementation on. If you have an LLM write the tests the results are far less impressive or valuable.

    • bharatsuthar

      today at 6:18 AM

      Yes and LLMs are known to cheat on tests written by them.

      • andai

        today at 8:58 AM

        Last year when they added computer use to Claude web I was excited to try it out.

        I just asked it for a code snippet and it ended up setting up a whole repo in a docker container or something.

        Even volunteered a test suite. This genuinely amazed me.

        ...until I checked the tests.

        It was just console.log("Tests passed!")

        AGI 2027

      • slopinthebag

        today at 6:50 AM

        It's not always cheating either. They aren't intelligent, so they don't actually understand the purpose of the tests or can build them to define the actual semantics of the problem space. It's literally just next-token prediction based on the codebase and prompt. Cheating implies that they have agency, and ironically agents don't.

• al_borland

  today at 4:55 AM

  I’ve found some success is small projects, with limited scope, in a greenfield.

  I’m terrified to attempt agentic anything in the repo my job actually cares about. I triggered it once by accident, when the agent was first rolled out and enabled by default… it broke everything. Now I just use ask mode, and even that is wrong half the time, and once it goes wrong it just keeps getting worse.

  I saw a post from Dave Plumber who vibe coded up a new cross platform task manager. He said his spec document for the AI was 107 pages long. So maybe what I’m doing wrong is not giving the AI a literal novel of spec.

  • applfanboysbgon

    today at 6:09 AM

    > He said his spec document for the AI was 107 pages long.

    This sounds like programming but with extra steps that make it take longer with less reliability.

    • 0x696C6961

      today at 6:35 AM

      Ikr, at that point the code itself is a better way of encoding the information.

• dosisking

  today at 7:16 AM

  There are two 'camps' with respect to AI.

  One camp already knows that Neural Nets don't work and are a dead end.

  The other camp hasn't yet figured out that Neural Nets don't work, but are convinced that they do (or eventually will), because they think everything always improves over time in a linear fashion.

• mw888

  today at 6:23 AM

  You're appealing to ambiguity. All you've said is you have failed—how is anyone supposed to know what went wrong?

• jaggederest

  today at 6:59 AM

  I'd be happy to screenshare with you if you like, we can work on something trivial or open source. Half an hour should be more than enough to see whether you're doing anything obviously self-sabotaging.

• MattGaiser

  today at 4:47 AM

  What is “it” specifically and what languages are you using?