How do you test edge cases?

You think about the implementation and how it can fail. If you don’t think about the implementation, or don’t understand the implementation, I would argue that you can earnestly try to test, but you won’t do a good job of it.

The issue of LLMs here is the proliferation of people not understanding the code they produce.

Having agents or LLMs review and understand and test code may be the future, but right now they’re quite bad at it, and that means that the parent comment is spot on; what I see right now is people producing AI content and pushing the burden of verification and understanding to other people.

> The issue of LLMs here is the proliferation of people not understanding the code they produce.

Let's ignore the code quality or code understanding: these juniors are opening PRs, according to the previous user, that simply do not meet the acceptance criteria for some desired behavior of the system.

This is a process, not tools issue.

I too have AI-native juniors (they learned to code along copilot or cursor or chatgpt) and they would never ever dare opening a PR that doesn't work or does not meet the requirements. They may miss some edge case? Sure, so do I. That's acceptable.

If OP's are, they have not been taught that they have to ask for feedback when their version of the system does what it needs to.

> pushing the burden of verification and understanding to other people.

Where was the burden prior to LLM's?

if a junior cannot prove his/her code as working and have an understanding, how was this "solved" before llm? Why can't the same methods work post-llm? Is it due to volume? If a junior produces _more_ code they don't understand, it doesn't give them the right to just skip PR/review and testing etc.

If they do, where's upper management's role here then? The senior should be bringing up this problem and work out a better process and get management buy-in.

> Why can't the same methods work post-llm?

>> If you don’t think about the implementation, or don’t understand the implementation, I would argue that you can earnestly try to test, but you won’t do a good job of it.

Previously the producers of the code were competent to test it independently.

This increasingly, to my personal observation, appears to no longer be the case.

They do test it, they just dont think about it deeply and so they do a shit job of testing it, and an incompetent job of writing tests for it.

Not by being lazy; smart diligent folk doing a bad job because they didn't actually understand what needed to be tested, and tested some irrelevant trivial happy path based on the requirements not the implementation.

Thats what LLMs give you.

Its not a process issue; its people earnestly thinking they've done a good job when they havent.

Testing is often very subtle. If you don't understand changes you made (or really didn't make because the LLM did them for you), you don't know how they can subtly break other functionality that also depends on it. Even before LLM's, this was a problem for juniors, as they would change some code, it would build, it would work on their feature, but it would break something else which was seemingly unrelated. Only if you understand what your code changes actually "touch", you know what to (manually or automatically) test.

This is of course especially significant in codebases that do not have strict typing (or any typing at all).

Writing a test requires you to actually know what you're trying to build, and understanding that often requires the slow cooking of a problem that an LLM robs from you. I think this is less of a problem when you've already been thinking deeply on the domain / codebase for a long time. Not true for interns, new hires, interns.

I agree, normally the process (especially of manual testing) is a cultural thing and something you instill into new devs when you get broken PRs - "please run the tests before submitting for review", or "please run the script in staging, here's the error I got: ...".

Catching this is my job, but it becomes harder if the PR actually has passing tests and just "looks" good. I'm sure we'll develop the culture around LLMs to make sure to teach new developers how to think, but since I learned coding in a pre-LLM world, perhaps I take a lot of things for granted. I always want to understand what my code does, for example - that never seemed optional before - but now it seems to get you much further than just copy-pasting stuff from Stack Overflow ever did.

Sometimes, orgs don!t mandate testing or descriptive PRs, and then you requiring it makes you look like a PITA.

PITA or senior developer that's too senior for that company? Honestly I think an organization has no say in discussions about testing or descriptive PRs, and on the other side, a decent developer does not defer to someone higher-up to decide on the quality of their work.

Some managers will do anything for velocity, even if the direction is towards a cliff with with sharp rocks below. You try to produce quality work and others are doing tornado programming all over the codebase and be praised for it.

The industry does not care what engineers think because engineers have no power at all. We will not be able to hold any line against a management class that has been assured that AI tooling doubles engineer productivity and have zero ability to judge that claim, because they've never built anything and run into the exact problems that AI tools cause; A lack of understanding. They don't even know what they don't know.

If you wanted software engineers to be able to hold any sort of quality line against a few trillion dollars worth of AI investment, we needed to unionize or even form a guild twenty years ago.

LLMs amplify the problem, so they are not that irrelevant.