Wow, sounds like you’ve consistently worked only in some really shitty environments, sorry for you. The worst companies I've ever worked for, had no testing.
Seriously, PL is all about TDD, I'm not joking. PP actually does matter because two people sign off on commits that they worked on together and it is quite an achievement to convince your coworker that you're sitting next to, to not write a test. Consider it a sort of checks and balances. PL's version of pair programming is 100% of the time, it works.
I have been running an open source project for a few years now with 40k downloads a month on NPM. This project has a heavy dependency on two other projects. Testing isn’t 100%, but it is enough to rely on it for releases. I also started out with TDD as well. Nothing is perfect, in fact we just had a release yesterday with a contributed fix in it, that came with a test, and still ended up with a bug... but out of 118 tags, a few here and there isn't bad. I routinely upgrade all the dependencies and never hear a peep out of people who depend on it.
Most recently, I wrote some software that ran on 20k servers... it was extremely well tested because if it failed, it could have caused massive amounts of downtime.
I've worked for a number of other companies where we had fantastic testing infrastructure, CI/CD... mostly because we started from day one with that. Bolting it on after the fact, never happens to the degree it should.
I just saw Hashicorp release Hermes today... and was saddened to see it had zero testing in it. I just sighed and closed the browser window. It just isn't worth it.
>
I have been running an open source project for a few years now with 40k downloads a month on NPM. This project has a heavy dependency on two other projects.
Are they dependencies on libraries, or are they dependencies on services?
Library dependencies are 'easy' to test. Service dependencies are very, very difficult. Tractable, but very difficult. That's just one of the difference between a small project, and a large product.
Just because we have vastly more than two dependencies doesn't mean that I'm working in a shitty environment. It just means I'm working on a large problem.
> I've worked for a number of other companies where we had fantastic testing infrastructure, CI/CD... mostly because we started from day one with that. Bolting it on after the fact, never happens to the degree it should.
I'm not sure why you believe that all of these things weren't with us at day 1, either.
Except it happens in real time, with a real person, sitting next to you.
> Are they dependencies on libraries, or are they dependencies on services?
In this case, other libraries... but I honestly don't see the difference. Libraries and services have APIs. They either work or they don't. Services have the additional complexity of network failures and runtime errors, but these are things that can be dealt with and tested in the primary project.
> Just because we have vastly more than two dependencies doesn't mean that I'm working in a shitty environment.
You made the wrong equation there. The environment is shitty because you said that you're in an environment where people have created products so complex that it is very difficult to achieve adequate testing. You also shrugged off Eclipse vs. IDEA... where I'd actually say that matters. You also talk about disabling tests for security... that seems absurd.
> It's entirely feasible, and highly desirable to get a small project to the point where just running the tests makes you feel 100% confident in the correctness of your changes.
I've built multiple companies that have achieved significant revenue on that confidence. Not small projects. I know it is possible to do.
> I'm not sure why you believe that all of these things weren't with us at day 1, either.
So you started off good and ended poorly? Another reason to believe that the environment is shitty.
Testing library and service dependencies is similar in theory. The difference between theory and practice is that in theory, there isn't one.
I assure you, I am a real person, generally sitting adjacent to people whose code I'm reviewing, or who review my code.
The projects are complex because the business needs are complex. This criticism is the 'I could build Twitter in a weekend, why does it have five thousand people working on it, anyways?'
I, like most people, have plenty of opinionated preferences on the subject of Intellij and Eclipse, but I'm not going to drive any broad conclusions regarding their impact on a product's test coverage. And I will postulate that most attempts to do so reduce down to cargo-culting. It was an example of a largely irrelevant technical decision, and I think it's telling that you are latching onto it.
And it's not disabling tests for security, it's disabling security for tests, as a distressingly common example of a routinely undertested interaction between two complicated spaces.
> Testing library and service dependencies is similar in theory.
Not really. The theories around testing are all well established and trodden at this point. People who've been around long enough, have this experience.
> The projects are complex because the business needs are complex.
Complex business needs are separate from the functional testing of individual components. Testing business logic is well... a matter of writing a test, seeing it fail and then writing the business logic (however complex), to make that test pass. Books and PhD's have been written about TDD. The stuff isn't rocket science.
> generally sitting adjacent to people whose code I'm reviewing
Pivotal Pair programming is literally sitting next to the person, sharing a single computer, two mirrored monitors, two keyboards, two mice. Two people independently working together to control a single computer. 8 hours a day (with breaks, of course). Is that what you do?
> I think it's telling that you are latching onto it.
I think it's telling that you dismiss it. Tools are important. It is like telling a car mechanic that every wrench is the same and unimportant. There is no debate on this, IDEA 'eclipsed' Eclipse, especially for Java development (and honestly, everything else), over a decade ago. The final straw for me was when Eclipse crashed for the 1000th time and torched my workspace to the point that I had to delete Eclipse to get it running again.
> it's disabling security for tests
Which says to me that the 'frameworks' used for testing aren't set up well or that the developers you're working with don't have an understanding of or care for how to do that. No thanks. Get it right from the start and make it friendly to the people you're working with so that they aren't so annoyed they have to disable tests. It also says to me that the way things are coded are so intertwined with the security layer that they can't be easily tested independently... which is also a fundamental design issue. Obviously, there are exceptions, but the underlying issue still exists if developers are disabling tests because of this.
> In this case, other libraries... but I honestly don't see the difference. Libraries and services have APIs. They either work or they don't. Services have the additional complexity of network failures and runtime errors, but these are things that can be dealt with and tested in the primary project.
In my personal experience, there are considerable differences.
From the top of my head:
1. Typically, your libraries don't shift under you without any action from your part. External services do.
2. In CI, you can either use the external service (which may require piercing holes in your CI's security and may break due to no change of your part), replicate it (which is more stable but can be extremely painful and resource-consuming) or mock it (which is even more stable but limits the realism of your tests).
3. And of course, as you mention, services have all sorts of failure modes, including but not limited to network failures. Normally, your tests should cover all these failure modes. I don't think I've ever seen code that does that, in part because service errors are very often notoriously under-documented.
> 1. Typically, your libraries don't shift under you without any action from your part. External services do.
External services should have versioning, just like libraries.
> 2. ...
They don't need to be 'or'.
> 3. ...
I write tests for this. If I'm using a networking library/function (let's use `fetch` as an example) to talk to a 3rd party external service, I write a wrapper around it to deal with error cases. I then write tests for that wrapper. Anything that uses that wrapper is then inherently tested at the networking level. I then write tests for my own code which uses the wrapper, to deal with the thrown exceptions and how that code should deal with them (retries, throw again, etc).
You don't do that? Do you just assume that all calls to `fetch` succeed?
Seriously, PL is all about TDD, I'm not joking. PP actually does matter because two people sign off on commits that they worked on together and it is quite an achievement to convince your coworker that you're sitting next to, to not write a test. Consider it a sort of checks and balances. PL's version of pair programming is 100% of the time, it works.
I have been running an open source project for a few years now with 40k downloads a month on NPM. This project has a heavy dependency on two other projects. Testing isn’t 100%, but it is enough to rely on it for releases. I also started out with TDD as well. Nothing is perfect, in fact we just had a release yesterday with a contributed fix in it, that came with a test, and still ended up with a bug... but out of 118 tags, a few here and there isn't bad. I routinely upgrade all the dependencies and never hear a peep out of people who depend on it.
Most recently, I wrote some software that ran on 20k servers... it was extremely well tested because if it failed, it could have caused massive amounts of downtime.
I've worked for a number of other companies where we had fantastic testing infrastructure, CI/CD... mostly because we started from day one with that. Bolting it on after the fact, never happens to the degree it should.
I just saw Hashicorp release Hermes today... and was saddened to see it had zero testing in it. I just sighed and closed the browser window. It just isn't worth it.