A little off topic, but I had to refresh my memory as to what Garuda was. As a long time user of arch in various "forms" (initially manjaro until I grew frustrated with all the extra/different things, then antergos because it was mostly just plain arch with a nice installer/sane default packages, then endeavouros because it was the closest thing to what antergos provided me), I'm not sure how I missed garuda. Some of their utilities look convenient.
Thanks for the mention! I'll have to try them again on my next machine.
Yet if the business model / customer's _existing_ service agreement is changed, the temperature of the water that the frog is in just went up a little bit, so folks continue using it, which is what often happens as well.
"well, I'm not sure if they're going to start collecting or using my data, because I don't actually really KNOW that or the extent of everything, just an email from them with a vague update to an equally vague privacy policy that I apparently implicitly agree to if I don't discontinue using their service."
Just like a manufacturer/seller on say, amazon shouldn't be able to revise their product with cheaper quality under the same model number (and yet it happens all the time), changes to the agreement of a service should be treated as a new service.
Whatever the solution, it should be a big enough deal that it cannot be implicitly agreed to, and clear enough language (maybe vetted by a third party review of the agreement) to communicate to all users, what is at stake and how, to which third parties, etc.
As a toddler in the early 80s, I playfully stole my older sister's pencil when she was working on her homework, ran away with it, tripped and fell, the lead breaking off in my eye. Fortunately I don't remember any of that myself...
Almost 40 years later it's still there, still scaring any new eye care establishment personnel with each visit.
Fortunately no noticable side effects, other than when I get something in my eye and back out again, my brain keeps telling me something is there for an hour or two afterward.
it took me two weeks to get the sealed pinetime working back when it launched. for some reason it would get stuck on the initial boot screen, and it would be in a state to where I couldn't update the firmware. eventually in a mindless "flap/spin it in a circle with my hand while I was thinking about work" I felt it vibrate and to my surprise, it booted for the first time ever.
It turned out, even with a firmware update months later applied to it, that whenever it gets rebooted, I have to do the above movement until it finally decides to boot, and it's random. Sometimes it takes a few spins/flicks, sometimes I'm doing it for minutes on end.
I confirmed this isn't a red herring by leaving it in its frozen state and motionless, and it would never recover, even 24hr later.
This was the second Pine purchase of mine, the first being the PinePhone Pro, which A) I couldn't get activated on AT&T (not Pine's fault with the 3g phase out and whitelist approach, but I digress), and when I used TMobile, I couldn't make a call with it without having audio problems or etc.
Look, I wanted to support them, and the idea of less expensive open hardware/software, even if it meant rough edges, but completely non functional for even the most basic usages... I might as well just build my own and understand all of the reasons why something doesn't work instead of spending days or months getting it to do what it said it would do on the tin out of the box.
That tracks well (both the quotes and your thoughts).
One example that comes to mind where I want to roll my own thing (and am in the process of doing so) is replacing our ci/cd usage of jenkins that is solely for running qa automation tests against PR's on github. Jenkins does way way more than we need. We just need github PR interaction/webhook, secure credentials management, and spawning ecs tasks on aws...
Every time I force myself to update our jenkins instance, I buckle up because there is probably some random plugin, or jenkins agent thing, or ... SOMETHING that will break and require me to spend time tracking down what broke and why. 100% surface area for issues, whilst we use <5% of what Jenkins actually provides.
The other issue I would point out is that building a database, while impressive with their quality, is still fundamentally different than an application or set of applications like a larger SaaS offering would involve (api, web, mobile, etc). Like the difference between API and UI test strategies, where API has much more clearly defined and standardized inputs and outputs.
To be clear, I am not saying that you can't define all inputs and outputs of a "complete SaaS product offering stack", because you likely could, though if it's already been built by someone that doesn't have these things in mind, then it's a different problem space to find bugs.
As someone who has spent the last 15 years championing quality strategy for companies and training folks of varying roles on how to properly assess risk, it does indeed feel like this has a more narrow scope of "bug" as a definition, in the sort of way that a developer could try to claim that robust unit tests would catch "any" bugs, or even most of them. The types of risk to a software's quality have larger surface areas than at that level.
There's a lot of assertions that I throw into business applications that would be very useful to test in this way. So I don't think this only applies to testing databases.
Also, when properties are difficult to think of, that often means that a model of the behavior might be more appropriate to test against, e.g. https://concerningquality.com/model-based-testing/. It would take a bit of design work to get this to play nicely with the Antithesis approach, but it's definitely doable.
Just to clarify, I am definitely not saying this is only useful or only applies to databases.
The point was more that, I don't see how this testing approach (at the level that it functions) would catch all of the bugs that I have seen in my career, and so to say "all of the bugs" or even "most of the bugs" is definitely a stretch.
This is certainly useful, just like unit tests, assertions, etc are all very useful. It's just not the whole picture of "bugs".
Yes, there are plenty of non-functional logic bugs, e.g. performance issues. I think this starts to drastically hone in on the set of "all" bugs though, especially by doing things like network fault injection by default. This will trigger complex interactions between dependencies that are likely almost never tested.
They should clarify that this is focused on functional logic bugs though, I agree with that.
I’d go so far as to say it is actually impossible to do UI testing in some kind of web based product unless it came from the browser makers themselves.
When outsourced, you either A) rely on someone in your org to tell them what to test and what the workflows are, ie use them as a warm body/monkey to click on things for you - this is what most people see QA as, which is silly - or B) you rely on the outsourced QA to know your product and know what is important or what all of the edge cases are.
If your product is non-trivial in size or scope, ie it is not a cookie-cutter solution, then the testing of your product will also be non-trivial if you want it to work and have a good reputation (including during those all-important live demos, poc's, etc).
QA does not mean "click on things and go through the happy path and everything is fine" - not saying you are implying that, but gosh the amount of companies that think it's child's play is crazy.
Are there many products that reach such sizes without achieving Product Market Fit (PMF)? I feel like after this step is achieved, QA becomes pivotal and involves a great combination of manual and automated procedures. So I agree with you in this regard.
But going back to my initial assumption. I think starting a fresh company without PMF and spending a lot on QA until that is achieved, might not be the best approach.
Doing every quality activity "after the fact" I agree is the issue. That's the root of the problem you're seeing, not that there was a separate quality team.
It’s not the “separate” part that I think is ridiculous. It’s the fact that the team is named “quality assurance.” It relies on a metaphor from
manufacturing that’s entirely inappropriate for software.
If you want to call it “Testing and Exploration” you’d get no argument from me. (Though I do think you’ll find that team is hard to staff.)
There are so so many instances where "rolling back" is just not a feasible solution. Working for a SaaS company with mobile/web/api and huge db's, migrations, payroll uses in the product, rolling back is and should always be a LAST RESORT. In 99% of the cases, something significant enough to want to roll back usually results in a "hot patch" workflow instead because rolling back or etc has its own risk.
> QA has always been about risk management.
100%.
QA should be related to identifying risk, likelihood of failure, impact of failure to user, client and company. The earlier this is done in the varying processes, the better. ("shift left" but I've seen a ton of differences with how people describe this, but generally QA should start getting involved in the "design phase")
Another example from my own first-hand experience:
A company I worked for made a product that plugged into machines that were manufacturing parts, and based on your parameters it would tell you whether or not the part was "good" or "bad".
When interviewing the leadership of the company, as well as the majority of the engineering group, "what is the biggest risk with this product" they all said "if the product locks up!". Upon further discussion, I pulled out a much larger, insidious risk; "what if our product tells the client that the part is 'good' when it is not?"
In this example, the part could be involved in a medical device that keeps someone alive.
I had this happen to me as well, though I remember that it was either a docker or snap/flatpak/etc version of it. Got things working but lost a few months of data, likely something I foolishly did while trying to make things work. Stopped using it and went with Seafile for a while - grew frustrated with that and stopped syncing files between computers altogether.
Later on, when I set up an old dell xeon workstation as a home server (using proxmox), I used the turnkey image of nextcloud, and have (knock on wood) not had any issues at all.
Anyway - I wanted to ensure it was fully virtualized this next time if/when this happens again. I have it backing up the base vm once a week (I have the file storage separate/outside of the vm). So... hopefully this doesn't happen again to the degree that it did.
Thanks for the mention! I'll have to try them again on my next machine.