> How costly can it be to test the file fully in a CI job?

It didn't need a CI job. It just needed one person to actually boot and run a Windows instance with the Crowdstrike software installed: a smoke test.

TFA is mostly an irrelevent discourse on the product architecture, stuffed with proprietary Crowdstrike jargon, with about a couple of paragraphs dedicated to the actual problem; and they don't mention the non-existence of a smoke test.

To me, TFA is not a signal that Crowdstrike has a plan to remediate the problem, yet.

They mentioned they do dogfooding. Wonder why it did not work for this update.

They discuss dogfooding “Sensor Content”, which isn’t “Rapid Response Content”.

Overall the way this is written up suggests some cultural problems.

You just got tricked by this dishonest article. The whole section that mentions dogfooding is only about actual updates to the kernel driver. This was not a kernel driver update, the entire section is irrelevant.

This was a "content file", and the first time it was interpreted by the kernel driver was when it was pushed to customer production systems worldwide. There was no testing of any sort.

All these people claiming they didn’t have canaries. They actually did but people are in denial that they are the canary for crowdstrike lol

It's worse than that -- if your strategy actually was to use the customer fleet as QA and monitoring, then it probably wouldn't take you an hour and a half to notice that the fleet was exploding and withdraw the update, as it did here. There was simply no QA anywhere.

Just reeks of incompetence. Do they not have e2e smoketests of this stuff?