That's sort of obvious. How do you know that wasn't attempted?

Even if it was fixed, in a probabilistic system like this, isn't it basically guaranteed to happen with some inputs?

Is that a real question? Of course it will happen. In this particular case there was a single misclassified video reported in the article.

Yes it's a real question, since there's nothing that says that a particular misclassification must happen. Watching cars go by on the road, one might suspect that at least one is driven by an alligator, but nothing says that it must be, per se, even the law of large numbers.

Yes they did, I said that. But it was a claim made as a question, because I didn't know whether it was actually true. I still can't demonstrate formally why this would be so, because again, the reasoning and even veracity of the claim is still in question due to lack of anything but a hand-waved answer.

There is no need to formally demonstrate. The veracity is clearly not in question. It must be true, due to the existence of the article we are commenting on now.

If you want to argue for the general case, you can simply prove the negation is false. Since it is incorrect to say that a network trained with a tiny percentage of possible inputs will never misclassify, it is true that a network trained in such a way will eventually misclassify. This is bolstered by training any network and seeing they always will misclassify something.

> Yes they did, I said that.

You didn't say that. You said a misclassification would happen on some inputs. That is different from saying on these specific inputs.

We don't have enough information to root cause the problem

No, that's not correct.

That makes it sound even worse that they knowingly released it without fixing it.

Are you saying an ML system should never be released if it doesn’t have perfect accuracy?

Tagging black people as monkeys is not a showstopper bug? If so it makes them look even worse than if it was an overlooked bug.

Quite honestly, in a team that's stressed and run down to the line, checking for that particular classification in a model that has hundreds of classification targets can be really tricky.

Say you have 1000 classification targets. You have to produce a model that checks, for each target, the odds of it being classified as one of other 999.

You have to check, specifically, for "adult male as primate" out of a million potential combinations. And apply secondary business rules or optimizations to prevent that classifications.

So yes it's possible, but it's not cheap, simple or easy.

Facebook just decided to shove the model out the door and not worry about the consequences.

Quality engineering work, costs money and time. Facebook didn't spend it.

I agree.

It does seem worse that way.

We don't actually know how to do that, or how rich is "rich enough." It's an open avenue of research to be able to extrapolate how well-tuned a neutral net is on data not in its training set.

Not to imply the problem is unsolvable, just that if an institution has zero tolerance for this mistake, the fix your describing is no guarantee it won't occur.

That’s not quite complete, right? It’s that we don’t know how to do that without sacrificing other things.

To be fair, I check a number of different forecast sites (including Windy) and follow a lot of different weather bloggers but no one knew this storm was a thing until a few days ago. And all of the models had it as a Cat 2 until yesterday.

Moral hazard has entered the chat.

Turbine failures, lack of redundancy, 100 year old equipment with no supply chain.

I live in a part of town affected by one of the pump outages. In any other city in the country this would be shocking and whatnot. Here it’s an opportunity to lampoon the local government because that’s all we have.

It's also just false outrage generated by media that wants to shock readers.

The reality is that, from the article...

>Other nearby stations are expected to have enough capacity to pick up the slack

At the individual pump level...

> 96 out of the S&WB’s 99 main drainage pumps are ready for Ida.

I’ve found that a lot of my co workers have stated to do this. It’s turned into a disconnected, tech siesta.

I worked at GE when this was done. Because a lot of things decided what was GE/not GE based on coming from a 3.x address it caused chaos. They called it 3-dot-geddon

It’s a race condition to the bottom.

In Dr Vogel's keynote presentation he brought up Netflix and Chaos Monkey specifically as precursors to this service.

It would be better if that context carried over to this document.

That would be superfluous to the content in which most people would refer to that document for. Amazon already state it’s for Chaos Engineering, they don’t need to list other similar projects on their own product page.

I have to use Ping ID quite a bit to log into customers' systems. I also live in south Louisiana.

The last 5 months have been extremely difficult to maintain focus because, for me at least, it's really difficult for my brain to not check tropical forecasts every time I look at my phone.

I've noticed that my productivity triples when there are no storms in the gulf due to the fact that it's much easier to maintain my attention on engineering tasks. I've thought of setting up a second phone that I have with ONLY ping on it, but I feel that it's the stress driving me to check the tropics, and if I took the time to create a 'just ping' device my mind would find another way to compulsively check.