I believe AGI is possible, but the more I study ML the more convinced I am that we don’t know what it (AGI) is. I also, don’t know if I agree that humans have GI (humanity as a whole does). I think we’re actually copycats more than anything else. Neural nets are the atoms of GI, but it’s not clear what the superstructure of GI is. We all assume that we understand what general intelligence is, but I don’t think we actually have the goods yet. We understand optimization, but we don’t understand the thing that chooses what to optimize. There is some notion of “values” in order for intelligence to be legible, and we don’t know what that is either.
Are you reading the deeper lesson though? The individual examples aren't meant to be authoritative. He was trying to illustrate the very thing you are bringing up. Namely that lazy heuristics create information cascades. The information cascades can have positive effects, as you point out, but they can have profoundly negative consequences, which is the point of the whole article.
We shouldn't use or intellectually tolerate lazy heuristics because they can create immense amounts of counter-productive sense-making, and consequent negative social outcomes (a poorly managed pandemic, for example). The reason this article is hitting a nerve is because he is basically describing the current state of sense-making in the US (and maybe even the West more broadly?), which is quite poor — worse in some areas than others, but still quite degraded all around.
On your doctor take, you do know that the other author of this post is a licensed and practicing Physician, right?
"On your doctor take, you do know that the other author of this post is a licensed and practicing Physician, right?" --
Your observation is indeed much more interesting than the whole article, since it shows a reasonable (and not a caricature with zero value) heuristic. You are saying that I (who may or may not be a physician, but for argument's sake let's say I am not) should not have an opinion that is different, when discussing the behavior of doctors, from the opinion expressed by a licensed and practicing physician. Valuable heuristic?
As for the article itself, my problem with is was not on the problems that reasonable heuristic can generate, but with the useless caricatures.
A doctor who does not visit any patient and simply gives aways a couple of aspirins, is criminally negligent. A doctor who does not call for an MRI for any common symptoms (think headache) that may have been caused, among many other possible causes (dehydration, stress, tension etc.), also by something much more serious (brain cancer) is using a reasonable heuristic, which sometimes may go wrong because for very aggressive cancers, a couple of weeks of delay in starting treatment or having surgery can make the difference between life and death.
A personal case. I went to a doctor with a dermatitis and the doctor recommended, guess what?, a topical steroid cream, which is recommended by dermatologist like a barber recommends a haircut. The heuristic is, dermatitis of unclear origins --> let's try a steroid cream. After I did a bit of research on my own (5 minutes, maybe less), I found out that for my conditions the steroid cream should be absolutely avoided since it makes the condition worse.
The question is and I let you choose the answer: (1) was the doctor using a reasonable heuristic?; (2) was the doctor incompetent and/or an idiot; (3) was the doctor negligent (there is some overlap with (2))?
Should I wait for the opinion of a "licensed and practicing Physician" or I can have my opinion?
Perhaps it could be more helpful to think of the caricatures as "spherical cow" models the author is using when trying to isolate and illustrate a particular social (?) dynamic.
Of course real-world models will be much more nuanced, but the really interesting bit is that you don't need all that nuance to produce the particular pathology outlined in the article. Specifically, selecting on "who is most right most of the time?" can end up causing your city to be covered in lava, missing a store break-in, or what-not.
There are even more levels to explore with this idea. For example, should you always ignore the heuristics and go for the earest, honest experts? Maybe. In the volcano example, the cost of a false negative is so high that you probably are okay with the incurred costs of false positives by the experts.
However, in the case of the Futurist, the false-positives incurred for a non-rock opinion might end up netting you less karma points or whatever. It's somewhat fun doing a re-read trying to evaluate the cost-benefit tradeoff yourself in each case!
I think OPs point is that the article ignores the reason why the heuristic is used in the first place, which is to optimize the process at scale.
The real cost of gathering additional data and knowledge about the circumstances in the 0.01% chance you know that you have at this point, at scale, means you'll end up with worse overall outcome by not being able to scale to all events that need the attention.
Now, if a doctor is just being lazy, doesn't check anything, says you'll be fine take Tylenol and then spends the next half hour reading a book until the next appointment. Or simply wants to go through twice as many patients to make more money. Ya sure, that's just being lazy and useless, and negligent, replace them by a rock at that point.
But if there are 100 people with initial symptoms, and only 1 doctor. And the deep dive to properly asses the likelihood of a 0.01% chance event in the case of symptom: "My hip hurts when I walk" takes multiple hours, a lab test, many follow ups, etc. While this happens and maybe out of the 100 patients waiting, some have symptoms like: "I'm actively bleeding out my mouth.". "I have spores on my skin." "I'm in so much pain I can't fall asleep." All with known much higher likelihood of something pretty bad and urgent.
That's why triage is so important. And this use of the heuristic at scale might make sense when considering the cost/time and available resources trade off.
At the individual level, it means eventually someone will get shafted by this, they'll be sent home with Tylenol, and 3 days later will have a stroke and it would have turned out they are the really rare case where hip pain could indicate a risk of stroke due to say a blood cloth.
But at a larger scale, many more people will have received the treatment they needed more urgently.
I agree: Null-confirming signals should not be considered evidence to discard the null hypothesis. Decision trees are OK, esp those that have "wait and see" near the root. "See if it goes away" is indeed an information-seeking behavior, and a low-cost one at that.
In other words, your doctor could have been losslessly replaced with a rock that says “apply a steroid cream”, and that heuristic would have harmed you.
If it had benefited thousands of others, does it matter in the larger view?
The naive alternative is the doctor always spending time to get to more nuanced advice, that will be more often than (every time the rock was as good) a waste of resources (time/money/opportunity cost), and which in a lot of cases would also be more harmful (iatrogenic harm) than the rock advice.
"But, why 'always'? It's enough that the doctor goes for more nuance when he sees a reason to!", you'll say.
Well, that's what doctors actually do. They are not glorified rocks, they are bimodal (rock mode vs looking deeper mode) - and they're most rock because (even if the miss a few 'loop deeper' cases) because it's way more efficient than the alternative (always or mostly non-rock, that is, digging deeper without first seeing strong indications that they should dig deeper).
The "deeper" (?) lesson and subtext I'm hearing from the OP is that we should get excited like little children about each and every new fad, because, you know, this may just be the 0,01% when it actually matters, and we don't want to miss it.
Well, I don't mind using "lazy heuristics" and waiting around a little to see if something happens. No need to hurry or jump around. Plenty of time.
...no the surface lesson is that lazy heuristics produce habits that provide the illusion of certainty without any rigour backing up those beliefs; the outcome may correspond with reality but the process getting you there isn't rational or dependable, because it always produces the same output regardless of input.
The "deeper lesson" if there is such a thing here is that experts are people too, and just as fallible, only in ways that are generally invisible to everyone but another expert.
Right. I think it has to be boiled down to a simple parable to be sufficiently clear, but it is really attacking a belief that is quite common.
I like this essay on Overconfident Pessimism[0] because I think it gets at the same thing, especially in the context of confident dismissals of important future technological changes:
>There may also be a psychological double standard for "positive" and "negative" predictions. Skepticism about confident positive predictions — say, that AI will be invented soon — feels like the virtuous doubt of standard scientific training. But oddly enough, making confident negative predictions — say, that AI will not be invented soon — also feels like virtuous doubt, merely because the first prediction was phrased positively and the second was phrase negatively.
Ditto. I'm a heavy(ish) user and I've never tested positive, despite being exposed more than once. Though admittedly I'm very cautious (always wear an N95, et al) and vaccinated.
I've operated environmental testing chambers for automotive customers. It's comparatively quick and easy to do the +85C high-temperature testing, but it takes a long time and/or an expensive, oversized refrigeration unit for a chamber and the parts in it to reach the target -40C/F low temperature. -70C/-100F would probably take a very, very long time.
Even in the brutally suspicious automotive industry, it's common to have an operator manually copy the chamber temperature or thermocouple reading to the test report. Especially if you believed that the test requirement was excessive, there would certainly be a temptation to lie and say that the temperature had reached the target.
I wonder how the lie got discovered. Did her successor notice that they were less productive than she had been, because the chamber was slower than what she claimed it could do? Was the chamber incapable of reaching the target temperatures? Did the measured test results come out different?
Einstein exists - he worked at the Institute for Advanced Study at Princeton and was a part of the institutional science of his day which was very much driven by showing expertise to gain attention from the established figures or start a career as a researcher - they don't accept walk-ups consults from strangers at universities, but most especially not at the IAS.
Ethos is very hard, but pathos is harder, especially in secular society.
Let me be clear, I think arguments about ethos generate the most argumentation and heat right now, but the silent killer is really pathos. People feel comfortable arguing ethos. Most people will not argue pathos openly though.
We have a crisis of pathos in our culture. How do you claim that something is important without an appeal to authority? You can't. We live in a culture that is fragmenting its sources of authority; different groups of people have different sources of authority.
Here's a good example. The external dialogue that a lot of conservatives give on climate change is that scientists can't be trusted. The internal dialogue that a lot of them (though not all) engage in goes something like, "The earth doesn't matter. God is going to come back and set things right. So we don't need to worry about it anyway."
Nah, the Pathetic appeal is easy. It isn't sustainable, but the simplest form of Pathos is "Hey, fuck THOSE guys, right? You're better than they are, you're special! Git 'em!", and we're practically swimming in it right now.
I don't know. Ostensibly it looks like pathos is easy right now. Politics seems super intense right now, but it also seems super fake. Mostly I think shows of pathos are fake right now. Getting people to actually do things and act seems like the hardest thing right now. Posting on social media doesn't count.
I was going to add that as my second sentence, but decided to keep it short and sweet. It seems to me that Democrats are desperately trying to show that it still works while the other party is coming up with creative ways to show that it doesn't.
> the other party is coming up with creative ways to show that it doesn't.
Chief among them, making it so that it doesn't. In a way, just a variation of their age-old way of justifying their "Governement is the problem, it never works!" stance by fucking up government whenever they're in power.
I don't disagree, but I think there is a lot of nuance to it. Thiel is an ardent disciple of Rene Girard. If people want insight into how Thiel sees the world, I would strongly recommend reading The Straussian Moment. There is a Peter Robinson interview about the essay that can serve as a good primer[1].
Girard and thus Thiel hold that "If there is a normal order in societies, it must be the fruit of an anterior crisis."
They're not wrong that this is A source of order, but they overlook the human tendency to form community and cooperate: this vision of theirs is about how to form societies when they are by definition all against all, a brutal struggle of nihilism and despair.
That's only one way humans can be, and it's a way that competes with the more leftist tendency to make everything about the community and cooperation.
Seems like a meta-narrative is needed that incorporates both of these positions that are held by their supporters as the ONLY position that can exist.
You need to read Girard more closely (I recommend reading Things Hidden...)
Girard does not embrace a Hobbesian view of the world, he is trying to explain the origin of human cooperation and society. The scapegoat narrative is not a nihilistic war of all-against-all it is a narrative of all-against-one to create order. Human cooperation can exist when violence is pared-down to one person (the scapegoat).
Where Girard admits that nihilism can come into play is when the scapegoat narrative becomes openly acknowledged. Which is what he argues Christianity effectively did (opened up the scapegoat mechanism for all to see, thus rendering it ineffective). Girard predicts that our lack of a scapegoat mechanism will likely lead to apocalyptic violence.
Frankly, I think the leftist view that humans can cooperate out-of-the-head-of-Zeus is naive and needs no account, because it has never happened.
Right, except very frequently, the scapegoat isn't a single figure, but rather a group.
The Bourgeoise, the Jews, the QAnons, the Blacks, the Whites, the Immigrants, the Rich, the Anti-Vaxxers, the Catholics, the Muslims, the Kulaks, the <insert group here>
If humanity could unite and create order under the periodic ritual sacrifice of a single King, we'd be much better off than the way it actually happens, which is scapegoating and sacrificing millions of people at once.
Sorry, but all this stuff about the “scapegoat narrative” and so forth is clearly the sort of syncretic pseudo-spiritual voodoo that crops up from time to time.
absolutely — my few sentences is just a rough summary of a small part of Thiel's thinking and ideology
summing his life as a Silicon Valley bad guy is a gross oversimplification — his actions and work is the product of decades of his study and thought, and has evolved in a number of ways over time if you follow his writings and interviews
it's clear that he holds the ideas he's arrived at with conviction, and has been in the process of enacting them through financial or political or intellectual means
As the rosy promises of Silicon Valley flounder and the naked reality of techno-capitalism becomes apparent (hello serfdom, err, gig economy, we've missed ya), it is imperative to find a scapegoat. What better scapegoat than a disciple of Girard?
IMO his intellectualism provides active cover for an autocratic-minded zero-sum sociopathy, which when married to his accumulated wealth represents an existential threat.
This is not irrelevant to the conception of and market of Palantir.
He is the Erik Prince of "tech" and like Prince, is a literal merchant of death.
If true that distinguishes him from nobody, including people without power. However, a serious investigation of Thiel's work dismisses any serious accusation of "Intellectualism" it is well thought out and philosophically rigorous.
Is he completely correct? Probably not. But then, who is?
If one person is an existential threat to humanity (which is what I assume you mean) then, frankly, we probably "deserve" it (in the evolutionary sense). Thiel is useful here to countermand this point. One of his most useful ideas is that the so called "powerful" and "elite" have much less power than we think they do and can can be easily circumvented if ignored (of course there are marginal cases).
Great example of an article which, I hope, invokes the Gell-Mann amnesia effect for most of its tech-educated readers. Its fascinating to read such articles being written with so little irony.
The Economist is a great exemplar of the Gell-Mann amnesia effect. Their subpar coverage of wars, Iraq WMDs, Syria, Afghanistan, Middle East, Asia, Latin America (they couldn't even call a coup in Bolivia a coup), Tech, Business all prove the Gell-Man amnesia effect. I don't have enough knowledge about their coverage of Finance though.
They seem to fail upwards like most of mainstream media acting as mouthpieces of the DC/London establishement, stenagrophers of the national security establishment and PR agency of many VC funded tech bros.
