Another (s)nail in the coffin of the superstition that more powerful computers will somehow magically result in strong AI. Our current AI is nowhere near the level of autonomy of animals like snails. If it was just a matter of more computing power, or more data, we'd have snail-level intelligence down pat by now.

Edit: Er, btw, the bbc article misleadingly talks of "brain" calls. Moluscs don't have brains. The Nature paper (linked from the parent) clarifies that it's about "two neuron types".

I agree, but doesn't that then lend support to the idea of a FOOM?

Hmm, dunno. It took millions of years for life to make crits with more than two ganglia to rub together. Again, why would it be any different with artificial intelligence?

Like, I don't want to be controversial and all (OK, I do) but it seems to me the folks who claim that strong AI is Just Around the Corner™ and it's just a matter of more data and bigger computers, have too much testosterone sloshing around in their system and just wanna show the world ... something. There's a real fixation with _big_ data, _powerful_ and _fast_ computers and _strong_ AI, stuff like that. The whole FOOM thing is just another one of those things- like somebody didn't hear what Alan Moore had to say about comics and growing up already.

I mean, I went through nerd-puberty myself, so maybe if people kept asking me what the ideal AI would be I'd start saying things like "obviously, it will be _elegant_ and it will make use of _pretty_ data" or something equally stereotypical.

But, it's all just fantasies. If there was going to be some sort of takeoff, why didn't it happen in a few billions of years of life on the earth? If we think we have the resources now for computers to take off (instead of imploding in some sort of paperclipocalypse) there certainly were more than enough resources for a couple of takeoffs of biological entities in the time since the dawn of life. Instead, what we got is one -count them- one species with the intelligence to start a technological civilisation and many, many that are almost there but not quite. All the data we have hints to the fact that singularities are extremely unlikely, and even the one we know occured -ours- took a lot of time to set up properly and had to evolve over millions of years. So it wasn't a singularity at all, more like a slow, steady process that eventually surpassed some sort of limit.

tl;dr FOOM my foot.

I'll bite. Biting is a fun animal activity. I think you make reasonable points. FOOMs are sexy, and sexiness is intrinsically suspicious because people want to believe in them. Still we do know of extraordinary change in the past.

My suspicion is that FOOM is both real and not real depending on your scope. Here I am defining a FOOM as a change in the rate of a variable speed.

If I were against FOOM I would point out that, as you say, you have millions of years of statis in the biological record. To quote Wikipedia on one element of this: "once species appear in the fossil record they will become stable, showing little net evolutionary change for most of their geological history." Makes sense. If you have adapted to your niche then you have little to do but occupy it.

If I were for FOOM I would argue that the human brain has not changed that fundamentally in hundreds of thousands of years. Yet the only way to attribute the explosion of complexity of our society in the last several thousand seems to point to changes in our brains. Perhaps change in our brains mostly in relation to artifacts in a feedback loop i.e. the brains move out of their shell casing and into books, cogs & wheels. The change to agriculture is one FOOM. The change to the Industrial Revolution is another FOOM. The invention of electricity, Internet are even more rapid FOOMs than what went before. The tempo is itself interesting.

Against the FOOM I would then say as you have.

> If there was going to be some sort of takeoff, why didn't it happen in a few billions of years of life on the earth? If we think we have the resources now for computers to take off (instead of imploding in some sort of paperclipocalypse) there certainly were more than enough resources for a couple of takeoffs of biological entities in the time since the dawn of life.

I have two ideas on this.

The first is that you're right. We are just a splendidly unusual animal with a terrific amount of good luck. We are talking about this and are able to do so because of a kind of anthropic principal.

The second is that you're right, but not for the reason of rarity but something more sinister. There 'were' other FOOMs but we've been deleting them. One FOOM to rule them all. We are extraordinarily good at killing. Getting away from the probable fact we genocided our closest competitors back in the day. Think of civilizations. Any civilization that refused to FOOM (think Chinese) got ultimately accosted by a different civilization that did. Then (usually) we destroyed anybody who couldn't keep up by absorbing their raw materials to build our faster growing civilization.

I find the second hypothesis more interesting but not sure how to weigh evidence for these cases.

The second hypothesis is not directly interesting but one of the interesting consequences of it being true is that collapse, of species and civilizations is very much more likely than we currently anticipate.

My overarching theory now becomes that both ideas are the same idea, just two sides of the same coin. FOOMing and extinction events are very much interrelated. The expansion of humanity (a FOOM) vs the disappearance of most wild animals. It is notable that species reproduce exponentially if they can. Suppose we discover for example a easy way to alter our DNA to make a superior human. Then it is likely the other humans are ultimately destroyed by being outcompeted and using up their available resources.

Basically FOOMs are not rare but they make other FOOMs rare. There is no control. I can't think of a less awkward way of putting this, I hope you understand my point.

Elon Musk thinks if you made the technology egalitarian the problem would be solved. I think this is not so because some of the technology's users are much smarter or are in better initial starting environments and would outcompete the others even if they had the same tools.

This means that only by a FOOM transcending i.e. using resources and concepts that are non-rival-goods, would there not be trouble.

Consider that when we meet a non-contacted tribe in the Amazon, we don't steal their tools and wives and murder the males. Today we don't need/want to. But our ancestors would have done that without a qualm. Even in recent history with the Spanish conquistadors you can see a shadow of how we can gobble things up. To the un-contacted tribes and Cargo Cult Islanders, our Western Civilization is a FOOM. Barely explicable and vast.

tldr; Don't get outcompeted.

Oh dear. Should have kept my mouth shut :)

I'm afraid it FOOMed.