> Still, we are the apex predator of this planet, thanks to our sole distinguishing feature, which is our intelligence and ability to cooperate.
Indeed, people seem to overlook the fact that superhuman intelligence already exists as human organizations. A company has the institutional knowledge to create things that no individual human could ever come close to, and many of them have the narrow focus of theoretical "paperclip maximizers." These organizations even have the ability to use there superhuman ability to improve themselves, and do to a certain extent, but they don't create the cartoonish runaway feedback loop people imagine. Reality ends up being a lot more difficult than handwaved fiction.
The idea of an AI apocalypse has become the quasi-religious dogma of certain groups, but it overlooks the real issues we face. I suppose when you see something everyday, it's easy to ignore its extraordinary nature.
> people seem to overlook the fact that superhuman intelligence already exists as human organizations
Culture is faster than meat.
Faster to inculcate. Faster to copy. Faster to communicate. Faster to update.
And it's a lot harder to kill.
The day our primate progenitors began passing down their own knowledge to subsequent generations was the greatest leap forward in global learning rate that our planet has ever seen.
I think the jury's still out on our impact of printing it out into a three ring binder.
> > people seem to overlook the fact that superhuman intelligence already exists as human organizations
> Culture is faster than meat.
> Faster to inculcate. Faster to copy. Faster to communicate. Faster to update.
> And it's a lot harder to kill.
> The day our primate progenitors began passing down their own knowledge to subsequent generations was the greatest leap forward in global learning rate that our planet has ever seen.
> I think the jury's still out on our impact of printing it out into a three ring binder.
Culture and memes are the software side of evolution.
Your body and genetics, it takes a really long time to evolve a different body (thousands of generations), but it takes very little time to update culture (couple of generations).
There is evidence that single generation evolution can happen with strong enough shocks. Nothing immediately obvious, just detectable DNA changes that lead to changes in risk for certain conditions.
Epigenetics and Genetics (DNA). Epigenetics allows for changes within an organism's lifetime, and include the ability to pass along environmental stressors to the next generation.
There is a great book from MIT press [Evolution in Four Dimensions](https://mitpress.mit.edu/books/evolution-four-dimensions) that goes into how Genetics, Epigenetics, behavior and symbolic transmission of data between generations guide evolution.
This is a good reason to start thinking about corporate lysing. Individual people who do bad stuff will die someday. Corporations don't have that problem and can grow unchecked, like a certain illness.
There's a good video by Robert Miles about that 'Why Not Just: Think of AGI Like a Corporation?' (youtube.com/watch?v=L5pUA3LsEaw)
Corporations are kind of like AIs, if you squint. How hard do you have to squint though, and is it worth it?
In this video we ask: Are corporations artificial general superintelligences?
> The idea of an AI apocalypse has become the quasi-religious dogma of certain groups, but it overlooks the real issues we face. I suppose when you see something everyday, it's easy to ignore its extraordinary nature.
Not at all.
Pointing out "here are some problems we face" does not mean that other problems don't exist. We can have more than one problem at the same time.
Oh that's a nice way to put it - members of an organisation pretty much are its neuron bundles, ones smart enough to introspect and reorganise themselves.
Though I think the organisation's intelligence might be more parallelised/horizontal than vertical, i.e. it can solve many problems simultaneously (manufacturing, accounting, strategy, R&D, HR, etc.), but not necessarily combine most of its members to solve extremely complex single problems (say, a mathematical one).
Extremely rarely relative to their power, and relative to control over individuals.
There is some regulation, and an occasional war to shatter a regime.
The problem with this exponential growth nonsense is that there are limits to growth. If we had 3% exponential growth for two thousand years we would be colonizing a big fraction of the galaxy by now.
The corona virus hasn't grown exponentially, there have been ups and downs.
With capitalist economies there are depressions and recessions. The napkin math doesn't work.
The concept of having uninterrupted exponential growth every single year is a fallacy. Even interest embodies this fallacy. The lender demands exponential interest, even if the economy isn't growing this year, so people must default on their loans.
Really, all the exponential growth is at best logistic growth with an unknown upper bound. That upper bound can increase over time as society develops but it doesn't grow exponentially. It's more like the exponential phase of logistic growth is an attempt to get to the upper bound as quickly as possible.
Indeed, people seem to overlook the fact that superhuman intelligence already exists as human organizations. A company has the institutional knowledge to create things that no individual human could ever come close to, and many of them have the narrow focus of theoretical "paperclip maximizers." These organizations even have the ability to use there superhuman ability to improve themselves, and do to a certain extent, but they don't create the cartoonish runaway feedback loop people imagine. Reality ends up being a lot more difficult than handwaved fiction.
The idea of an AI apocalypse has become the quasi-religious dogma of certain groups, but it overlooks the real issues we face. I suppose when you see something everyday, it's easy to ignore its extraordinary nature.