I always have time for a well reasoned argument even if it contradicts my own view. These arguments are not well argued and they contradict my view. I regret reading that article.
I agree. Some of the arguments are even wrong. Specially: "Godel’s Theorem shows that no computer, or mathematical system, can match human reasoning." This is plainly false, in many ways. Gödel't Theorem says nothing about computers (thanks to Turing we know we can't have an algorithm that solves theorems, that was not Gödel's work and Gödel's work still allowed for that possibility to exist), and in any case, we aren't sure our brains are just Turing machines or not, so we can't say neither Gödel nor Turing proved anything about the relative power of computers and human reasoning.
You know, I always figured that a computer-simulated world would have smart compression techniques. Thus, the massive quantity of electrons flying around necessary for a brain impulse, for instance, only "exists" when someone does an MRI (or whatever tool they use for that sort of thing). Otherwise, a reference to "pleasure" or "pain" is passed into the perceptions.
More interesting is the lower paragraph which discusses objections to creating a friendly transhuman AGI, given that it is possible to create a transhuman AGI. I am aware that singinst thinks a lot about these issues..is anyone aware of any other organizations working with these things?
For a far, far more entertaining take on the topic I recommend Bruce Sterling's "The Singularity: Your Future as a Black Hole", available from this page:
We already build very powerful intelligences all the time, in the form of children. But despite having a lot of raw computing power, they aren't very useful without knowledge. They need to learn a lot of things before they can live effectively, solve important problems, etc...
Artificial intelligence and intelligence are not fundamentally different things. If you want AI's that turn out better than children do, you'll need improved parenting and educational techniques, or an alternative. I haven't seen any commentary on how singularity people plan to deal with this. And if they could, why not just use that technique on human children and vastly improve the world, now?
The main difference between AI and children I've seen proposed is that the AIs can run on superior hardware. But it's not like people currently use all the hardware resources their brain makes available; maybe that will be a bottleneck in the future, but it isn't one we've hit yet.
You think the input format will be easier for AI? I don't know. We have evolved knowledge of the input format for children. Meanwhile the majority of our knowledge is inexplicit, and we are bad at turning that into vague English, let alone into something as precise as code.
Back to children, we're so good at teaching them that we manage to teach them many inexplicit ideas without ever having much understanding of those ideas.
