Yes, at the end it’s just a huge n-dimensional cube of numbers. But that’s my point : can human-level intelligence be built on top of that ? I was just as skeptical as you for this reason (and many others more important in my opinion, such that those machines don’t have any personal embodied experience of the information they manipulate, and as such meaning shouldn’t be able to emerge from them). But looking at the facts i have to admit i’m at least partially wrong.

So the question might be: is the capacity to build "internal intellectual model" the sign of intelligence? Can we consider that attributing weight to AI neurons a "internal representation of an abstract model"? When we dump an trained AI model, can we consider this as an "abstract model construction"?

There's a (maybe subtle) point being implied in this statement: it assumes the idea that we'd be able to poke the AI in some way that makes sense to us that would demonstrate that the AI doesn't "understand".

As a trivial example, it's possible to have an extended conversation with chat gpt about ways to treat male pattern baldness, and then ask it "what do you think the likelihood is that I am balding?" and have chat gpt reply along the lines of "I have no knowledge of your physical body." (I made that example up based on previous conversations -- chat gpt might do better in real life)

But if chat gpt were also able to answer that question reasonably -- and every other question we might think to ask, all using its current (non-"understanding") method -- then can we even say that it doesn't "understand as we consider it usually" if we can't demonstrate that it doesn't?

If not, how does it work exactly? Is there any difference between "a sophisticated pattern matching on a (very huge) corpus" and how a human would invent a joke?

When you are an novice artist it's usually bad thing to rely on your mental model of reality. Children draw like that. Instead it's better to just observe patterns of light and darkness on the model and reproduce them. But at some point of that, when you start being creative with your art it really pays off to actually know how many fingers humans do have.

It doesn't mean neural networks are completely incapable of creating the model of reality, AlphaGo creates models of Go reality that's better than human players, they just can't do that through language alone.

Surely there's some jokes in the training database... enought to allow generating "new ones" by switching pieces

The question about: are humans doing differently is the one about creation. Let's say that maybe 90% a "just" simple pattern matching (same mechanic as already existing jokes) and 10% is new mechanic (original joke)

I think that's a bit of a simplification :-)

You’re right we are prone to attributing intelligence to things that aren’t. But what actually is intelligence and is there anyplace you are sure it exists rather than a clever and thorough simulation?

There's some kind of philosophical debate about what creation is... Another debate might be "is creation the most specific sign of intelligence ?" Is there anything like "pure creation" or are all creation only adaptation from prior art ? And if so: when did the initial parts come from ? Is creation only the systematic exploration of a predefined space ?