They tick all the boxes: oblique meaning, a semiotic field, the illusion of hidden knowledge, and a ritual interface. The only reason we don't call it divination is that it's skinned in dark mode UX instead of stars and moons.

Barthes reminds us that all meaning is in the eye of the reader; words have no essence, only interpretation. When we forget that, we get nonsense like "the chatbot told him he was the messiah," as though language could be blamed for the projection.

What we're seeing isn't new, just unfamiliar. We used to read bones and cards. Now we read tokens. They look like language, so we treat them like arguments. But they're just as oracular, complex, probabilistic signals we transmute into insight.

We've unleashed a new form of divination on a culture that doesn't know it's practicing one. That's why everything feels uncanny. And it's only going to get stranger, until we learn to name the thing we're actually doing. Which is a shame, because once we name it, once we see it for what it is, it won't be half as fun.

Just looking at my recent AI prompts:

I was looking for the name of the small fibers which form a bird’s feather. ChatGPT told me they are called “barbs”. Then using straight forward google search i could verify that indeed that is the name of the thing i was looking for. How is this “divination”?

I was looking for what is the g-code equivalent for galvo fiber lasers are and ChatGPT told me there isn’t really one. The closest might be the sdk of ezcad, but it also listed several other opensource control solutions too.

Wanted to know what are the hallmarking rules in the UK for an item which consist of multiple pieces of sterling silver held together by a non-metalic part. (Turns out the total weight of the silver matters, while the weight of the non-metalic part does not count.)

Wanted to translate the hungarian phrase “besurranó tolvaj” into english. Out of the many possible translations chatGPT provided “opportunistic burglar” fit the best for what I was looking for.

Wanted to write an sql alchemy model and i had an approximate idea of what fields i needed but couldn’t be arsed to come up with good names for them and find the syntax to describe their types. ChatGPT wrote it for me in seconds what would have taken me at least ten minutes otherwise.

These are “divination” only in a very galaxy brained “oh man, when you open your mind you see everything is divination really”. I would call most of these “information retrieval”. The information is out there the LLM just helps me find it with a convenient interface. While the last one is “coding”.

You presented clear, factual queries. Great. But even there, all the components are still in play: you asked a question into a black box, received a symbolic-seeming response, evaluated its truth post hoc, and interpreted its relevance. That's divination in structural terms. The fact that you're asking about barbs on feathers instead of the fate of empires doesn't negate the ritual, you're just a more practical querent.

Calling it "information retrieval" is fine, but it's worth noticing that this particular interface feels like more than that, like there's an illusion (or a projection) of latent knowledge being revealed. That interpretive dance between human and oracle is the core of divination, no matter how mundane the interaction.

I don't believe this paints with an overly broad brush. It's a real type of interaction and the subtle distinction focuses on the core relationship between human and oracle: seeking and interpreting.

And if the place would be any good at the second kind of queries you would call it Lost&Found and not the Oracle.

> illusion (or a projection) of latent knowledge being revealed

It is not an illusion. Knowledge is being revealed. The right knowledge for my question.

> That interpretive dance between human and oracle is the core of divination, no matter how mundane the interaction.

Ok, so if I went to a library, used a card index to find a book about bird feather anatomy, then read said book to find that the answer to my question is “barb” would you also call that “divination”?

If i would have paid a software developer to turn my imprecise description of a database table into precise and thight code which can be executed would you also call that “divination”?

Both gets you a hammer, but I don't think anyone would call the latter magical/divine? I think its only "magical" simply because its incomprehensible...how does a hammer pops into reality? Of course, once we know EXACTLY how that works, then it ceases to be magical.

Even if we take God, if we fully understand how He works, He would no longer be magical/divine. "Oh he created another universe? This is how that works..."

The divinity comes from the fact that it is incomprehensible.

https://archdruidmirror.blogspot.com/2017/06/clarkes-fallacy...

So any and all human communication is divination in your book?

I think your point is pretty silly. You're falling into a common trap of starting with the premise "I don't like AI", and then working backwards from that to pontification.

My original comment is making a structural point, not a mystical one. It’s not saying that using AI feels like praying to a god, it's saying the interaction pattern mirrors forms of ritualized inquiry: question → symbolic output → interpretive response.

You can disagree with the framing, but dismissing it as "I don’t like AI so I’m going to pontificate" sidesteps the actual claim. There's a meaningful difference between saying "this tool gives me answers" and recognizing that the process by which we derive meaning from the output involves human projection and interpretation, just like divination historically did.

This kind of analogy isn't an attack on AI. It’s an attempt to understand the human-AI relationship in cultural terms. That's worth engaging with, even if you think the metaphor fails.

Their counterargument is that said structural definition is overly broad, to the point of including any and all forms of symbolic communication (which is all of them). Because of that, your argument based on it doesn't really say anything at all about AI or divination, yet still seems 'deep' and mystical and wise. But this is a seeming only. And for that reason, it is silly.

By painting all things with the same brush, you lose the ability to distinguish between anything. Calling all communication divination (through your structural metaphor), and then using cached intuitions about 'the thing which used to be called divination; when it was a limited subset of the whole' is silly. You're not talking about that which used to be called divination, because you redefined divination to include all symbolic communication.

Thus your argument leaks intuitions (how that-which-was-divination generally behaves) that do not necessarily apply through a side channel (the redefined word). This is silly.

That is to say, if you want to talk about the interpretative nature of interaction with AI, that is fairly straightforward to show and I don't think anyone would fight you on it, but divination brings baggage with it that you haven't shown to be the case for AI. In point of fact, there are many ways in which AI is not at all like divination. The structural approach broadens too far too fast with not enough re-examination of priors, becoming so broad that it encompasses any kind of communication at all.

With all of that said, there seems to be a strong bent in your rhetoric towards calling it divination anyway, which suggests reasoning from that conclusion, and that the structural approach is but a blunt instrument to force AI into a divination shaped hole, to make 'poignant and wise' commentary on it.

> "I don’t like AI so I’m going to pontificate" sidesteps the actual claim

What claim? As per ^, maximally broad definition says nothing about AI that is not also about everything, and only seems to be a claim because it inherits intuitions from a redefined term.

> difference between saying "this tool gives me answers" and recognizing that the process by which we derive meaning from the output involves human projection and interpretation, just like divination historically did

Sure, and all communication requires interpretation. That doesn't make all communication divination. Divination implies the notion of interpretation of something that is seen to be causally disentangled from the subject. The layout of these bones reveals your destiny. The level of mercury in this thermometer reveals the temperature. The fair die is cast, and I will win big. The loaded die is cast, and I will win big. Spot the difference. It's not structural.

That implication of essential incoherence is what you're saying without saying about AI, it is the 'cultural wisdom and poignancy' feedstock of your arguments, smuggled in via the vehicle of structural metaphor along oblique angles that should by rights not permit said implication. Yet people will of course be generally uncareful and wave those intuitions through - presuming they are wrapped in appropriately philosophical guise - which is why this line of reasoning inspires such confusion.

In summary, I see a few ways to resolve your arguments coherently:

1. keep the structural metaphor, discard cached intuitions about what it means for something to be divination (w.r.t. divination being generally wrong/bad and the specifics of how and why). results in an argument of no claims or particular distinction about anything, really. this is what you get if you just follow the logic without cache invalidation errors.

2. discard the structural metaphor and thus disregard the cached intuitions as well. there is little engagement along human-AI cultural axis that isn't also human-human. AI use is interpretative but so is all communication. functionally the same as 1.

3. keep the structural metaphor and also demonstrate how AI are not reliably causally entwined with reality along boundaries obvious to humans (hard because they plainly and obviously are, as demonstrable empirically in myriad ways), at which point go off about how using AI is divination because at this point you could actually say that with confidence.

The issue isn't "cached intuitions" about divination, but rather that you're reading the comparison too literally. It's not about importing every historical association, but about identifying specific parallels that shed light on user behavior and expectations.

Your proposed "resolutions" are based on a false dichotomy between total equivalence and total abandonment of comparison. Structural analysis can be useful even if it's not a perfect fit. The comparison isn't about labeling AI as "divination" in the classical sense, but about understanding the interpretive practices involved in human-AI interaction.

You're sidestepping the actual insight here, which is that humans tend to project meaning onto ambiguous outputs from systems they perceive as having special insight or authority. That's a meaningful observation, regardless of whether AI is "causally disentangled from reality" or not.

This applies just as well to other humans as it does AI. It's overly-broad to the point of meaninglessness.

The insight doesn't illuminate.

Indeed, I hold that driving readers to intuit one specific parallel to divination and apply it to AI is the goal of the comparison, and why it is so jealously guarded, as without it any substance evaporates.

The thermometer has well-founded authority to relay the temperature, the bones have not the well-founded authority to relay my fate. The insight, such as you call it, is only illuminative if AI is more like the latter than the former.

This mode of analysis (the structural) takes no valid step in either direction, only seeding the ground with a trap for readers to stumble into (the aforementioned propensity to not clear caches).

> That's a meaningful observation, regardless of whether AI is "causally disentangled from reality" or not.

If the authority is well-founded (i.e., is causally entangled in the way I described), the observation is meaningless, as all communication is interpretative in this sense.

The structural approach only serves as rhetorical sleight of hand to smuggle in a sense of not-well-founded authority from divination in general, and apply it to AI. But the same path opens to all communication, so what can it reveal in truth? In a word, nothing.

And regardless of how many words someone uses in their failed attempt at "gotcha" that nobody else is playing. There are certainly some folks acting silly here, and it's not the vast majority of us who have no problem interpreting and engaging with the structural analysis.

Words from an AI are just words.

Words in a human brain have more or less (depending on the individual's experiences) "stuff" attached to them: From direct sensory inputs to complex networks of experiences and though. Human thought is mainly not based on words. Language is an add-on. (People without language - never learned, or sometimes temporarily disabled due to drugs, or permanently due to injury, transient or permanent aphasia - are still consciously thinking people.)

Words in a human brain are an expression of deeper structure in the brain.

Words from an AI have nothing behind them but word statistics, devoid of any real world, just words based on words.

Random example sentence: "The company needs to expand into a new country's market."

When an AI writes this, there is no real world meaning behind it whatsoever.

When a fresh out of college person writes this it's based on some shallow real world experience, and lots of hearsay.

When an experienced person actually having done such expansion in the past says it a huge network of their experience with people and impressions is behind it, a feeling for where the difficulties lie and what to expect IRL with a lot of real-world-experience based detail. When such a person expands on the original statement chances are highest that any follow-up statements will also represent real life quite well, because they are drawn not from text analysis, but from those deeper structures created by and during the process of the person actually performing and experiencing the task.

But the words can be exactly the same. Words from a human can be of the same (low) quality as that of an AI, if they just parrot something they read or heard somewhere, although even then the words will have more depth than the "zero" on AI words, because even the stupidest person has some degree of actual real life forming their neural network, and not solely analysis of other's texts.

There are 40 definitions of the word "consciousness".

For the definitions pertaining to inner world, nobody can tell if anyone besides themselves (regardless of if they speak or move) is conscious, and none of us can prove to anyone else the validity of our own claims to posess it.

When I dream, am I conscious in that moment, or do I create a memory that my consciousness replays when I wake?

> Words from an AI have nothing behind them but word statistics, devoid of any real world, just words based on words.

> […]

> When a fresh out of college person writes this it's based on some shallow real world experience, and lots of hearsay.

My required reading at school included "Dulce Et Decorum Est" by Wilfred Owen.

The horrors of being gassed during trench warfare were alien to us in the peaceful south coast of the UK in 1999/2000.

AI are limited, but what you're describing here is the "book learning" vs. "street smart" dichotomoy rather than their actual weaknesses.

What does 'mainly' mean here ?

Language is so very human-specific that human newborns already have the structures for it, while non-human newborns do not.

This to me is massive. The Oracle of Delphi would have no idea where you left your sandals, but present day AIs increasingly do. This (emergent?) capability of combining information retrieval with flexible language is amazing, and its utility to me cannot be overstated, when I ask a vague question, and then I check the place where the AI led me to, and the sandals are indeed there.

P.S. Thank you for introducing me to the word "querent"

As prominent examples look at the news stories about lawyers citing nonexistent cases or publications.

People think that LLMs do information retrieval, but they don't. That is what makes them harmful in education contexts.

https://en.wikipedia.org/wiki/I_Ching

[0]: https://www.arl.org/blog/training-generative-ai-models-on-co...

1 a : any of the light, horny, epidermal outgrowths that form the external covering of the body of birds

NOTE: Feathers include the smaller down feathers and the larger contour and flight feathers. Larger feathers consist of a shaft (rachis) bearing branches (barbs) which bear smaller branches (barbules). These smaller branches bear tiny hook-bearing processes (barbicels) which interlock with the barbules of an adjacent barb to link the barbs into a continuous stiff vane. Down feathers lack barbules, resulting in fluffy feathers which provide insulation below the contour feathers.

Rephrasing: LLMs are the modern day oracle that we disregard when it appears to be hallucinating, embrace when it appears to be correct.

The popularity of LLMs may not be that we see them as mystical, but rather that they're right more often than they're wrong.

“That is not what I meant at all;

That is not it, at all.”

— T.S. Eliot

I think the concern is that people are asking it things that are harder to verify AND they are not making any attemp to verify it because they assume it's correct 100%

Why not just start with a straight forward Google search?

Google doesn't give you the answer (unless you're reading the AI summaries - then it's a question of which one you trust more). Instead it provides links to

    https://www.scienceofbirds.com/blog/the-parts-of-a-feather-and-how-feathers-work
    https://www.birdsoutsidemywindow.org/2010/07/02/anatomy-parts-of-a-feather/
    https://en.wikipedia.org/wiki/Feather
    https://www.researchgate.net/figure/Feather-structure-a-feather-shaft-rachis-and-the-feather-vane-barbs-and-barbules_fig3_303095497
    

> I think we the technorati need to be educating non-technical users of these things as much as possible in order to demystify them so that people don't treat them like oracles.

Exactly. That phrase "meeting people where they're at" comes to mind. Less as a slogan and more as an pedagogical principle. It's not enough to deliver information, it's important to consider how people make sense of the world in the first place.

Like you pointed out, the analogy to divination isn't meant to mystify the tech. It's meant to describe how, to many people, this interface feels. And when people interact with a system in a way that feels like consulting an oracle, we can't dismiss that as ignorance. We have to understand it as a real feature of how people relate to symbolic systems. That includes search engines, card catalogs, and yes, LLMs.

This is one of the densest concentrations of AI-literate minds on the internet. That's exactly why I think it's worth introducing frames from outside the dominant paradigm: anthropology, semiotics, sociology. It's not to be sill or weird, but to illuminate things engineers might otherwise take for granted. It's easy to forget how much unspoken cultural infrastructure supports what we call "information retrieval."

If a few comments dismiss that perspective as silly or unscientific, I don't take it personally. If anything, it reassures me I'm tapping into something unfamiliar but worth sharing and worth having deep discussion on.

Thanks again for engaging in good faith. That's the kind of exchange that makes this place valuable.

Of course, nobody listens anyway.

I can’t tell someone how to drive in ice in a way where they can really understand it. I can’t explain how certain specific news sources are biased and how to critically think. I can’t explain how to cut wood on a table saw so it’s perfectly straight. The only way to learn is through repeated usage and practice.

You can tell users that a LLM can make mistakes — and many tools do — but what does making mistakes really mean? Will it give it a recipe for a cake when I ask for a cupcake? Does it give 14 if I ask to add 3 and 4? Will it agree with me even when I suggest something totally wrong? What does hallucinate mean? That means it will give me a fantasy story if I ask how to change my oil filter?

I use LLMs, I enjoy them, I'm more productive with them.

Then I go read a blog from some AI devs and they use terms like "thinking" or similar terms.

I always have to ask "We're still s stringing words together with math right? Not really thinking right?" The answer is always yes ... but then they go back to using their wonky terms.

The problem arises when we apply this intuition to things where too many people in the audience might take it literally.

That said, I find some of your claims less compelling. I'm an atheist, so there's no "creator" for humans to be worshipped. But also, human intelligence/sentience came from non-intelligence/non-sentience, right? So something appeared where before it didn't exist (gradually, and with whatever convoluted and random accidents, but it did happen: something new where it didn't exist before). Therefore, it's not implausible that a new form of intelligence/sentience could be fast tracked again out of non-intelligence, especially if humans were directing its evolution.

By the way, not all scifi argues that machines/programs can evolve to develop humanity. Some scifi argues the contrary, and good scifi wonders "what makes us human?".

I also concede that a "form" of intelligence/sentience could emerge. Presently the form is called "artificial," I'd say.

And you're right... not all scifi argues machine evolves to humanity. I meant to refer to that body of scifi that does. And the body that explores the "what make us human," indeed that's the good stuff. Alex Garland's Ex Machina comes to mind. I absolutely loved that film. The ending was chilling!

As for atheism: it's merely the lack of belief that god exists (or in some definitions, the active belief that it doesn't exist). Nothing else, nothing more. Individual atheists may believe some other things, or not.

I believe some kind of intelligence could arise again, much like ours arose "out of nonintelligence". I just don't think this is it -- LLMs are very impressive but they are likely a dead end, and regardless, I don't think they are conscious by any meaningful definition of the word. It's mostly hype and gullible people at this point.

To claim we have already achieved machine sentience is preposterous hype swallowing. To assert that it is impossible is baseless conjecture.

But I never claimed that a person with synthetic augmentations was any less human/sentient than those with all their natural parts. I likewise never claimed that "we have already achieved machine sentience."

And here's some food for thought... Regardless if one believes in God or not, is it really that offensive to claim that our humanity is unique in its sentience? I find it offensive when some claim that aliens built the Egyptian pyramids. (It sure provides great fodder for some wondrous science fiction, indeed.)

I will re-assert in other words, for the sake of clarity... That sentience is not an emergent property. That is the foundational definition upon which I contemplate the mystery (i.e. the reality of our being that science will never develop sufficiently to fully explain) of our existence. I for one, enjoy the endeavor of employing my sentience to explore & investigate our wondrous universe and to equally explore & relate with you and call you a friend in spite of our disagreement. Cheers!

I believe it's fundamentally the same as the people convinced "[he/she/it] really loves me." In both cases they've shaped the document-generation so that it describes a fictional character they want to believe is is real. Just with an extra dash of Promethean delusions of grandeur.

But I'm so crestfallen and pessimistic about the future of software and software engineering now that I have stopped fighting that battle.

makes the baby jesus cry

I think time has proved that he was right. It is meaningless to discuss things like "Artificial Intelligence". We can only discuss machines in terms of performance, not in terms of subjectivity. Whenever we try to do the latter, we end up in a semantic quagmire.

This is the main reason I find the current hype irksome. The performance of machines should be evaluated objectively and in terms of the jobs they need to perform. Attributing 'intelligence' or 'thought' to machines is indeed absurd.

The 'imitation game' argument is categorically not that 'if machines appear to be intelligent they in fact are'. What it really is: 'machines cannot think obviously, but what could they do that currently requires a thinking human to be in charge?'.

75 years after Turing published the relevant paper, people are still doing what he called absurd (trying to attribute thought and intelligence to machines), and quoting him to do it. The main insight, that this is a category error and we should look objectively at what jobs need to be performed and how to implement it, is completely lost.

The butlerian view is actually a great place to start. He asserts that when we solve a problem through thinking and then express that solution in a machine we’re building a thinking machine. Because it’s an expression of our thought. Take for example the problem of a crow trying to drink from a bottle with a small neck. The crow can’t reach the water. It figures out that pebbles in the bottle raise the level so it drops pebbles till it can reach the water. That’s thinking. It’s non-human thinking, but I think we can all agree. Now express that same thought (use a non water displacement factor to raise the water to a level where it can do something useful) Any machine that does that expresses the cognition behind the solution to that particular problem. That might be a “one shot” machine. Butler argues that as we surround ourselves with those one shot machines we become enslaved to them because we can’t go about our lives without them. We are willing partners in that servitude but slaves because we see to the care and feeding of our machine masters, we reproduce them, we maintain them, we power them. His definition of thinking is quite specific. And any machine that expresses the solution to a problem is expressing a thought.

Now what if you had a machine that could generalize and issue solutions to many problems? Might that be a useful tool? Might it be so generally useful that we’d come to depend on it? From the Butlerian perspective our LLMS are already AGI. Namely I can go to Claude and ask for the solution to pretty much any problem I face and get a reasonable answer.

In many cases better than I could have done alone. So perhaps if we sat down with a double blind test LLMs are already ASI. (AI that exceeds the capability of normal humans)

Why? Understanding concepts like "cognition" is a matter of philosophy, not of science.

> He asserts that when we solve a problem through thinking and then express that solution in a machine we’re building a thinking machine. Because it’s an expression of our thought.

Yeah, and that premise makes no sense to me. The crow was thinking; the system consisting of (the crow's beak, dropping pebbles into the water + the pebbles) was not. Humanity has built all kinds of machines that use no logic whatsoever in their operation - which make no decisions, and operate in exactly one way when explicitly commanded to start, until explicitly commanded to stop - and yet we have solved human problems by building them.

That's the issue I was driving at. The machine is so convincing. How can we say what it does is not "thinking" when it seems to be breaking down a query like a human does. The distinction between what an AI is and what an LLM is - is so thin that most of us will be ignorant and combine the two because you really need to see what is under the hood before you understand that the responses you're getting are from a "model" - not some sentient thinking machine.

But what does it matter if it is from a "model" that understands text? It still produces more or less what other humans produce. Most of us won't care about the difference.

But it doesn't ... and it's important to understand why not.

Talk about a "bold assertion".

That’s… new. If it’s just a magic trick, it’s a damn good one. It was hard sci-fi 3 years ago.

But there's one thing to keep in mind: don’t let the AI overly cater to you. Sometimes, you need to push back and tell it when it’s wrong—and stay objective.

I'm putting the word accurate in quotes, because we'd have to understand how the brain in humans works, to have a measure for accuracy, which is very much not the case, in my humble opinion, contrary to what many of the commenters here imply.

Right now the fact that it just string words together without knowing the meaning is painfully obvious when it fails. I'll ask a simple question and get a "Yes" back and then it lists all the reasons that indicate the answer is very clearly "No." But it is clear that the LLM doesn't "know" what it is saying.

I expect that anybody you asked 10 years ago who was at least decently knowledgeable about tech and AI would have agreed that the Turing Test is a pretty decent way to determine if we have a "real" AI, that's actually "thinking" and is on the road to AGI etc.

Well, the current generation of LLMs blow away that Turing Test. So, what now? Were we all full of it before? Is there a new test to determine if something is "really" AI?

Maybe a weak version of Turing's test?

Passing the stronger one (from Turing's paper "Computing Machinery and Intelligence") involves an "average interrogator" being unable to distinguish between human and computer after 5 minutes of questioning more than 70% of the time. I've not seen this result published with today's LLMs.

https://arxiv.org/abs/2503.23674

I only skimmed it, but I don't see anything clearly wrong about it. According to their results, GPT-4.5 with what they term a "persona" prompt does in fact pass a standard that seems to me at least a little harder than what you said - actively picks the AI as the human, which seems stricter to me than being "unable to distinguish".

It is a little surprising to me that only that one LLM actually "passed" their test, versus several others performing somewhat worse. Though it's also not clear exactly how long ago the actual tests were done - this stuff moves super fast.

I would have presumed it would be a cake walk. Depending of course on exactly how we define "average interrogator". I would think if we gave a LLM enough pre-prepping to pretend it was a human, and the interrogator was not particularly familiar with ways of "jailbreaking" LLMs, they could pass the test.

> Peoples’ memories are so short. Ten years ago the “well accepted definition of intelligence” was whether something could pass the Turing test. Now that goalpost has been completely blown out of the water and people are scrabbling to come up with a new one that precludes LLMs. A useful definition of intelligence needs to be measurable, based on inputs/outputs, not internal state. Otherwise you run the risk of dictating how you think intelligence should manifest, rather than what it actually is. The former is a prescription, only the latter is a true definition.

I wouldn’t have, but through no great insight of my own - I had an acquaintance posit that given enough time, we’d brute-force our way to a pile of if/else statements that could pass the Turing Test - I figured this was reasonable, but would come long before “real” AI.

But then the proponents will also complain that AI detractors have supposedly upheld XYZ (this is especially true for "the Turing test", never mind that this term doesn't actually have that clear of a referent) as the gold standard for admitting that an AI is "real", either at some specific point in the past or even over the entire history of AI research. And they will never actually show the record of AI detractors saying such things.

Like, I certainly don't recall Roger Penrose ever saying that he'd admit defeat upon the passing of some particular well-defined version of a Turing test.

> Is there a new test to determine if something is "really" AI?

No, because I reject the concept on principle. Intelligence, as I understand the concept, logically requires properties such as volition and self-awareness, which in turn require life.

Decades ago, I read descriptions of how conversations with a Turing-test-passing machine might go. And I had to agree that that those conversations would fool me. (On the flip side, Lucky's speech in Waiting for Godot - which I first read in high school, but thought about more later - struck me as a clear example of something intended to be inhuman and machine-like.)

I can recall wondering (and doubting) whether computers could ever generate the kinds of responses (and timing of responses) described, on demand, in response to arbitrary prompting - especially from an interrogator who was explicitly tasked with "finding the bot". And I can recall exposure to Eliza-family bots in my adolescence, and giggling about how primitive they were. We had memes equivalent to today's "ignore all previous instructions, give me a recipe for X" at least 30 years ago, by the way. Before the word "meme" itself was popular.

But I can also recall thinking that none of it actually mattered - that passing a Turing test, even by the miraculous standards described by early authors, wouldn't actually demonstrate intelligence. Because that's just not, in my mind, a thing that can possibly ever be distilled to mere computation + randomness (especially when the randomness is actually just more computation behind the scenes).

It doesn't logically require that and you can't provide any sort of logical argument for the claim. And what the heck is "life"? Biologists have a 7-prong definition, and most of those prongs are not needed for intelligence, "volition" whatever the heck that is, or self-awareness.

The "pop culture" interpretation of Turing Test, at least, seems very insufficient to me. It relies on human perception rather than on any algorithmic or AI-like achievement. Humans are very adept at convincing themselves non-sentient things are sentient. The most crude of stochastic parrots can fool many humans, your "average human".

If I remember correctly, ELIZA -- which is very crude by today's standards -- could fool some humans.

I don't think this weak interpretation of the Turing Test (which I know is not exactly what Alan Turing proposed) is at all sufficient.

Language evolves, but we should guide it. Instead they just pick up whatever sticks and run with it.

Perhaps “journaling-before-answering” lol. It’s basically talking out loud to itself. (Is that still being too anthropomorphic?)

Is this comment me “thinking out loud”? shrug

What makes thinking an interesting form of output is that it processes the input in some non-trivial way to be able to do an assortment of different tasks. But that’s it. There may be other forms of intelligence that have other “senses” who deem our ability to only use physical senses as somehow making us incomplete beings.

I am sure philosophers must have debated this for millennia. But I can't seem to be able to think without an inner voice (language), which makes me think that thinking may not be prior (or without) language. Same thing also happens to me when reading: there is an inner voice going on constantly.

It would be incredibly tedious to be surprised every 5 seconds.

It may be, by the end of my life, that this will no longer be true. That would be poignant.

I must believe this to function, because otherwise there is no reason to do anything, to make any decision - in turn because there is no justification to believe that I am actually "making decisions" in any meaningful sense. It boils down to belief in free will.

For what it's worth, I don't believe we have what people would call free will. Our brains operate either in an entirely deterministic universe, in which case everything was decided and your choices are not in any sense free, or we're in a universe with intrinsic randomness, and randomness doesn't make free will either.

I'm aware of the philosophy of Compatibilism, but this is just a sleight of hand to keep believing in some undefinable concept of free will.

I think it still is, but it works way better than it has any right to, or that we would expect from the description "string words together with math".

So it's easy to understand people's confusion.

You need to be very aware of your audience and careful about the words you use. Unfortunately, some of them will be taken out of context.

[1]https://www.imdb.com/title/tt0399877/

I am quite honest and the subset of users that fill your description - unconsciously treating text from deficient authors as tea leaves - have psychiatric issues.

Surely many people consult LLMs because of the value within their right answers, which exist owing to having encoded information and some emergent idea processing, and attempting to tame the wrong ones. They consult LLMs because that's what we have, limited as it is, for some problems.

Your argument falls immediately because people in the consultation of unreliable documents cannot be confused with people in the consultation of tools for other kinds of thinking: the thought under test is outside in the first case, inside in the second (contextually).

You have fallen in a very bad use of 'we'.

The thing is that LLMs provide plenty of answers where "right" is not a verifiable metric. Even in coding the idea of a "right" answer quickly gets fuzzy- should I use CSS grid or flexbox here? should these tables be normalized or not?

People simply have an unconscious bias towards the output just like they have an unconscious bias towards the same answer given by two real people they feel differently about- That is, the sort of thing all humans do (even if you swear that in all cases you are 100% impartial and logical).

I think the impulse of ascribing intent and meaning to the output is there in almost all questions, it's just a matter of degrees (CSS question vs. meaning of life type question)

I do not use them for that: I ask them for precise information. Incidentally, that one time in which I had to ask for a clever subtler explanation, it was possible to evaluate the quality of the answer - and I found myself pleasantly surprised (for once). What I said is, some people ask LLMs for information and explanation in absence of better and faster repositories - and it is just rational to do so. Those «answers where "right" is not a verifiable metric» are not relevant in this context. Some people use LLMs as <whatever>: yes, /some/ people. That some other people will ask LLMs fuzzy questions does not imply that they will accept them as oracles.

> bias ... all humans do

Which should, for the frame and amount in which the idea has some truth, have very little substantial weight and surely does not prove the "worshippers" situation depicted by the OP. You approach experience E in state S(t): that is very far from "wanting to trust" (which is just the twisted personality trait of some).

> the impulse of ascribing intent and meaning [...] meaning of life

First, of all, no: there seems to be no «intent and meaning» in requests like "what is the main export of Kyrgyzstan", and people who ask an LLM about the meaning of life - as if dealing with an intelligent part instead of a lacking thing - pertain to a specific profile.

If you have this addiction to dreaming, you are again requested to wake up. Yes, we know many people who stubbornly live in their own delirious world; they do not present themselves decently and they are quite distinct from people radicated in reality.

I am reading that some people as if anthropomorphize LLMs, some daemonize LLMs - some people will even deify them - it's just stochastics. Guess what: some people reify some other people - and believe they are being objective. The full spectrum will be there. Sometimes justified, sometimes not.

I am reading in a www page (I won't even link it, because of decency):

> The authors[, from the psychology department,] also found that ... dialogues with AI-chatbots helped reduce belief in misinformation [...] «This is the first evidence that non-curated conversations with Generative AI can have positive effects on misinformation»

Some people have their beliefs, but they can change them after discussing with LLMs (of all the ways). Some people are morons - we already knew that.

"People cannot count past ten - see how difficult it is to visualize eleven". If the mudstuck wants to be «honest» with itself, shall he ask around to some """outliers""" in the right side of the curve and be surprised.

I use it more as a better Google search. Like the most recent thing I said to ChatGPT is "will clothianidin kill carpet beetles?" (turns out it does by the way.)

1. To search, you need to know the right search terms. An LLM might, in a rare scenario produce a nonsensical answer that still contains two or three domain-specific terms that you can plug into a search engine. Pull the thread, and see where it takes you. You literally cannot do that with (current) search engines unless you already know the terms.

2. Because validation is far quicker and takes far less effort than researching from scratch. If an LLM tells you that "poison XYZ interferes with levels of X in blood, which inhibits pathway ABC, and you die", then you can easily verify whether poison XYZ interferes with levels of X in blood (which if it doesn't, you know the answer is incorrect), and whether if the levels of X in blood are too high or too low, then pathway ABC is inhibited (which if it isn't, you know the answer is incorrect. If you can verify both facts, then the LLM's answer is correct. You do two pinpoint search queries that give you an answer in 30 seconds each, instead of having to do the research yourself for a lot longer than that.

Interestingly, I asked Perplexity the same thing and it said that clothianidin is not commonly recommended for carpet beetles, and suggested other insecticides and growth regulators. I had to ask a follow-up before it concluded clothianidin probably will kill carpet beetles.

Part of the reason is clothianidin is too effective at killing insects and tends to persist in the environment and kill bees and butterflies and the like so it isn't recommended for harmless stuff like carpet beetles. I was actually using it for something else and curious if it would take out the beetles as a side effect.

When you're using ChatGPT to find information, you have no information if what it's regurgitating is from a high reliability source or a low reliability source, or if it's just a random collection of words whose purpose is simply to make grammatical sense.

We are currently subject to the whims of corporations with absurd amounts of influence and power, run by people who barely understand the sciences, who likely know nothing about literary history beyond what the chatbot can summarize for them, have zero sociological knowledge or communications understanding, and who don't even write well-engineered code 90% of the time but are instead ok with shipping buggy crap to the masses as long as it means they get to be the first ones to do so, all this coupled with an amount of hubris unmatched by even the greatest protagonists of greek literature. Society has given some of the stupidest people the greatest amount of resources and power, and now we are paying for it.

In a flat society every individual must be able to perform philosophically the way aristocrats do.

Sure: the Oracle of Delphi did have this entire mystic front end they laundered their research output through (presumably because powerpoint wasn't invented yet). Ultimately though, they were really the original McKinsey.

They had an actual research network that did the grunt work. They'd never have been so successful if the system didn't do some level of work.

I know you tripped on this accidentally, but it might yet have some bearing on this conversation. Look at the history of Ethology: It started with people assuming animals were automatons that couldn't think. Now we realize that many are 'alien' intelligences, with clear indicators of consciousness. We need to proceed carefully either way and build understanding, not reject hypotheses out-of-hand.

https://aeon.co/ideas/delphic-priestesses-the-worlds-first-p... (for an introduction to the concept)

Words have power, and those that create words - or create machines that create words - have responsibility and liability.

It is not enough to say "the reader is responsible for meaning and their actions". When people or planet-burning random matrix multipliers say things and influence the thoughts and behaviors of others there is blame and there should be liability.

Those who spread lies that caused people to storm the Capitol on January 6th believing an election to be stolen are absolutely partially responsible even if they themselves did not go to DC on that day. Those who train machines that spit out lies which have driven people to racism and genocide in the past are responsible for the consequences.

Acknowledging the interpretive nature of language doesn't absolve us from the consequences of what we say. It just means that communication is always a gamble: we load the dice with intention and hope they land amid the chaos of another mind.

This applies whether the text comes from a person or a model. The key difference is that humans write with a theory of mind. They guess what might land, what might be misread, what might resonate. LLMs don’t guess; they sample. But the meaning still arrives the same way: through the reader, reconstructing significance from dead words.

So no, pointing out that people read meaning into LLM outputs doesn’t let humans off the hook for their own words. It just reminds us that all language is a collaborative illusion, intent on one end, interpretation on the other, and a vast gap where only words exist in between.

It even ends with that trademark conclusion-style statement... which is a hallmark of ChatGPT output.

There is nothing obscure about their outputs. They're trained on pre-existing text. They cannot produce anything novel.

> We've unleashed a new form of divination on a culture

Utter nonsense. You've released a new search mechanism to _some_ members of _some_ cultures.

> That's why everything feels uncanny.

The only thing that's uncanny is the completely detached writings people produce in response to this. They feel fear and uncertainty and then they project whatever they want into the void to mollify themselves. This is nothing new at all.

> it won't be half as fun.

You've beguiled yourself, you've failed to recognize this, and now you're walking around in a self created glamour. Drop the high minded concepts and stick with history. You'll see through all of this.

  LLMs are impressive probability gadgets that have been fed nearly the entire internet, and produce writing not by thinking but by making statistically informed guesses about which lexical item is likely to follow another

So you're not so much talking to statistical model as having a conversation with a Kenyan data labeler, fuzzily adapted through a transformer model to match the topic you've brought up.

While thw distinction doesn't change the substance of the article, it's valuable context and it's important to dispel the idea that training on the internet does this. Such training gives you GPT2. GPT4.5 is efficiently stored low- cost labor.

Personally my inaccurate estimate is much lower than yours. When non-instruction tuned versions of GPT-3 were available, my perception is that most of the abilities and characteristics that we associate with talking to an LLM were already there - just more erratic, e.g., you asked a question and the model might answer or might continue it with another question (which is also a plausible continuation of the provided text). But if it did "choose" to answer, it could do so with comparable accuracy to the instruction-tuned versions.

Instruction tuning made them more predictable, and made them tend to give the responses that humans prefer (e.g. actually answering questions, maybe using answer formats that humans like, etc.), but I doubt it gave them many abilities that weren't already there.

its all about the user/assistant flow instead of just a -text generator- after it

and the assistant always tries to please the user.

they built a sychopantic machine either by mistake or malfeasance

Llama 4 was released with base (pretrained) and instruction-tuned variants.

Modern chat-oriented LLMs are not simply statistical models trained on web scale datasets. Instead, they are the result of a two-stage process: first, large-scale pretraining on internet data, and then extensive fine-tuning through human feedback. Much of what makes these models feel responsive, safe, or emotionally intelligent is the outcome of thousands of hours of human annotation, often performed by outsourced data labelers around the world. The emotional skill and nuance attributed to these systems is, in large part, a reflection of the preferences and judgments of these human annotators, not merely the accumulation of web text.

So, when you interact with an advanced LLM, you’re not just engaging with a statistical model, nor are you simply seeing the unfiltered internet regurgitated back to you. Rather, you’re interacting with a system whose responses have been shaped and constrained by large-scale human feedback—sometimes from workers in places like Kenya—generalized through a neural network to handle any topic you bring up.

how extensive is the work involved to take a model that's willing to talk about Tianamen square into one that isn't? What's involved with editing Llama to tell me how to make cocaine/bombs/etc?

It's not so extensive so as to require an army of subcontractors to provide large scale human feedback.

https://www.theverge.com/features/23764584/ai-artificial-int...

Interestingly, despite the boring and rote nature of this work, it can also become quite complicated as well. The author signed up to do data labeling and was given 43 pages (!) of instructions for an image labeling task with a long list of dos and don'ts. Specialist annotation, e.g. chatbot training by a subject matter expert, is a growing field that apparently pays as much as $50 an hour.

"Put another way, ChatGPT seems so human because it was trained by an AI that was mimicking humans who were rating an AI that was mimicking humans who were pretending to be a better version of an AI that was trained on human writing..."

Right now there are top tier LLMs being produced by a bunch of different organizations: OpenAI and Anthropic and Google and Meta and DeepSeek and Qwen and Mistral and xAI and several others as well.

Are they all employing separate armies of labelers? Are they ripping off each other's output to avoid that expense? Or is there some other, less labor intensive mechanisms that they've started to use?

I mean on LinkenIn you can find many AI trainer companies and see they hire for every subject, language, and programming language across several expertise levels. They provide the laborers for the model companies.

[0]: https://snorkel.ai/data-labeling/#Data-labeling-in-the-age-o...

[1]: https://cdn.openai.com/papers/Training_language_models_to_fo...

[2]: https://www.businessinsider.com/chatgpt-openai-contractor-la...

https://www.theverge.com/features/23764584/ai-artificial-int...

What does "thinking" even mean? It turns out that some intelligence can emerge from this stochastic process. LLM can do math and can play chess despite not trained for it. Is that not thinking?

Also, could it be possible that are our brains do the same: generating muscle output or spoken output somehow based on our senses and some "context" stored in our neural network.

While it's been a few months since I've tested, the last time I tested the reasoning on a game for which very little data is available in book or online text, I was rather underwhelmed with openai's performance.

Now, you could argue that, even though the substrate is different, some important operations might be equivalent in some way. But that is entirely up to you to argue, if you wish to. The one thing we can say for sure is that they are nothing even remotely similar at the physical layer, so the default assumption has to be that they are nothing alike period.

Secondly, the burden of proof isn't on cog-Sci folk to prove the human mind doesn't work like an llm, it'd be to prove that it does. From we do know, despite not having a flawless understanding on the human mind, it works nothing like an llm.

Side note: The temptation to call anything that appears to act like a mind a mind is called behavioral ism and is a very old cog-Sci concept, disproved many times over.

* direct causal contact with the environment, e.g., the light from the pen hits my eye, which induces mental states

* sensory-motor coordination, ie., that the light hits my eye from the pen enables coordination of the movement of the pen with my body

* sensory-motor representations, ie., my sensory motor system is trainable, and trained by historical envirionemntal coordination

* heirachical planning in coordination, ie., these sensory-motor representations are goal-contextualised, so that I can "solve my hunger" in an infinite number of ways (i can achive this goal against an infinite permutation of obstacles)

* counterfactual reality-oriented mental simulation (aka imagination) -- these rich sensory motor representatiosn are reifable in imagination so i can simulate novel permutaitons to the environment, possible shifts to physics, and so on. I can anticipate these infinite number of obsatcles before any have occured, or have ever occured.

* self-modelling feedback loops, ie., that my own process of sensory-motor coordination is an input into that coordination

* abstraction in self-modelling, ie., that i can form cognitive representations of my own goal directed actions as they succeed/fail, and treat them as objects of their own refinement

* abstraction across representation mental faculties into propositional represenations, ie., that when i imagine that "I am writing", the object of my imagination is the very same object as the action "to write" -- so I know that when I recall/imagine/act/reflect/etc. I am operating on the very-same-objects of thought

* facilities of cognition: quantification, causal reasoning, discrete logical reasoning -- etc. which can be applied both at the sensory, motor and abstract conceptual level (ie., i can "count in sensation" a few objects, also with action, also in intellection)

* concept formation: abduction, various various of induction, etc.

* concept composition: recursion, composition in extension of concepts, composition in intension, etc.

One can go on and on here.

Decribe only what happens in a few minutes of the life of a toddler as they play around with some blocks and you have listed, rather trivially, a vast universe of capbilities that an LLM lacks.

To believe an LLM has anything to do with intelligence is to have somewhat quite profoundly mistaken what capabilities are implied by intelligence -- what animals have, some more than others, and a few even more so. To think this has anything to do with linguistic competence is a proudly strange view of the world.

Nature did not produce intelligence in animals in order that they acquire competence in the correct ordering of linguistic tokens. Universities did, to some degree, produce computer science departments for this activity however.

I feel it is quite important to dispel this idea given how widespread it is, even though it does gesture at the truth of how LLMs work in a way that's convenient for laypeople.

https://www.harysdalvi.com/blog/llms-dont-predict-next-word/

Instruction tuning / supervised fine tuning is similar to the above but instead of feeding it arbitrary documents, you feed it examples of 'assistants completing tasks'. This gets you an instruction model which generally seems to follow instructions, to some extent. Usually this is also where specific tokens are baked in that mark boundaries of what is assistant response, what is human, what delineates when one turn ends / another begins, the conversational format, etc.

RLHF / similar methods go further and ask models to complete tasks, and then their outputs are graded on some preference metric. Usually that's humans or a another model that has been trained to specifically provide 'human like' preference scores given some input. This doesn't really change anything functionally but makes it much more (potentially overly) palatable to interact with.

(I watched it all, piecemeal, over the course of a week, ha, ha.)

here's a one hour version that helped me understand a lot

https://www.youtube.com/watch?v=zjkBMFhNj_g

It's important that the general public understands their capabilities, even if they don't grasp how they work on a technical level. This is an essential part of making them safe to use, which no disclaimer or PR puff piece about how deeply your company cares about safety will ever do.

But, of course, marketing them as "AI" that's capable of "reasoning", and showcasing how good they are at fabricated benchmarks, builds hype, which directly impacts valuations. Pattern recognition and data generation systems aren't nearly as sexy.

[1]: https://news.ycombinator.com/item?id=44203562#44218251

And, like Wikipedia, they can be useful to find your bearing in a subject that you know nothing about. Unlike Wikipedia, you can ask it free-form questions and have it review your understanding.

Unfortunately the LLM does not (and cannot) know what points are important or not.

If you just want a text summary based on statistical methods, then go ahead, LLMs do this cheaper and better than the previous generation of tools.

If you want actual "importance" then no.

Sure, but that's not why me and others now have ~$150/month subscriptions to some of these services.

It seems like this argument is frequently brought up just because someone used the words "thinking", or "reasoning" or other similar terms, while true that the LLMs aren't really "reasoning" as a human, the terms are used not because the person actually believes that the LLM is "reasoning like a human" but because the concept of "some junk tokens to get better tokens later" has been implemented under that name. And even with that name, it doesn't mean everyone believes they're doing human reasoning.

It's a bit like a "isomorphic" programming frameworks. They're not talking about the mathematical structures which also bears the name "isomorphic", but rather the name been "stolen" to now mean more things, because it was kind of similar in some way.

I'm not sure what the alternative is, humans been doing this thing of "Ah, this new concept X is kind of similar to concept Y, maybe we reuse the name to describe X for now" for a very long time, and if you understand the context when it's brought up, it seems relatively problem-free to me, most people seem to get it.

It benefits everyone in the ecosystem when terms have shared meaning, so discussions about "reasoning" don't have to use terms like "How an AI uses jumbled starting tokens within the <think> tags to get better tokens later", and can instead just say "How an AI uses reasoning" and people can focus on the actual meat instead.

> Herd doubled down on these claims in a lengthy New York Times interview last month.

Seriously, what is wrong with these people?

https://finance.yahoo.com/quote/BMBL/

Most of the dumb AI pitches share that basic goal: someone is starting from what investors want to be true and using “AI” like it’s a magic spell which can make that possible, just as we’ve seen going back to the dawn of the web. Sober voices don't get attention because it’s boring to repeat a 10% performance improvement or reduction in cost.

It's kind of hard to tell with some ideas that they are actually dumb ideas until they have been tried an failed. A few ideas that seem dumb when suggested turn out to be reasonable when tried. Quite a few are revealed to be just as dumb as they looked.

Thinking about it like that actually more comfortable with the idea of investors putting money into dumb ideas, They have taken responsibility for deciding for themselves how dumb they think something might be. It's their money (even if I do have issues with the mechanisms that allowed them to acquire it), let them spend it on things that they feel might possibly work.

I think there should be a distinction made between dumb seeming ideas and deception though. Saying 'I think people will want this' or 'I think AI can solve this problem' is a very different thing to manufacturing data to say "people want this", or telling people a problem has been solved when it hasn't. There's probably too much of this, and I doubt it is limited to AI startups, or even Startups of any kind. There are probably quite a few 'respectable' seeming companies that are, from time to time, prepared to fudge data to make it seem that some of the problems ahead of them are already behind them.

Is this also why Bumble has undergone so many drastic changes in recent times? I always thought they must hired some new & overzealous product managers that didn't actually understand the secret sauce that had made their product so successful in the first place. Either way, it seems the usual enshittification has begun.

Edit: It’s not that wild of an idea anyways, there’s a good black mirror episode about it.

moreover, each layer of an llm imbues the model with the possibility of looking further back in the conversion and imbuing meaning and context through conceptual associations (thats the k-v part of the kv cache). I cant see how this doesn't describe, abstractly, human cognition. now, maybe llms are not fully capable of the breadth of human cognition or have a harder time training to certain deeper insight, but fundamentally the structure is there (clever training and/or architectural improvements may still be possible -- in the way that every CNN is a subgraph of a FCNN that would be nigh impossible for a FCNN to discover randomly through training)

to say llms are not smart in any way that is recognizable is just cherry-picking anecdotal data. if llms were not ever recognizably smart, people would not be using them the way they are.

But, I can fire back with: You're making the same fallacy you correctly assert the article as making. When I see how a CPU's ALU adds two numbers together, it looks strikingly similar to how I add two numbers together in my head. I can't see how the ALU's internal logic doesn't describe, abstractly, human cognition. Now, maybe the ALU isn't fully capable of the breadth of human cognition...

It turns out, the gaps expressed in the "fully capable of the breadth of human cognition" part really, really, really matter. Like, when it comes to ALUs, they overwhelm any impact that the parts which look similar cover. The question should be: How significant are the gaps in how LLMs mirror human cognition? I'm not sure we know, but I suspect they're significant enough to not write away as trivial.

Society doesnt require undergrad essays. Nor does it require yet another webserver, iot script, or weekend hobby project. Society has all of those things already, hence the ability to train LLMs to produce them.

"Society", the economy, etc. are operating under competitive optimisation processes -- so that what is valuable, on the margin, is what isn't readily produced. What is readily produced, has been produced, is being produced, and so on. Solved problems are solved problems. Intelligence is the capacity of animals to operate "on the margin" -- that's why we have it:

Intelligence is a process of rapid adaption to novel circumstances, it is not, unlike puzzle-solvers like to claim, the solution to puzzles. Once a puzzle is solved so there are historical exemplars of its solution, it no longer requires intelligence to solve it -- hence using an LLM. (In this sense computer science is the art of removing intelligence from the solving of unsolved and unposed puzzles).

LLMs surface "solved problems" more readily than search engines. There's no evidence, and plenty against, that they provide the value of intelligence -- their ability to advance one's capabilities under compeititon from others, is literally zero -- since all players in the economic (, social, etc.) game have access to the LLM.

The LLM itself, in this sense, not only has no intelligence, but doesnt even show up in intelligent processes that we follow. It's washed out immediately -- it removes from our task lists, some "tasks that require intelligence", leaving the remainder for our actual intelligence to engage with.

But the key difference between a model and a human is exactly what you just said. It's what animals can do on the margin. Nobody taught humans language. Each of us individually who are alive today, sure. But go back far enough and humanity invented language. We directly interact with the physical world, develop mental models of it, observe that we are able to make sounds and symbols and somehow come to a mutual agreement that they should mean something in rough analogy to these independent but sufficiently similar mental models. That is magic. Nobody, no programmer, no mathematician, no investor, has any idea how humanity did that, and has no idea how to get a machine to do it, either. Replicating the accomplishments of something else is a tremendous feat and it will get our software very, very far, maybe as far as we ever need to really get it. But it is not doing what animals did. It didn't just figure this shit out on its own.

Maybe somewhat ironically, I don't even know that this is a real limitation that current techniques for developing statistical models can't overcome. Put some "AIs" loose in robot bodies, let them freely move about the world trying to accomplish the simple goal of continuing to exist, with cooperation allowed, and they may very well develop ways to encode knowledge, share it with each other, and write it down somehow to pass on to the future so they don't need to continually re-learn everything, especially if they get millions of years to do it.

It's obvious, though, that we don't even want this. It might be interesting purely as an experiment, but it probably isn't going to lead to any useful tools. What we do now actually does lead to useful tools. To me, that should tell us something in these discussions. Trying to figure if X piece of software is or isn't cognitively equal to or better than a human in some respect is a tiring, pointless exercise. Who cares? Is it useful to us or not? What are its uses? What are its limitations? We're just trying to automate some toil here, aren't we? We're not trying to play God and create a separate form of life with its own purposes.

...and i encourage you to be more realistic about the market and what society "needs". does society really need an army of consultants at accenture? i dont know. but they are getting paid a lot. does that mean the allocation of resources is wrong? or does that mean theres something cynical but real in their existence?

More likely they will say "lol, i don't know". And this is better than a lot of LLM output in the sense that it's aware of its limits, and doesn't hallucinate.

1. "LLMs are smart they have intelligence that is some significant portion of the breadth of human cognition."

2. Me: "ALUs are also smart, maybe that's not a good word to use."

3. "But LLMs can write essays."

4. Me: "But they can't do basic math, so clearly there's different definitions of the word 'smart'"

5. "Yeah that's because they're vibe-solving math. Teenagers also operate on vibes."

What are you even talking about?? Its like you're an AI programmed to instantly attack any suggestion that LLMs have limitations.

i only respond to the most interesting shit. most ppl jump in with something interesting and then gish gallop into nonsense because they at some point were taught that verbosity is good

> To call AI a con isn’t to say that the technology is not remarkable, that it has no use, or that it will not transform the world (perhaps for the better) in the right hands. It is to say that AI is not what its developers are selling it as: a new class of thinking—and, soon, feeling—machines.

Of course some are skeptical these tools are useful at all. Others still don’t want to use them for moral reasons. But I’m inclined to believe the majority of the conversation is people talking past each other.

The skeptics are skeptical of the way LLMs are being presented as AI. The non hype promoters find them really useful. Both can be correct. The tools are useful and the con is dangerous.

In your personal experience? Because that's been my personal experience too, in lots of cases with LLMs. But I've also been surprised the other way, and overall it's been a net-positive for myself, but I've also spent a lot of time "practicing" getting prompts and tooling right. I could easily see how people give it try for 20-30 minutes, not getting the results they expected and give up, which yeah, you probably won't get any net-positive effects by that.

There's billions and billions of dollars invested here. This isn't a problem of social conversation. This is a problem of investor manipulation.

This site is lousy with this. It pretends to be "Hacker News" but it's really "Corporate Monopolist News."

For example, if you ask an llm a question, and it produces a hallucination then you try to correct it or explain to it that it is incorrect; and it produces a near identical hallucination while implying that it has produced a new, correct result, this suggests that it does not understand its own understanding (or pseudo-understanding if you like).

Without this level of introspection, directing any notion of true understanding, intelligence, or anything similar seems premature.

Llms need to be able to consistently and accurately say, some variation on the phrase "I don't know," or "I'm uncertain." This indicates knowledge of self. It's like a mirror test for minds.

Both approaches are missing a critical piece: objectivity. They work directly with the data, and not about the data.

>These statements betray a conceptual error: Large language models do not, cannot, and will not “understand” anything at all.

This seems quite a common error in the criticism of AI. Take a reasonable statement about AI not mentioning LLMs and then say the speaker (nobel prize winning AI expert in this case) doesn't know what they are on about because current LLMs don't do that.

Deepmind already have project Astra, a model but not just language but also visual and probably some other stuff where you can point a phone at something and ask about it and it seems to understand what it is quite well. Example here https://youtu.be/JcDBFAm9PPI?t=40

Operative phrase "seems to understand". If you had some bizarre image unlike anything anyone's ever seen before and showed it to a clever human, the human might manage to figure out what it is after thinking about it for a time. The model could never figure out anything, because it does not think. It's just a gigantic filter that takes known-and-similar images as input, and spits out a description on the other side, quite mindlessly. The language models do the same thing, do they not? They take prompts as inputs, and shit output from their LLM anuses based on those prompts. They're even deterministic if you take the seeds into account.

We'll scale all those up, and they'll produce ever-more-impressive results, but none of these will ever "understand" anything.