Hacker News new | past | comments | ask | show | jobs | submit login

AI isn't really statistics at this point either though. The primitives are statistics tools, but parameter dynamics and macro-scale behavior of these models are very much their own area of study.



No, it's still very much at the intersection of optimization theory and statistics/probability.


In the last year or so I've seen a big shift upwards in the focus of papers. Academics don't have the resources to beat Meta at building foundation models, so the low hanging fruit is in understanding the behavior of existing models more deeply, and how to extend or leverage them in new ways.

Within a few years the fraction of papers in AI about new architectures, training, hyperparamter optimization, etc will be dwarfed by papers about things like controlnets, LoRA combinators, multi-model dispatch networks and few-shot embedding methods.


If we're talking about "AI" as a nebulous catch-all and arbitrary label for whatever hype people are chasing, then sure, you're free to define it however you like, by construction.

Popularity and status quo doesn't change the definition of the underlying theory. I will push back forever on some demotion of the importance of ML (i.e. theory) as distinct from some hype-driven notion of AI.


It's not really a hype-driven notion. The underlying machinery is statistical models, but those models are being trained into higher order structures which are worthy of research in their own right due to emergent properties, and eventually work in understanding and controlling those emergent properties will be more important than foundational ML advances.


I am sympathetic to this idea of emergence creating a new field of AI as distinct from the statistics that underlies it.

Then the focus should be to identify something like a fundamental unit of intelligence in order to formalize a higher-order science out of the foundations. An analogy can be drawn between physics and chemistry: we needed to properly identify "the atom" and its component parts to get anywhere with the science of chemistry. But it took a whole lot of physics to get to that point. It seems similar with the ML-cum-AI transition where we'll still need to dig very deep into statistics and information theory before being able to abstract them away in favor of higher-order concepts.

To me it seems we're really far from anything like that yet. Like at least a couple decades if not more. Friston's got some cool ideas that make me think he may have his name on some stuff later on but again the theory on learning systems is barely getting started.


It sounds like that’s the dividing line. Generating a new foundational model is CS+engineering, doing research on such models or derivatives thereof which treats them as alien artefacts to be studied is “AI.”




Join us for AI Startup School this June 16-17 in San Francisco!

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: