Yes, gold is very similar, but it has the benefit of being centuries old and not dependent on complex infrastructure.
Specifically, there is value in a global peer to peer and agreed upon standard of exchange of a guaranteed scarce resource that can't be double spent, such as gold and some cryptos.
Imagine a war, a natural catastrophe, societal collapse or upheaval.
You have to pack up, go elsewhere, or you're suddenly occupied.
Basically you have to ask yourself, what's more likely to be worth something in the future or elsewhere that I can park my money into until I need it?
Since Gold has such a history of being accepted by various cultures and people around the world, and it is very resilient, it doesn't require power, infra, computers, nodes, won't get easily destroyed to environmental incidents, can be stashed away for centuries without degrading, etc. It is arguably more likely to still be used as an exchange of value in the future.
Crypto, well, you have to be specific, let's say Bitcoin BTC, how likely is it that your wallet on your hard drive if you migrate from a war and find yourself in a new world order at the other end of the world, you can still use them to trade for goods/services and they're worth close too or more of what they were before?
It's hard to predict, but arguably it seems less resilient than Gold and therefore less likely for it to hold its value over time. That said, it may still appear better than USD, Euros, or shares in some company, etc.
That's why people say BTC is a "store of value", like gold. You use it to stow away value for when you need it later (even generations later), because it appears to be good at holding value even through geopolitical shifts, passage of time, and so on.
But, if people aren't actually using it for storing value, but instead for speculative bets, it means they are taking money out of it and not leaving it in, it becomes volatile, and volatility is a bad "store of value", because when you might need the value if it's at a "low" it's gone, and it failed at the use case.
If you go outside BTC, it becomes even less likely the other cryptos are good stores of value, and more and more they become speculative bets and a game of chicken.
And even BTC has high volatility and is used for speculative bets a lot. And gold isn't immune to his either.
There's no good answer here, nobody knows the future for sure, but that's the idea.
When people defend crypto as a store of value, now you know what they mean. They're basically hoping it'll hold value through borders, time, and so on.
Well, most LLM are fine tuned over higher quality data, this is kind of how they've kept improving them amongst other things.
The first pass is to learn the fundamentals of language, and then it is refined on curated datasets, so you could refine them on high quality curated C code.
This is treating the LLM like it is the computer or has some kind of way of thinking. But LLM is a "language" model, I'm pretty sure the easier for human to read, the easier for LLM to learn and generate. Abstractions also benefit the model, it does not need to generate a working 2s complement, just a working call to addition of abstracted types.
And just in my experience, I feel everyone is slowly learning, all models are better at the common thing, they are better at bash, they are better at Python and JS, and so on. Everyone trying to invent at that layer has failed to beat that truth. That bootstrapping challenge is dismissed much too easily in the article in my opinion.
Binary bits are also a language. A structured language that transistor-based computers execute into some result we humans find valuable. Why wouldn't a model be able to write these binary instructions directly? Why do we need all these layers in between? We don't.
Because the learned function to generate binary code is likely more complex than that for Python.
I admit I can't say for sure until we try it. If someone were to train a model at the same scale on the same amount of raw binary code as we do these models on raw language and code, would it perform better at generating working programs. Thing is, it would now fail to understand human language prompts.
From what I know and understand though, it seems like it would be more complex to achieve.
My meta point is, you shouldn't think of it as what would a computer most likely understand, because we're not talking about a CPU/GPU. You have to think, what would a transformer architecture deep neural net better learn and infer? Python or binary code? And I think from that lens it seems more likely it's Python.
Why would you think its more complex? There are less permutations of generating transistor on/off states than there are all the different programming languages in use that result in the exact same bits.
Who said that creating bits efficiently from English to be computed by CPUs or GPUs must be done with transformer architecture? Maybe it can be, maybe there are other ways of doing it that are better. The AI model architecture is not the focus of the discussion. It is the possibilities of how it can look like if we ask for some computation, and that computation appears without all the middle-men layers we have right now, English->Model->Computation, not English->Model->DSL->Compiler->Linker->Computation.
Binary code takes more space, and both training and inference is highly capped by memory and context sizes.
Models tokenize to a limited set of tokens, and then learn relations between those. I can't say for sure, but I feel it be more challenging to find tokenization schemes for binary code and learn their relationships.
The model needs to first learn human language really well, because it has to understand the prompt and map it accurately to the binary code. That means the corpus will need to include a lot of human languages that it learns and also binary code, I wonder if the fact they differ so much would conflict the learning.
I think coming up with a corpus of mapped human language to binary code will be really challenging. Unless we can include the original code's comments at appropriate places around the binary code and so on.
Binary code is machine dependent, so it would result in programs that aren't portable between architecture and operating system and so on. The model would need to learn more than one binary code and be able to accurately generate the same program for different target platforms and OS.
> Who said that creating bits efficiently from English to be computed by CPUs or GPUs must be done with transformer architecture?
We've never had any other method ever do as well and by a magnitude. We may invent a whole new way in the future, but as of now, it's the absolute best method we've ever figured out.
> The AI model architecture is not the focus of the discussion. It is the possibilities of how it can look like if we ask for some computation, and that computation appears without all the middle-men layers we have right now, English->Model->Computation, not English->Model->DSL->Compiler->Linker->Computation.
Each layer simplifies the task of the layer above. These aren't like business layer that take a cut of the value out at each level, software layers remove complexity from the layers above.
I don't know why we wouldn't be talking about AI models? Isn't the topic that it may be more optimal for an AI model to be trained on binary code directly and to generate binary code directly? At least it's what I was talking about.
So if I stick to AI models. With LLMs and image/video diffusion and such, we've already observed that inference through smaller steps and chains of inference work way better. Based on that, I feel it's likely going from human language to binary code in a single hop to also work worse.
Diffusion models for images are already pretty much binary code generators. And we don't need to treat each bit individually, even in binary code there are whole segments that can be tokenized into a single token.
Regarding training, we have many binaries all around us, for many of them we also have the source code in whichever language. As a first step we can use the original source code and ask a third party model to explain what it does in English. Then use this English to train the binary programmer model. Eventually the binary programmer model can understand binaries directly and translate them to English for its own use, so with time, we might not even need binaries that have source code, we could narrate binaries directly.
It is basically a big game of hot potato. If you are OK with that, fine, but you are still just gambling on a party game chip.
There is a little bit of real value in having a rail that is hard to block, so you can move money across borders or around account freezes, sanctions, or runaway inflation. But that probably only justifies a tiny slice of the current price.
Most of the rest is speculation.
Fiat is different. A national currency is tied to the economy that uses it. Wages, rent, groceries, taxes, all live in that unit. Even if it crashes on FX markets you can still use it inside that country to buy local goods and services, and over time prices and wages shift along with it. The currency may fall against the dollar, but the local cost of living in that currency moves with it too.
The dream was that crypto gets adopted economically in a similar fashion, and even globally, that simply didn't happen, so it ended up just being gambling.
I think the missing piece is that "more expressive" languages do not automatically create more value at the team or company level.
Languages like Lisp, Clojure, Rust, Haskell, Erlang give strong engineers room to build powerful abstractions, but they also increase cognitive load and ramp up cost. In most orgs with churn and constant hiring, you do not get to enjoy "we built great abstractions and now we are fast". You live in "someone new is trying to understand what the last person did".
That is why hand holding and guard rails win. Not because Python or similar are technically superior, but because they support a commoditised, fungible workforce. Even if a wizard in a high dimension language is 2x more productive, that does not necessarily beat a slightly larger team in a mainstream language once you factor in turnover and ramp up. Companies mostly optimise for business impact, predictable delivery, and ease of staffing, not for maximising the ceiling of the top few programmers.
That said, at the individual level, as a programmer, you definitely can benefit from learning and mastering those added dimensions, even if you are to never use them again professionally, they expand your mindset.
> Languages like Lisp, Clojure, Rust, Haskell, Erlang give strong engineers room to build powerful abstractions ... You live in "someone new is trying to understand what the last person did".
In a language like C++, Rust or even Haskell, those "powerful abstractions" are mostly about building libraries, not application code. You still benefit from powerful libraries that were built as part of the ecosystem, even while keeping your own application code simple and intuitive to ease onboarding of new coders. Part of the "powerful abstractions" is to enable a "hand-holding and guard rails" approach for application code that's going to interface with that full-featured library.
I always felt simplifying all this, it would probably be possible to consolidate and offer proper health care, and a real welfare, along with a supplemental program for those inapt to work, with it all costing less than the mountain of piled complexity and paper work that we have today.
The system in its current insane state serves the purposes of multiple groups of people:
1. The bureaucrats administering the system because the more complex and insan the system is, the more bureaucrats you need.
2. The politicians who can promise $X of benefits to N people while knowing all along that due to the complexity of the system the actual cost of the program will be well under X*N due to failure to claim the benefits.
3. Companies that can now hire workers for less than their cost of living because they know they can just make it up out of government benefits.
And it hurts, of course, the poor people who depend on these programs. But that doesn't matter politically, because these people are mostly not smart enough to even figure out how they are being screwed. Whereas the people in 1-3 above are absolutely smart enough to figure out how they benefit from the current system and they vote and donate accordingly.
3. has always bothered me - if you run a McDonalds, you have to pay market price for the building, kitchen equipment, both when buying and maintaining it. If you don't pay enough to maintain the fridge, it won't unionize - it will break down.
The idea proposed by these 'free market types', that somehow people working there below sustenance salaries 'deserve it' - that the cost of labor needs to be essentially subsidized just doesn't make sense to me.
Practice what you preach - either pay the wages people need to get by, or close down.
Agreed, 3 is the one that consistently annoys me in arguments I see across the internet. The reason why our welfare programs exist and are in such a disasterous place is partially because they are literal corporate subsidies so they can buy workers at a premium. If you were to kick out the ladder and fully remove the minimum wage and remove our welfare systems it would outright collapse the economy because A) working would be a full net negative in income generation and B) corporations would crash and burn realizing that they can no longer hire workers at their subsidized price.
The sane way to approach a system like this is just UBI + socialized healthcare but I think we'll approach labor collapse before we see movement in this direction by out of touch bureaucrats.
I wouldn't go so far - I would just raise the minimum wage to a level where a person can actually make ends meet.
Sure some folks would be fired, and their jobs automated, but they could be redirected towards better paying opportunities.
If all else fails, then the money saved by such approaches could be used for welfare. It's just putting things from one pocket into the other for the government.
As things stand, these corporate subsidies are just for enshrining inefficiencies in the system.
I don't think the article argues for superior solutions, but I understand how it can feel as such.
I think it's just trying to say there's another dimension, the meta-level enabled by macros and Lisp's syntax that opens up the possibility of new solutions, which may or may not be better, as that's so context dependent.
But what I feel it's saying is you can't even begin to imagine solutions that leverage this new dimension without learning about it and getting to grip with it mentally.
In that sense, it's saying when you don't know, you can't even explore the space of solutions at that higher dimension, not necessarily that they're better for all problems.
I take Lisp more like artisanal work. It actually requires more skill and attention to use, but in good hands it can let someone really deliver a lot quickly.
That said, like in anything else, this kind of craftsmanship doesn't translate to monetization and scale the markets demands. What markets want is to lower barrier for entry, templatize, cheapen things, and so on.
It's normal then that languages optimized for the lowest common denominator, with less expressive power and more hand holding have won in popularity in enterprise and such, where making money is the goal, but that Lisp remains a strong and popular language for the enthousiasts looking to level up their craft or just geek out.
I think that's where OP might have realized that, the ideal hasn't happened and it's use case is just a game of chicken.