> > 'The same language' is usually just a desguised nationalistic claim
> It's the opposite: "it's a different language" is usually just a nationalistic desire for differentiation of what are essentially dialects/variants of a language.
It's both. The idea that Ukrainian is an uneducated farmer's dialect of Russian is a common talking point in the "Greater Russia / Russkiy Mir" narrative. Conversely, asserting the status of the Ukrainian language is a big part of Ukrainian identity in the face of an imperial invasion.
That's not true in practice. Floating point arithmetic is not commutative due to rounding errors, and the parallel operations introduce non-determinisn even at temperature 0.
It's pretty important when discussing concrete implementations though, just like when using floats as coordinates in a space/astronomy simulator and getting decreasing accuracy as your objects move away from your chosen origin.
What? You can get consistent output on local models.
I can train large nets deterministically too (CUBLAS flags). What your saying isn't true in practice. Hell I can also go on the anthropic API right now and get verbatim static results.
"Hell I can also go on the anthropic API right now and get verbatim static results."
How?
Setting temperature to 0 won't guarantee the exact same output for the exact same input, because - as the previous commenter said - floating point arithmetic is non-commutative, which becomes important when you are running parallel operations on GPUs.
Shouldn't it be the fact that they're non-associative? Because the reduction kernels will combine partial results (like the dot‑products in a GEMM or the sum across attention heads) in a way that the order of operations may change (non-associative), which can lead to the individual floats to be round off differently.
It's also the way the model runs. Setting temperature to zero and picking a fixed seed would ideally result in deterministic output from the sampler, but in parallel execution of matrix arithmetic (eg using a GPU) the order of floating point operations starts to matter, so timing differences can produce different results.
Good point. Though sampling generally happens on the CPU in a linear way. What you describe might influence the raw output logits from a single LLM step, but since the differences are only tiny, a well designed sampler could still make the output deterministic (so same seed = same text output). With a very high temperature these small differences might influence the output though, since the ranking of two tokens might be swapped.
I think the usual misconception is to think that LLM outputs are random "by default". IMHO this apparent randomness is more of a feature rather than a bug, but that may be a different conversation.
Mike Israetel is very vocal and open about both his own PED use and its health risks, and how taking them is a very bad idea for 99.9% of people. That does not make his content about exercising and nutrition for general health any less superb.
Steroids a fact of life in strength sports and not even really considered cheating, as long as you compete in "untested" divisions. Training for a top-level competition is not the same as exercising for health, even if you don't take PEDs. The people doing it consider it an acceptable tradeoff.
The original article is encouraging that even little bit of daily exercise is great. But the links are about doing conventional gym stuff.
If you take steroids your body will respond different then normal people. Using steroids and then to promote your programs that act like it's possible to achieve them without is dangerous.
The strain they will be able to put on their bodies are not the same as without. Even with steroids the strain is so high it can lead to serious life changing injuries like at least one the some of the linked have found out. There many young people that have used steroids and have completely worn out joints.
Obviously since some of them dance around the topic, and won't say outright say it. We just have the occasional hiccups during interviews where they give it away.
naturals shouldn't take advice from people that juiced up. The amount of volume they can do bizarre, and way more then naturals can do. The wear and tear on the ligaments, joints and spine is dangerous.
It's even dangerous for them, at least one of them got very badly hurt.
Genetic mutations and recombinations, as has been known for centuries. There is no need to make up some overarching esoteric narrative out of thin air that has people on the losing side of the lottery somehow "deserve" it.
That's a huge generalization for something that differs so heavily even within countries. Cities vs rural, east vs west, catholic vs protestant... Compare Poland and the Czech Republic, both V4 countries, neighbors, former Eastern Bloc, yet one is pretty much the least and the other one of the most religious countries in Europe.
I mean, this is just trivially wrong on a basic factual level. Look at ants, look at bees, even some mammals like mole-rats.
Not that these "biological facts" argument ever hold any water for complex social issues, but would you mind at least using actual facts?