Nor is that inequality an oddity at all. If you were to think NaN should equal NaN, that thought would probably stem from the belief that NaN is a singular entity which is a misunderstanding of its purpose. NaN rather signifies a specific number that is not representable as a floating point. Two specific numbers that cannot be represented are not necessarily equal because they may have resulted from different calculations!
I'll add that, if I recall correctly, in R, the statement NaN == NaN evaluates to NA which basicall means "it is not known whether these numbers equal each other" which is a more reasonable result than False.
> "it is not known whether these numbers equal each other"
Equality, among other operations, are not defined for these inputs. NaN's really are a separate type of object embedded inside another objects value space. So you get the rare programmers gift of being able to construct a statement that is not always realizable based solely on the values of your inputs.
It's the only "primitive type" that does that. If I deserialize data from wire, I'll be very surprised when the same bits deserialize as unequal variables. If it cannot be represented, then throwing makes more sense than trying to represent it.
Other primitive types also do this, but this is not clearly visible from high-level programming languages, because most HLLs have only incomplete support for the CPU hardware.
If you do a (signed) integer operation, the hardware does not fit the result in a register of the size expected in a HLL, but the result has some bits elsewhere, typically in a "flags" register.
So the result of an integer arithmetic operation has an extra bit, usually named as the "overflow" bit. That bit is used to encode a not-a-number value, i.e. if the overflow bit is set, the result of the operation is an integer NaN.
For correct results, one should check whether the result is a NaN, which is called checking for integer overflow (unlike for FP, the integer execution units do not distinguish between true overflow and undefined operations, i.e. there are no distinct encodings for infinity and for NaN). After checking that the result is not a NaN, the extra bit can be stripped from the result.
If you serialize an integer number for sending it elsewhere, that implicitly assumes that wherever your number was produced, someone has tested for overflow, i.e. that the value is not a NaN, so the extra bit was correctly stripped from the value. If nobody has tested, your serialized value can be bogus, the same as when serializing a FP NaN and not checking later that it is a NaN, before using one of the 6 relational operators intended for total orders, which may be wrong for partial orders.
It is not an IEEE 754 oddity. It is the correct mathematical behavior.
When you define an order relation on a set, the order may be either a total order or a partial order.
In a totally ordered set, there are 3 values for a comparison operation: equal, less and greater. In a partially ordered set, there are 4 values for a comparison operation: equal, less, greater and unordered.
For a totally ordered set you can define 6 relational operators (6 = 2^3 - 2, where you subtract 2 for the always false and always true predicates), while for a partially ordered set you can define 14 relational operators (14 = 2^4 - 2).
For some weird reason, many programmers have not been taught properly about partially-ordered sets and also most programming languages do not define the 14 relational operators needed for partially ordered sets, but only the 6 relational operators that are sufficient for a totally ordered set.
It is easy to write all 14 relational operators by combinations of the symbols for not, less, greater and equal, so parsing this in a programming language would be easy.
This lack of awareness about partial order relations and the lack of support in most programming languages is very bad, because practical applications need very frequently partial orders instead of total orders.
For the floating-point numbers, the IEEE standard specifies 2 choices. You can either use them as a totally-ordered set, or as a partially-ordered set.
When you encounter NaNs as a programmer, that is because you have made the choice to have partially-ordered FP numbers, so you are not allowed to complain that this is an odd behavior, when you have chosen it. Most programmers do not make this choice consciously, because they just use the default configuration of the standard library, but it is still their fault if the default does not do what they like, but nonetheless they have not changed the default settings.
If you do not want NaNs, you must not mask the invalid operation exception. This is actually what the IEEE standard recommends as the default behavior, but lazy programmers do not want to handle exceptions, so most libraries choose to mask all exceptions in their default configurations.
When invalid operations generate exceptions, there are no NaNs and the FP numbers are totally ordered, so the 6 relational operators behave as naive programmers expect them to behave.
If you do not want to handle the invalid operation exception and you mask it, there is no other option for the CPU than to use a special value that reports an invalid operation, and which is indeed not-a-number. With not-numbers added to the set of FP numbers, the set becomes a partially-ordered set and all relational operators must be interpreted accordingly.
If you use something like C/C++, with only 6 relational operators, then you must do before any comparison tests to detect any NaN operand, because otherwise the relational operators do not do what you expect them to do.
In a language with 14 relational operators, you do not need to check for NaNs, but you must choose carefully the relational operator, because for a partially-ordered set, for example not-less is not the same with greater-or-equal (because not-less is the same with greater-or-equal-or-unordered).
If you do not expect to do invalid operations frequently, it may be simpler to unmask the exception, so that you will never have to do any test for NaN detection.
>With not-numbers added to the set of FP numbers, the set becomes a partially-ordered set and all relational operators must be interpreted accordingly.
The same not-number, produced by the same computation, occupying the same memory, is still not equal to itself. It is true that I haven't been able to brush up my knowledge on partial ordering, but isn't being identical is the same as being equal in math?
If you test if a not-a-number is equal to itself, the result is false.
If you test if a not-a-number is not equal to itself, the result is also false.
The reason is that the result of comparing a not-number with itself is neither "equal" nor "not equal", but "unordered".
This should make perfect intuitive sense, because a NaN encodes the fact that "an undefined operation has happened".
For instance, you have 2 instances of the same NaN, both having been obtained by multiplying zero with infinity.
However it may happen that one was obtained while computing values of a sequence that converges to 10 and the other may have been obtained by computing values of a sequence that converges to 100.
Then there is no doubt that equality is not true for this 2 NaNs.
However, those 2 NaNs may have been generated while computing values of sequences that both converge to 10, which shows that neither non-equality may be trued for these 2 NaNs.
When you have NaNs, that means that it is unknown which value, if any, the invalid operations should have produced.
When comparing 2 unknown values, you cannot know whether they are equal or not equal, so both testing for equality and for non-equality must fail.
Okay, but how could the fact that these elements are in a partially ordered set, or whatever set, trump the basic law of logic, the law of identity, "a = a"?
Or the argument is that NaNs are not actually the values themselves, but the representations of the facts of different failures, and because we can't compare the facts, we shouldn't compare NaNs? Well, I guess one could say that numbers in general are also such incomplete representations; 2 is 2, I could get 2 by adding one crayon to another crayon, or by taking 10 crayons and removing 8 of them. That doesn't stop me from comparing these 2s.
No, you cannot, because that is logically inconsistent and it leads to bugs.
NaN means either an unknown number or that no number can satisfy the condition of being the result of the computed function.
When you compare unknown numbers, you cannot know if they are equal and you cannot know if they are not equal.
That is why both the equality operator and the non-equality operator must be false when comparing a NaN with itself.
When you see a binary FP number, the encoding no longer has any memory about where the number has been generated.
For deciding that a NaN is equal to itself, it is not enough to examine the encoding, you must know the history of the 2 instances of the same NaN value, where they had been generated. Only if the 2 instances are duplicates of a single value, i.e. they had been generated by the same operation, then you could say that they are equal.
If you would want such a facility in a program, then it is not enough to propagate just a floating-point value. You would have to define a custom data type and accompany the FP value by another value that encodes its provenance.
Then you could make custom equality and non-equality operators, which when comparing identical NaNs also compare their provenances, and they decide whether the NaNs are equal or non-equal based on the provenances.
Such a use case is extremely rare. In decades of experience I have never seen a situation when such a feature would be desirable. Nevertheless, if someone wants it, it would be easy to implement it, but it will add considerable overhead.
For example, the provenance could be encoded as a pointer to strings of the form "File ... Line ...", produced using the normal compiler facility for this, which will refer to the source line where the result of a function is tested for being a NaN, and if so the provenance pointer is stored, instead of a null pointer.
Even this encoding of the provenance may not be foolproof, because some functions may generate NaNs with different meanings. For a complete provenance encoding, one would need not only a pointer identifying the function that had generated the NaN, but also the value of a static counter that is incremented at each function invocation.
The provenance could be encoded compactly by storing the string pointers in an array and storing in the provenance an index into that array together with the value of the invocation counter.
So it can be done, but I am pretty sure that this would never be worthwhile.
Recall for a minute that floating-point arithmetic is inherently inexact, and it is thus pretty rare that you query for equality using an == operator rather than a relative or absolute error check (both of which involve < or > against a non-NaN value and thus would fail anyways for NaN). No, the main reason to ever actually use == on floating-point is if you're doing a bunch of tests to make sure expressions get the correct exact result (so you're expecting the inexactness to resolve in a particular way) or if you're doing non-computational stuff with floats (such as using them as keys in a map), at which point it becomes a problem that x != x is ever true.
> If you would want such a facility in a program, then it is not enough to propagate just a floating-point value. You would have to define a custom data type and accompany the FP value by another value that encodes its provenance.
IEEE 754 was way ahead of you. The entire rationale for the existing of NaN payloads was to be able to encode the diagnostic details about operation failure. And this suggests that you really want is the rule that NaNs are equal only to other NaNs with the same payload but are unordered with respect to everything else.
From the research I've done, it seems that the main reason that x != x for NaNs isn't some deep theoretical aspect about the properties of NaNs, but rather because IEEE 754 wanted a cheap way to test for NaN without having to call a function.
What if there was a voluntary indication of LLM content? Like, you press a checkbox "yes, I'm going to post some content that is partially or fully created by AI", and there would be a visible mark "slop" next to a post/comment.
There's a Japanese version of that page, written in classical text writing direction, in columns. Which is cool. Makes me wonder, though - how readable is it with so many English loanwords which should be rotated sideways to fit into columns?
Total digression but yeah, that layout is stupid and the way those words are dropped in using Romaji makes no sense. That's not how Japanese people lay out pages on the web. In fact I don't think I've ever seen a Japanese web page laid out like a book like this, and in general I'd expect the English proper nouns and words that don't have obvious translations to get transliterated into Katakana. Smells like automatic conversion added by someone not really familiar with common practices for presenting Japanese on the web.
He also has a Korean vertical layout that lays out Latin-character words the same way. Is this common in Korea when vertical layout is used? The author seems to be Korean.
>That fancy ARM-based MacBook with RAM soldered on the CPU package? We've got plenty of crashes from those, good luck replacing that RAM without super-specialized equipment and an extraordinarily talented technician doing the job.
CPU caches and registers - how exactly are they different from a RAM on a SoC in this regard?
In just about every way. CPU caches are made from SRAM and live on the CPU itself. Main system RAM is made from DRAM and live on separate chips even if they are soldered into the same physical package (system in package or SiP). The RAM still isn't on the SoC.
For one thing, static vs dynamic RAM. Static RAM (which is what's used for your typical CPU cache) is implemented with flip-flops and doesn't need to be refreshed, reads aren't destructive like DRAM, etc.
At that level, they are not different. They could suffer from UE due to defect, marginal system (voltage, temperature, frequency), or radiation upset, suffer electromigration/aging, etc. And you can't replace them either.
CPUs tend to be built to tolerate upsets, like having ECC and parity in arrays and structures whereas the DRAM on a Macbook probably does not. But there is no objective standard for these things, and redundancy is not foolproof it is just another lever to move reliability equation with.
Caches and registers are also subject to bitflips. In many CPUs the caches use ECC so it's less of a problem. Intel did a study showing that many bits in registers are unused so flipping them doesn't cause problems.
What this capturing software also does is it lies to the demo program about the time that passed between the frames, so the demo makers don't even care about running in realtime, because for them, it's like running on a PC that's almost infinitely powerful.
An old iPadOS means an old safari, which means some of the websites are going to get suspicious. I remember one day not being able to open any Cloudflare website.
How old was the device? I have a Late 2018 iPad Pro and have never encountered anything like this. It still works perfectly fine, and having invested in the nice keyboard case for it, I'm hoping to not have to upgrade for a while longer. It honestly might last me 10 years without breaking a sweat.
People just don't care. Even Stallman is okay with a microwave with closed-source firmware as long as it doesn't try to update its firmware.
For most people, a computer is just another appliance. They don't consider the security implications that this appliance can leak credit cards and such.
But I think they ought to. I also suspect that the current state of affairs is largely due to lack of understanding.
> as long as it doesn't try to update its firmware
I agree. But that isn't what we're talking about here. Things that can't update their firmware generally don't need you to upload a binary blob to them on startup.
reply