> When programming, it is still important to write code that runs correctly on systems with either byte order
I contend it's almost never important and almost nobody writing user software should bother with this. Certainly, people who didn't already know they needed big-endian should not start caring now because they read an article online. There are countless rare machines that your code doesn't run on--what's so special about big endian? The world is little endian now. Big endian chips aren't coming back. You are spending your own time on an effort that will never pay off. If big endian is really needed, IBM will pay you to write the s390x port and they will provide the machine.
> There are countless rare machines that your code doesn't run on--what's so special about big endian?
One difference is that when your endian-oblivious code runs on a BE system, it can be subtly wrong in a way that's hard to diagnose, which is a whole lot worse than not working at all.
That sounds like a problem to deal with as part of your paid IBM s390x porting contract. I guess my point is: why deal with this before IBM is paying you? No other big endian platform matters, and s390x users are 100% large commercial customers. If IBM or one of their customers isn't paying you, there's nobody else who would need it. If IBM is paying you, you can test on a real z/VM that they provide. I see big endian as entirely their burden now; nobody else needs it. If they want it, they can pay for the work.
Also, endian-correct code is usually semantically clearer. For example, if you're reading network-ordered bytes into an int, an unconditional endian swap (which will produce correct results on LE systems but not BE) is less clear than invoking a "network bytes to u32" helper.
I think that's a bit different than the argument being made. We should still always use htonl() and ntohl() etc. when dealing with protocols that use network byte order (a shame we're stuck dealing with that legacy). I think even if all big-endian machines magically disappeared tomorrow, we should still do that (instead of just unconditionally doing a byte-swap).
But for everything else, it's fine to assume little-endian.
You sound like some sort of purist, so sure, if you really want to be explicit and support both endiannesses in your software when needed, go for it. But as general advice to random programmers: don't bother.
It might sound outrageous but I guard against this sort of thing. When I write utility code in C++ I generally include various static asserts about basic platform assumptions.
> running on a real Sun3, compiled with a non-ANSII compiler (Sun cc 1.22)
> this is fatal in HP-UX 10 with the bundled compiler
> OpenWatcom 1.9 compiler
> OS/2 builds
> making sure that all functions are declared in both ANSI format and K&R format (so C-Kermit can built on both new and old computers)
Oooooh! A clang complaint: 'Clang also complains about perfectly legal compound IF statements and/or complex IF conditions, and wants to have parens and/or brackets galore added for clarity. These statements were written by programmers who understood the rules of precedence of arithmetic and logical operators, and the code has been working correctly for decades.'
There's platform and there's platform. I assume a POSIX platform, so I don't need to check for CHAR_BIT. My code won't work on some DSP with 64-bit chars, and I don't care enough to write that check.
Many of the tests I did back in the 1990s seem pointless now. Do you have checks for non-IEEE 754 math?
Using C++ under Clang 17 and later (possibly earlier as well, I haven't checked) std::numeric_limits<T>::is_iec559 comes back as true for me for x86_64 on Debian as well as when compiling for Emscripten. Might it be due to your compiler flags? Or is this somehow related to a C/C++ divergence?
If I am not mistaken, is_iec559 concerns numerical representation, while __STDC_IEC_559__ is broader, and includes the behavior of numerical operations like 1.0/-0.0 and various functions.
I do, yes. I check that the compiler reports the desired properties and in cases where my code fails to compile because it does not I special case and manually test each property my code depends on. In my case that's primarily mantissa bit width for the sake of various utility functions that juggle raw FP bits.
Even for "regular" architectures this turns out to be important for FP data types. Long double is an f128 on Emscripten but an f80 on x86_64 Clang, where f128 is provided as __float128. The last time I updated my code (admittedly quite a while ago) Clang version 17 did not (yet?) implement std::numeric_limits support for f128.
Honestly there's no good reason not to test these sorts of assumptions when implementing low level utility functions because it's the sort of stuff you write once and then reuse everywhere forever.
And as I wrote, "There's platform and there's platform."
I don't support the full range of platforms that C supports. I assume 8 bit chars. I assume good hardware support for 754. I assume the compiler's documentation is correct when it says it map "double" to "binary64" and uses native operations. I assume if someone else compiles my code with non-754 flags, like fused multiply and add, then it's not a problem I need to worry about.
For that matter, my code doesn't deal with NaNs or inf (other than input rejection tests) so I don't even need fully conformant 754.
This doesn't seem to be the same thing. The article isn't about being unable to type before JavaScript starts executing. If I understand correctly, you're unable to type until a network request to Cloudflare returns. The question is: why not allow typing during that network request? JavaScript is running and it's observing the keystrokes. Everyone understands that you can't use a React application until JavaScript is running. They're asking why the network request doesn't happen in the background with the user optimistically allowed to type while waiting for it to return.
(Separately, I don't think the article has adequately demonstrated this claim. They just make the claim in the title. The actual article only shows that some network request is made, and that the request happens after the React app is loaded, but not that they prevent input until it returns. Maybe it's obvious from using it, but they didn't demonstrate it.)
I don't think that's true in this case; the React application loads first, fully initializes, and then sends its state via Cloudflare request. It can't happen at the same time, by design. It has to happen serially. The article's claim is that you can't type during this second request. Frankly, I wonder if this is actually true at all. The article did not demonstrate this, and there's no problem if you can actually interact as soon as the React application is running. ChatGPT running abuse prevention and React applications requiring JavaScript to work are both uncontroversial, I think.
OK, I haven't looked at the exact sequencing here. But generally, once the action goes back to the anti-abuse service for checking the user can't be allowed to change what they're submitting. The view the anti-abuse system saw has to match what the app server sees.
From some minor historical experience with Mechanical Turk, I bet you could get humans to do this for one or two cents per receipt. You do them all three times for error checking for $0.03-$0.06 per receipt. I used to pay a nickel for much, much more than 5x this amount of transcription per job, and I got the feeling that I was overpaying based on how eagerly I got responses in and that I saw a lot of the same workers repeatedly.
These days, are MTurk workers simply feeding it into AI anyway, though? It's been a few years since I've run an MTurk campaign. At the time it was clear that humans were really doing it, as you get emails from the workers sometimes.
You can use `xxd` from the vim package to generate these. You'll find out pretty quickly that this is only suitable for small resources: gcc and clang blow up in both time and space on very large literal arrays. If you need to ship more than a few megabytes, find a different technique.
I used this technique for awhile, but it was too problematic for my use case. Now, I use https://github.com/lief-project/LIEF -- among other things, this project can modify Windows PE, macOS Mach-O, and Linux ELF binaries to add resources to them, then offers an API to read them back later. It's a little different for each format, but it's capable of doing all three and I was able to build a cross-platform resource-bundling system that doesn't care how big the resources are.
Yeah xxd is the correct answer. If you don't want to install a dependency and have Lua installed, or if you're just feeling a little bit frisky, you can use my function which is Production Ready™.
Xxd = function(name, input)
if not name:find'^[_%a][_%w]*$' then error('bad name: '..tostring(name)) end
local ans = {
'const unsigned int '..name..'_len = '..(#input)..';',
'const unsigned char '..name..'[] = {',
}
local t = {}
for i=1,#input do
table.insert(t, ('0x%02x,'):format(input:byte(i)))
if #t == 16 then -- 16 columns per row. arbitrary, change this if you want
table.insert(ans, table.concat(t))
t = {}
end
end
if #t ~= 0 then
table.insert(ans, table.concat(t))
end
table.insert(ans, '};\n')
return table.concat(ans, '\n')
end
I am distributing it under the terms of the GNU GPL v3. So if you put this in your codebase I will sue you into releasing your entire source. Just kidding it's MIT licensed.
Honestly that's a terrible joke. Seriously it's MIT. Here I will put the full license in this comment to illustrate how serious I am:
Copyright 2026 rweichler
Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the “Software”), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
Feels like Lua is a more exotic dependency. I used to use xxd but this gets problematic when files grow (and they don't need to grow much at all), objcopy is much faster (which I didn't think would matter much, but it did) and don't have the same issues that accidentally opening the .h file in your editor or code-searching having to traverse that mess.
Yes, you'd want to gitignore it and exclude it from search etc. but you still have size issues etc. See gucci-on-fleek's comment on objcopy above for usage.
I shipped 20MB of XML with a product back in 2014; we loaded it at startup, validated it against the XSD, and the performance for this use case was fine. It was big because we did something kinda like what TFA suggests: I designed a declarative XML "DSL" and then wrote a bunch of "code" in it. We had lots of performance problems in that project, but the XML DSL wasn't the cause of any of them; that part was fine. I think "expensive" can mean a lot of different things. It was cheap in terms of development time and the loading/validation time, even on 20MB of XML, was not a problem. Visual Studio ships a tool that generates C# classes from the XSDs which was handy. I just wrote the XSDs and the framework provided the parsing, validation, node classes, and tree construction. This is as "XML proper" as I think it's possible to get.
I don't believe that .NET's XML serializer uses any of the open source projects mentioned in your post, so maybe we just have especially good XML support in .NET. I think Java has its own XML serializer, too. I bet most XML generated and consumed in the world is not one of those three open source C/C++ libraries. I think Java alone might be responsible for more than half of it.
Yes, Scintilla is a text editor engine library. It's not tied to any particular UI or technology. Out of the box it's not a text editor yet; you provide the frontend. You get all the "table stakes" right away if you build your editor on this library.
Same engine, different frontends. The engine has a series of hooks that you implement in whichever way you please for your particular interface. It's definitely the presumptive choice here.
Ah, I see! Very cool! Yeah, this is the kind of thing I was looking for, so this should give me what I need to test some proof of concepts. Thanks for the links! I do wish there were something a little more ergonomic, but I'm way too far into the begging to be choosing, here, so I'm quite happy to take what I can get.
In any case, I really do appreciate the dual links. It's so much harder to suss out the boundaries of a library with only one implementation. This was really helpful.
I don't know about everyone else, but slow Julia compilation continues to cause me ongoing suffering to this day. I don't think they're ever going to "fix" this. On a standard GitHub Actions Windows worker, installing the public Julia packages I use, precompiling, and compiling the sysimage takes over an hour. That's not an exaggeration. I had to juice the worker up to a custom 4x sized worker to get the wall clock time to something reasonable.
It took me days to get that build to work; doing this compilation once in CI so you don't have to do it on every machine is trickier than it sounds in Julia. The "obvious" way (install packages in Docker, run container on target machine) does not work because Julia wants to see exactly the same machine that it was precompiled on. It ends up precompiling again every time you run the container on other machines. I nearly shed a tear the first time I got Julia not to precompile everything again on a new machine.
R and Python are done in five minutes on the standard worker and it was easy; it's just the amount of time it takes to download and extract the prebuilt binaries. Do that inside a Docker container and it's portable as expected. I maintain Linux and Windows environments for the three languages and Julia causes me the most headaches, by far. I absolutely do not care about the tiny improvement in performance from compiling for my particular microarch; I would opt into prebuilt x86_64 generic binaries if Julia had them. I'm very happy to take R's and Python's prebuilt binaries.
I am very interested in improving the user-experience around precompilation and performance, may I ask why you are creating a sysimage from scratch?
> I would opt into prebuilt x86_64 generic binaries if Julia had them
The environment varial JULIA_CPU_TARGET [1] is what you are looking for, it controls what micro-architecture Julia emits for and supports multi-versioning.
As an example Julia is built with [2]:
generic;sandybridge,-xsaveopt,clone_all;haswell,-rdrnd,base(1)
I have a monorepo full of Julia analysis scripts written by different people. I want to run them in a Docker container on ephemeral Linux EC2 instances and on user Windows workstations. I don't want to sit through precompilation of all dependencies whenever a new machine runs a particular version of the Julia project for the first time because it takes a truly remarkable amount of time. For the ephemeral Linux instances running Julia in Docker, that happens on every run. Precompiling at Docker build time doesn't help you; it precompiles everything again when you run that container on a different host computer. R and Python don't work like this; if you install everything during the Docker image build, they will not suddenly trigger a lengthy recompilation when run on a different host machine.
I am intimately familiar with JULIA_CPU_TARGET; it's part of configuring PackageCompiler and I had to spend a fair amount of time figuring it out. Mine is [0]. It's not related to what I was discussing there. I am looking for Julia to operate a package manager service like R's CRAN/Posit PPM or Python's PyPI/Conda that distributes compiled binaries for supported platforms. JuliaHub only distributes source code.
My point is if you set JULIA_CPU_TARGET during the docker build process, you will get relocatable binaries that are multi-versioned and will work on other micro-architecture? It's not just for PackageCompiler, but also for Julia's native code cache.
It worked! I was able to drop the Windows install on a standard GitHub Actions worker from 1 hour to 27 minutes. Here's what worked:
ARG JULIA_CPU_TARGET="generic;skylake-avx512,clone_all;cascadelake,clone_all;icelake-server,clone_all;sapphirerapids,clone_all;znver4,clone_all;znver2,clone_all"
ARG JULIA_PROJECT=[...]
ENV JULIA_PROJECT=[...]
RUN julia -e "using Pkg; Pkg.activate(\"[...]\"); Pkg.instantiate(); Pkg.precompile();"
What I got wrong the first time: I failed to actually export JULIA_CPU_TARGET so it would take effect in the "Pkg.precompile()" command. In reality, I hadn't correctly tested with that environment variable set at all. I was only correctly setting it when running PackageCompiler.
Thank you so much for this! It's too late for me to edit my original post, but cutting the install time in half is a major win for me. Now it only needs to precompile, not also compile a sysimage.
... Actually, this only worked for Linux. My Windows container is back to precompiling every time, again, and Windows was the slow one that I wanted to fix in the first place. I had to revert this. Back to the drawing board. I wish Julia would print some diagnostics about why it decided to precompile again.
I think I finally nailed it? I needed to update JULIA_DEPOT_PATH since I was relocating the cache files. I knew that and was already doing it, but I didn't do it right: I needed a trailing semicolon so it would have an empty entry at the end! The documentation talks about this but I didn't understand it at the time: https://docs.julialang.org/en/v1/manual/environment-variable...
That was the very first thing I tried, and I couldn't get it to work, but I'm sure I am doing something wrong. Everything seemed great at build time, and then it just precompiles again at runtime, without anything saying why it decided to do that. I'll give it another shot if you say it should be working. The PackageCompiler step is the longest part; if that can be removed, it'll make a big difference. I'd rather be wrong and have this working than the other way around :) I'll report back with what I find.
> It took me days to get that build to work; doing this compilation once in CI so you don't have to do it on every machine is trickier than it sounds in Julia
You may be interested in looking into AppBundler. Apart from the full application packaging it also offers ability to make Julia image bundles. While offering sysimage compilation option it also enables to bundle an application via compiled pkgimages which requires less RAM and is much faster to compile.
I believe it is. Just tested it. You can make the link "C:\windows\system32\cmd.exe" and clicking it will launch the Command Prompt. I noticed you can't make it "C:\windows\system32\cmd.exe /c some-nefarious-thing"; it doesn't like the space. Exploiting may require you to ship both the malicious EXE and the MD, then trick the user into clicking the link inside the MD. But then you could have just tricked them into directly clicking the EXE.
>Exploiting may require you to ship both the malicious EXE and the MD, then trick the user into clicking the link inside the MD. But then you could have just tricked them into directly clicking the EXE.
1. You can use UNC paths to access remote servers via SMB
2. Even if it's local, it's still more useful than you make it out to be. For instance, suppose you downloaded a .zip file of some github project. The .zip file contains virus.exe buried in some subfolder, and there's a README.md at the root. You open the README.md and see a link (eg. "this project requires [some-other-project](subfolder\virus.exe)". You click on that and virus.exe gets executed.
Programs (this is true for most mainstream operating systems) can become network facing without realizing it. I've sometimes found a bunch of Windows programs sometimes tends to assume that I/O completes "instantly" (even if async I/O has been common on Windows for a very long time) and don't have a good UX for cancelling long running I/O operations
I won't be paying extra to use this, but Claude Code's feature-dev plugin is so slow that even when running two concurrent Claudes on two different tasks, I'm twiddling my thumbs some of the time. I'm not fast and I don't have tight deadlines, but nonetheless feature-dev is really slow. It would be better if it were fast enough that I wouldn't have time to switch off to a second task and could stick with the one until completion. The mental cost of juggling two tasks is high; humans aren't designed for multitasking.
Hmm I’ve tried two modes: one is to stay focused on the task at hand, but spin up alternative sessions to do documentation, check alternative hypotheses, second-guess things the main session is up to. — The other is to do an unrelated task in another session. I find this gets more work done in a day but is exhausting. With better scaffolding and longer per-task run times (longer tasks in the METR sense), could be more sustainable as a manager of agents.
3-4 parallel projects is the norm now, though I find task-parallelism still makes overlap reduction bothersome, even with worktrees. How did you work around that?
I contend it's almost never important and almost nobody writing user software should bother with this. Certainly, people who didn't already know they needed big-endian should not start caring now because they read an article online. There are countless rare machines that your code doesn't run on--what's so special about big endian? The world is little endian now. Big endian chips aren't coming back. You are spending your own time on an effort that will never pay off. If big endian is really needed, IBM will pay you to write the s390x port and they will provide the machine.
reply