That's not dash, it's bash in POSIX compatibility mode. dash does not implement all the POSIX stuff in this "bible".
Imagine using this "bible" to write a "pure sh" script thinking "this will be highly portable", only to have it fail because the Linux scripting shell is not bash in POSIX compatibility mode, it's dash. It would fail on NetBSD, too, whose scripting shell is the version of ash from which dash is derived.
> That's not dash, it's bash in POSIX compatibility mode.
I was under the impression that bash's POSIX mode was 'leaky': that a lot of non-POSIX stuff/extensions are available even though it's called as /bin/sh.
I know this caused a lot of our users issues when we upgraded Debian/Ubuntu during its switchover to dash. We told folks you could either rewrite those parts of the code or call /bin/bash.
One major reason that Debian chose dash is plainly written on the bash manual page:
$ man bash | sed -n '/BUGS/,/^$/p'
BUGS
It's too big and too slow.
Another major reason is adherence to the POSIX standard; bash was written a decade prior to standardization, and has interesting issues because of this.
> bash was written a decade prior to standardization, and has interesting issues because of this.
bash can do whatever it wants to do if it's called as "bash". But it should only do specific things when called as "sh". This is true of any shell:
> When interpreted with dash instead of bash, the same script will fail. This is because dash is much stricter than bash in following the sh standard. Since dash is designed as a minimal implementation of the sh standard, it has to be stricter. The double brackets [[ … ]] are a ‘bashism,’ or a feature only available in bash and other, later shells such as ksh and zsh.
> Even though zsh also interprets most of these bashisms, zsh in sh compatibility mode is also stricter than bash and will error.
POSIX.2: Shell and Utilities (IEEE Std 1003.2-1992) is a few decades old now: regardless of when bash development started, I would think that by the 2010s it would have managed to better compartmentalize compatibility and extensions.
Do you have any specific examples to cite? The snippets are meant to be run in POSIX-compliant shells like dash/yash/ash/etc, not just Bash in compatibility mode.
In fact, there used to be a section which listed workarounds[1] for bugs in dash, but they have since been fixed[2]. If you are still using an old version of dash, you may need to use them.
Perhaps dash now has these features but does NetBSD sh or FreeBSD sh have them.
What's the point of "pure sh" if it's restricted to specific versions of shells. Opinions may differ but I'd rather learn the "lowest common denominator". I use the same scripts on both Linux and BSD so I need portability. I do not use bash for scripting. It's larger and slower.
I'm using the latest version of dash. I just tested it and it does have these ternary operators. However, I'm not going to use them in scripts unless I know they'll work on older dash versions, OpenWRT sh, Android sh, NetBSD sh, FreeBSD sh, etc. The process of updating these shells is generally very slow. It can take years.
"Pure sh" implies there is some advantage over "bashisms". I thought maybe it was portability but perhaps it is something else.
Yes and yes. My FreeBSD machine has not even been updated in 5 years, if that helps.
> What's the point of "pure sh" if it's restricted to specific versions of shells.
The aforementioned features have been implemented for a very long time. The issues with old versions of dash I mentioned were straight-up crashes for very simple things, not some fancy new-fangled feature that was yet to be implemented. Even then, the bible specifically listed workarounds for them.
dylanaraps has quite a prolific collection of shell programs, and they run on a variety of operating systems. Most notably, pfetch[1] runs on Linux, Android, NetBSD, FreeBSD, OpenBSD, Minix, Haiku, macOS, Solaris and IRIX. I assure you that he is fully aware of the importance of compatibility.
His pure Bash bible even has very thorough warnings for Bash versions required, since macOS uses Bash 3.2 (released in 2006).
I like to write posix sh scripts for the sake of portability and, funnily enough, future proofing as I don't like having to maintain stuff against changes that break compatibility, which is something bash does (archlinux is still on an older bash even though debian stable has the latest, because bash 5.2 broke some of archlinux's own scripts. This is why you should not write bash scripts.), but bash being slow isn't one of the good reasons. If your scripts do that much work that your shell's speed matters, you should reconsider writing shell scripts and start thinking about using something like perl, python or ruby. I'd suggest perl, if only because unlike the latter, perl doesn't constantly pull the rug under you and make you do busy work to make sure your scripts work on the latest runtimes.
I had the misfortune of having to maintain an 18'000 line bash 3 script. Array handling broke in many subtle ways between bash 3 and 4.
> If your scripts do that much work that your shell's speed matters, you should reconsider writing shell scripts and start thinking about using something like perl, python or ruby.
Usually yes.
But there is a small subset of use cases which need a really portable solution. Once I would have recommended to just write it in perl, because it was on virtually all systems. But now perl is getting phased out on some OSs/distros, and with python you never know if you get 2 or 3. Or embedded systems which don't have python/perl in the first place.
My go to solution for slow scripts is usually to limit subshells/forks as much as possible, and/or run it with busybox with the "exec prefers applets" option
edit: it's actually the "run 'nofork' applets directly" option. Can give quite a speed boost compared to bash if you have to call "external" utilities like grep/head/etc.
> I had the misfortune of having to maintain an 18'000 line bash 3 script
My hats off to you, I do not think I have the fortitude to stomach.. this.
> run it with busybox with the "exec prefers applets" option
> edit: it's actually the "run 'nofork' applets directly" option. Can give quite a speed boost compared to bash if you have to call "external" utilities like grep/head/etc.
Nifty trick, I had no idea about that, thanks for sharing this.
It is nothing as major as what Python does, but there are routine paper cuts through deprecation and breaking changes across versions, the most notable being 1.9 and 3.0.
But the language itself isn't the worst offender, the whole ecosystem around it tends to not even make reasonable attempts at stable APIs. Which isn't helpful with the way dynamically typed languages typically lack in tooling to safely, properly deal with refactoring - there are exceptions like smalltalk I've heard, but overall the cavalier attitude toward delivering a constant treadmill isn't pleasant to me. Of course, some may argue that the reason Perl doesn't suffer as much from it is because it is "dead", but I'll take dead over busywork.
I must say, I don't like Java the language, but Java the platform almost feels like the promised land compared to this. I don't use it to develop myself, but when I ran into an old unmaintained jar and ran it on a current JVM and it just works.. it feels.. fantastic. Absolutely wonderful. The same sort of feeling I also have when I run old games on Wine and it just works. win32 is the stable API I always wished linux had for GUI apps but will never have because the culture is all about CADT.
There are people commenting on HN who do not understand the difference between compatibiity and portability. This doesn't stop them from commenting on someone else's preference for portability.
Because if you are not 100% compatible with bourne shell you shouldn't automatically execute script made for bourne shell.
I have no problem with people using #!/bin/dash shebang for their scripts and using dash as their default shell but it shouldn't pass itself as something else. It is fine with bash because it is bourne shell compatible.
> Because if you are not 100% compatible with bourne shell you shouldn't automatically execute script made for bourne shell.
The same can be said about bash.
> It is fine with bash because it is bourne shell compatible.
I was running an HPC cluster when we upgraded Debian during the bash->dash change, and there were all sorts of problems: IIRC most folks simply changed their scripts from /bin/sh to /bin/bash.
I would argue reaching for the shell to do your scripting is never a good idea, unless your problem is trivial to begin with (e.g. fits in less than a hundred lines of extremely readable and straightforward code). My rule of thumb is: "can I fit it in a one-liner?" - then I refactor it until it's readable, or otherwise use something else.
Once you need to do fancy array manipulation, hash tables, data structures, etc you're very likely much better off with Python, Perl, Ruby, Lua, Tcl, JS, or literally anything else. Notably, one or both of Python and Perl are available out of the box on Debian (and derivatives), macOS, most (all?) BSDs, and so on. Targeting Python 3.6+ will probably cover 99%+ of the systems you can find in the wild (e.g. Ubuntu 18.04 and newer).
I have also had some minor success using Go for that purpose. The toolchain can trivially cross-compile executables for ca 40 different OS/arch combinations; you need to consider the os/arch if you're running something different than Linux/x86-64 but that will again likely put you in the 1%. I'm working on some tooling to make this approach more straightforward.
One-liners are meant to be thrown away, so it does not matter if it's pretty or not. If you care enough to keep it, you should consider rewriting it for clarity. That last step does not require using any particular language; e.g. my PS1 started off as an increasingly convoluted one-liner, until I rewrote it in Go.
Depends on what you're trying to do. If you're shelling out to git(1) or docker(1), rather than e.g. recursively checking for the presence of .git in parent directories, or inspecting ~/.docker/config.json, then the fork+exec overhead is already quite significant. Next if you're parsing ~/.docker/config.json in shell, you're most likely either asking for trouble or (again) shelling out to jq. Writing it all in an interpreted language means you're paying the cost of interpreter startup, which on underpowered systems can take hundreds of milliseconds even when idle. OTOH loading a static binary to memory happens only once, and with Go you can trivially cross-compile.
I also have a fallback shell one-liner, without any of the fanciness like displaying the current git branch:
