In my horrible opinion, you can have a fairly portable terminal application without using the abomination that is libncurses, just straight escape codes / control sequences.
Main reason is that most of current terminal emulators are actually xterm emulators: they support a good subset of xterm features. Also, the exceedingly strange console that was in Windows 95 is finally out. What quirks and differences are left, can easily be detected at runtime, no need for $TERM and terminfo/termcap
This is my korn-shell fragment that I use to set up color codes. Everything but the final print is posix-compliant (change it to a printf to run under bash).
N='\033[' x=30
for a in Bl R G Y B M C W # 4-bit Black Red Green Yellow Blue Magenta Cyan White
do eval $a='$N'"'"$(( x ))"m'" \
b$a='$N'"'"$((60+x ))"m'" \
${a}bg='$N'"'"$((10+x ))"m'" \
b${a}bg='$N'"'"$((70+x++))"m'" # bX=bright Xbg=background bXbg=brgt bgnd
done
N=$N'0m' x=0
print $bB'This is bright blue.'$N
This is one of the many cool things that zsh comes with out of the box. It ships an autoload-able colors¹ script that does largely the same thing as your fragment, but you know it available wherever zsh is.
In much the same way it comes with both a termcap² and terminfo³ module, allowing you to easily avoid hardcoding system specific escapes in to your config.
We made two decisions early on: xterm is the only terminal protocol, and UTF-8 is the only encoding. This has spared us unending amounts of pain, because we don't abstract over these things. There is a bit of divergence between xterm implementations, but for what we do, not much: if you want to print images there are a few strategies, none of them universal, and the closest (sixel) isn't great imho.
We've also found that, with a modern terminal, a lot of what curses brings to the table isn't necessary. While we did divide the terminal up into zones, and only repaint changed zones, we shouldn't have bothered: as long as you hide the cursor when you start a paint, and show it after you put it back, filling the entire screen with a complex/colorful xterm pattern happens within one refresh, so no flicker.
There is certainly no need to calculate a minimal delta change and only print that, on a reasonably modern system (anything from the last ten years will do).
> the exceedingly strange console that was in Windows 95
That only ever tried to be a terminal emulator when ansi.sys had been loaded. But it's been gone for quite a while now. The Windows console host on the other hand only gained terminal emulator capabilities in somewhat recent Windows 10 versions.
The funny thing is that termcap and vi were not created for a DEC terminal.
They were created specifically for the Lear Siegler ADM-3A, which predates the VT100. The work was done entirely by Bill Joy, who later joined Sun Microsystems.
It seems like all the remaining terminals and emulators have gravitated to the same feature set so people just assume it doesn't matter? There was, at one point, a fair amount of really oddball terminals, like the Hazeltine 1500 that weren't VT1xx compatible. And they were at least minimally useful on a Unix box with terminfo/termcap.
> It seems like all the remaining terminals and emulators have gravitated to the same feature set so people just assume it doesn't matter?
In the context of Unix-like systems, that is pretty much true. But, broader than that context, there are still terminal emulators in use which are completely unrelated to DEC VT. People still use IBM 3270 emulators to talk to IBM mainframes, and IBM 5250 to talk to IBM i (formerly known as OS/400). People still use Unisys T27 terminal emulators to talk to Unisys mainframes.
There are also some terminal emulators out there with unusual features. Consider AccuTerm, a terminal emulator specifically designed for use with PICK systems. If you read page 207 onwards of [0], you will see escape sequences to display images, run commands on the client machine, trigger file uploads and downloads, and even a facility for an escape sequence to embed a VBA script which is then executed. A lot of that is quite dangerous if you don't trust the system you are connecting to, or if the system you are connecting to doesn't correctly filter escape sequences out of untrusted data; but, assuming it does, it can be used to easily create some integrations between legacy PICK apps and Windows desktop apps such as Microsoft Office.
Indeed, both of these terminals were part of a Cambrian explosion of glass TTYs.
I mentioned the VT100 specifically as that was what the comment referred to. Also because for some reason it somehow became iconic, though I don't know why (I always thought the VT52 superior, though the best IMHO was the Ann Arbor Ambassador).
Because just like the code page in locales, you should not need to care about terminfo if all you want to do is show some basic color. In $current_year it is OK to assume that the current terminal (if stdout/stderr is a TTY) can at least parse color escape sequences - if yours doesn't then create your own wrapper that strips them. Generally the way git does it should be considered best practice. If
- Output is a TTY
- The NO_COLOR environment variable is not defined
- The TERM environment variable is not set to "dumb"
The git approach here is not "not caring about terminfo", but delegating the caring (of the difficult parts at least) to another program, namely the pager. The default pager `less` depends on the information in TERM and the terminfo database, and it has to, since functions like toggling to the alternate screen buffer are not standardized per ansi.
I agree that you will need something like terminfo if you writing a TUI like a pager, but for most CLI programs you only need simple color output and maybe cursor movement on and erasing of the current line and for that terminfo is an unnecessary dependency.
Having been recently delving into getting a TRS-80 DT-1 emulating a Televideo 910 working with Linux, I couldn't agree more.
Even now for modern devices it wouldn't be the worst idea to avoid hard coding things like color (e-ink?). And it's not like it's difficult to avoid hard coding, using tput ( https://linux.die.net/man/1/tput ) means you don't have to write out the arbitrary escape sequences yourself while being compatible with any terminal.
"My team writes a lot of command line tools" is a valid excuse for internal use. But not for writing a educational post.
> And it's not like it's difficult to avoid hard coding
There are upsides to hard-coding the escape sequences that you aren't mentioning. Using terminfo or ncurses requires an extra dependency with a clunky interface. Using tput requires spawning a process, which gets slow if you have to do it repeatedly. But simply hard-coding the escape sequences is dependency-free, as fast as printing, can be done in any language, and is going to work for at least 99.9% of the users out there.
I'm usually a big fan of "use a standard interface rather than one particular implementation", but I haven't found a solution, other than hard-coding the ANSI escape code set, that I'm happy with.
> Using terminfo or ncurses requires an extra dependency with a clunky interface.
Are there any linux distros for any platform that aren't shipping terminfo? I could see some embedded systems cutting it down for small storage solutions. But I don't think it's unfair to assume that terminfo is installed. ncurses is more complex so I won't disagree with that one, but tput isn't part of ncurses.
Clunkiness is an opinion, I see needing to look up what "\x1b[7m" means as being clunky compared to "tput rev".
> Using tput requires spawning a process, which gets slow if you have to do it repeatedly.
If you're pretty printing text for a human to read then I don't think the couple extra ms it will take to spawn a handful of tput processes is going to be an actual performance concern. If you're logging data that's another thing, but this was specifically about CLI programs.
> can be done in any language, and is going to work for at least 99.9% of the users out there.
> I haven't found a solution, other than hard-coding the ANSI escape code set, that I'm happy with.
This is most likely true and I don't disagree that my stated use case is in the 0.01%. I was going to refute the any language comment with libraries based on terminfo, but all the ones I ran across in 30 seconds just hard code the ANSI escape sequences. Since it will likely work 99.9% of the time, it is "easy" in a way to hard code them. So I understand why it is being done that way and why you would also prefer to hard code them. But there is a standard interface to use, it just may not be perfect.
This may be something where we are technically stuck between a rock and a hard place. We have a good resource database of technical information on how to format text. But there isn't a significant reason to put in the effort to use it.
> Are there any linux distros for any platform that aren't shipping terminfo?
Ah, I was referring to higher-level languages such as Ruby here (I have a lot of scripts where I use it a shell script replacement) rather than C, where you can just download a compiled version and use it without any fuss. Ruby moved its curses interface out of the standard library a long time ago, so now you have to add it to your gems list.
> If you're pretty printing text for a human to read then I don't think the couple extra ms it will take to spawn a handful of tput processes is going to be an actual performance concern.
For scripts, I agree — I used to use tput, and only stopped because I'd eventually seen the ANSI escape codes enough time to just memorise them. But I find that if your program needs to use enough colours (my software has been called "overly colourful" before), the time it takes to run a tput once for every bold and normal variant of all eight colours quickly makes it seem like your program is running on the JVM, even if I cache the output. I appreciate that not all software needs every terminal style available to it, but it seems to me that the best interface should be able to scale to many styles, not just a few.
> I was going to refute the any language comment with libraries based on terminfo, but all the ones I ran across in 30 seconds just hard code the ANSI escape sequences.
This doesn't surprise me! I used to be concerned with the growing number of comments and guides on the internet that say things like "this is how you do bold in a terminal" as though it applies to all terminals, so I asked a question on Stack Exchange a while ago:
I was hoping to find an answer from someone like yourself, who regularly works with non-ANSI terminals and was becoming increasingly annoyed at the proliferation of standards-shirking shell scripts... but I only got comments saying I was doing it wrong, as though I was the only one who didn't get the memo. You're the only obscure terminal user I've ever talked to in the wild, congratulations.
Anyway, I agree that we're stuck in this situation, and even though I am taking the easy way out, I still find that a shame.
You'll also find a lot of Google search results for "tput: No value for $TERM and no -T specified". I wish it was as simple as just calling tput, but you need to write a bunch of logic anyway.
Funny, I write terminal codes directly, because where I learned to program (PC DOS), there was no terminfo, only ANSI.SYS. I don't ever plan to port anything to DOS ever again, but old habits do die hard.
Back in the 90s I played a MUD called Ancient Anguish. It's the type of MUD where inventory is dropped on logout. I learned they didn't filter for control sequences, and so I would rewrite the text on screen to do all sorts of nefarious things, like make people think they were ambushed and almost dead (thus possibly trying to quit to save EXP loss). I could also forge tells/yells to make them look like they came from someone else. It was funny to me at the time, lol.
edit: for fun i just checked out it's current status and it's still around, awesome. and rule #4 is now don't abuse control codes. hahaha, sorry guys. AFAIK they started filtering them soon after me and a cohort caused some chaos.
The terminal hyperlink thing seems quite useful. Kitty has some magic where if you click a hyperlink over a direct ssh connection it shows you a menu that has some options like download or download/open in editor/reupload when done. I haven't checked the details yet how it determines that you are in an ssh session.
Thanks for that. I’ve mentally added it to my list of useful man pages not related to a specific command (`man ascii` and `man hier`). I see the `manpages` package provides a few more manuals describing computing concepts in Section 7, e.g., `cgroups`, `charsets`.
useful if you want to take your colorful terminal output and share it on a web page, OR if you're really geeky you can write CLI tools with colored output, to summarize data then convert it to HTML, then to PDF and share it with your coworkers. (Chrome's command line interface is the best HTML to PDF converter I've found).
> The majority of these escape codes start with \x1b[
This is actually the 7-bit version for CSI. A less known variation is the 8-bit CSI which is simply \x9b and is the same as \x1b[ combined. So, for example \x9b33m does the same as \x1b[33m .
Unfortunately, there exist a couple of (8-bit) terminals that don't support this version.
I really like vttest. I used to use it when I was still working on Console Telnet for Win32 (https://github.com/cout/consoletelnet). It revealed a lot of bugs in how escape sequences were handled that I might never have noticed since they were obscure (yet important) corner cases.
I've (edited and) compiled VTTEST to run under Windows. The latest iteration of the Command Prompt does surprisingly well in the tests. Of course the new Windows Terminal does well too.
Hah! I saw your note about programmatically entering Tek mode from xterm. You can do: echo -e "\e[?38h" from the vt100 window to launch the Tek window. Then echo -e "\e\x03" from the Tek window to switch input back to the vt100 window.
Edit: Hrm. I did try it out before posting, and it worked. So not sure why it isn't working for you.
Seems to me it's bad practice to combine the text you want to show with the formatting (escape codes) in string literals. Everything is then immediately dependent on specific type of UI technology (terminal).
Eh, depends on context, for bash scripts that might be just fine. You need to have some styling information go along with it anyways, so instead of having some dedicated markup language and then having an output library that converts that to whatever your output uses, you could just declare those ansi escape codes your "markup language" and should you ever need to output it to something else, say a web page, I'm sure there's code around that does the conversion. But I guess to make it more readable you might want to put those commonly used sequences into some variables so you can do
At least call tput or something so that your script doesn't do anything wacky just because you SSH'd in this time instead of using your customary terminal.
Until someone runs it in eshell, which is a shell but not a terminal emulator and you have to configure an override just for that special program so that it isn't dumping unreadable garbage into the buffer.
Of course. Just view those strings as the output of the formatting backend for the terminal. The article is about terminal escape codes, not software modularization.
Depending on what you are doing you can always parse and translate the escape codes to something else for the systems where they are not supported. One advantage of using in-line escape codes in your string is that you can compose multi-colored text and buffer them in whatever strings your language provides.
> 0 and 7 are less useful for text because one or the other will generally look nearly-unreadable depending on whether the user has a light or a dark background.
This really annoys me about the bright themes used by default in some distros - if you make the default background white instead of black then please also swap the "white" and "black" foreground colors. And make sure all the other ones are readable on the default background for that matter.
I'm a maintainer of chalk. I'm not the first to mention this, but this article is only really parroting the information found on the Wikipedia article for escape sequences, which in turn only really covers xterm-256color terminals.
To be quite honest, I feel as though terminal technology is the most antiquated thing still in wide use today. It's slow, there are so many competing standards and none of them are really that well standardized, and the libraries needed to be truly compatible (terminfo in particular, for which I've written several full parsers and boy what a mess that is) are clunky, outdated and also very slow.
This is a hard problem to solve, however. There are more CSI sequences than just \x1b for example, though that is by far the most common these days. Not treating output as text streams would be an interesting start but that would require rewriting pretty much all software ever to exist.
Meh. I've spent too much time working with escape sequences and thinking about this problem. I am young, but I'll probably die before TTYs do.
This brings back memories. In the beginning of my career (early 90's) i implemented an entire UI for a Personnel Information System in COBOL on a Mainframe (CDC Cyber 180/840A) using ANSI escape sequences.
Thanks, i learnt a lot from that experience (my first job, sole developer, figure out a solution and get things done etc.). It was on those old monochrome visual display terminals and instead of using system specific languages/toolkits i hit upon this solution and it worked out.
Wanna read something about terminals and terminal control sequences? https://invisible-island.net/ncurses/terminfo.src.html
Disclaimer: i use a DEC VT520