> First, there’s “the baseline” – what happens if a program just accepts text by calling fgets() or whatever and doing absolutely nothing else to provide a nicer experience.
[...] there are actually a few features that you get for free just from your terminal, without the program needing to do anything special at all.
The things you get for free are: [...]
- backspace
- Ctrl+W, to delete the previous word
- Ctrl+U, to delete the whole line
Linux noob here, so this may be a stupid question, but how does that work under the hood?
The documentation for fgets() says [1]:
> Reads at most count - 1 characters from the given file stream and stores them in the character array pointed to by str. Parsing stops if a newline character is found (in which case str will contain that newline character) or if end-of-file occurs.
Does that mean that, by default, fgets() blocks until the user enters a newline and, before they do, it lets them edit the line buffer using the backspace, CTRL+W and CTRL+U shortcuts?
But there seems to be no guarantee that a program is using fgets() to read an entire line - it could also set count to 2 and read individual characters. Those could not be "unread" anymore when a backspace occurs. Is there some magic that lets fgets() buffer characters internally in such a situation, or would the backspace/line editing functionality just be broken then?
Programs like zsh, bash, ksh, fish and most TUIs put the terminal into cooked mode so that they can manage the terminals behavior manually. In cooked mode, the terminal is not new line buffered and instead every character is read. The terminal only appears to be new line buffered but is actually being managed by the shell, which then in turn allows the devs to add nice features like manipulating the text buffer to do tab completion or prepend the line with sudo.
> Does that mean that, by default, fgets() blocks until the user enters a newline
Yes. By default, terminals operate in canonical mode. I/O does not happen at all until the user inputs a full line. Until the user hits Enter, the characters on the screen are just sitting there in the terminal's memory while the application's read system call is blocking on the terminal's file descriptor. The terminal does not write the data until the line is complete.
> and, before they do, it lets them edit the line buffer using the backspace, CTRL+W and CTRL+U shortcuts?
Provided that "it" refers to the terminal, then yes. Normally it's the terminal itself which allows you to edit the line you are typing on. This is also provided by the default canonical mode. Even the fact that letters show up on the line when you type is a terminal feature called echoing. So are things like backspace, delete, Ctrl+C for SIGINT, Ctrl+D for EOF.
Line and text editors just turn off all of this stuff. They place the terminal in raw mode where everything you type is sent immediately to the application where it is handled immediately. A wide variety of everyday programs do stuff like this. For example, password prompts simply turn off echoing to hide the characters.
Even the traditional Unix line ending \n is actually a feature of terminals. To the terminal, \n just moves the cursor down one line, \r is also needed to go back to the beginning of the line. The true line ending is therefore \r\n. The terminal just happens to invisibly turn \n into \r\n. This too can be disabled.
> The terminal just happens to invisibly turn \n into \r\n.
No, it was actually done by the Unixen itself, in the part of the kernel that handled ttys: it convert '\n' to '\r\n' on writes to ttys, and '\r' to '\n' on reads from ttys. Linux only recently have moved this functionality out into the user-land layer.
Hmmm, I could swear I have read about moving line discipline out of the kernel in some LWN article but apparently no, it never happened (although it probably should've, there is no much reason to perform mapping of NL to CR-NL in the privilleged mode). Maybe I've confused it with what happened on the Windows side of things? They moved almost all of the console infrastructure out of the kernel recently and into the user space, and that definitely has happened.
I suspect Linux actually can't move it to user space. It would break binary compatibility with all user space code that sets OPOST via the ioctl. They can't hide it in a libc either.
To be clear, it's not the terminal emulator (xterm, mintty, iTerm2, etc.) that does this. It's the tty layer. You can use the stty command to change the settings.
One of the first challenges I faced as someone who got paid to program: port the VMS "help" program to SunOS (a *nix). Initial challenge: how to respond to each keystroke rather than wait for enter/return. My boss was intentionally unhelpful and forced me to figure it out more or less all by myself (remember: no web, and hardly any O'Reilly books in 1986). I remain eternally grateful for his judgement to this day.
[...] there are actually a few features that you get for free just from your terminal, without the program needing to do anything special at all.
The things you get for free are: [...]
- backspace
- Ctrl+W, to delete the previous word
- Ctrl+U, to delete the whole line
Linux noob here, so this may be a stupid question, but how does that work under the hood?
The documentation for fgets() says [1]:
> Reads at most count - 1 characters from the given file stream and stores them in the character array pointed to by str. Parsing stops if a newline character is found (in which case str will contain that newline character) or if end-of-file occurs.
Does that mean that, by default, fgets() blocks until the user enters a newline and, before they do, it lets them edit the line buffer using the backspace, CTRL+W and CTRL+U shortcuts?
But there seems to be no guarantee that a program is using fgets() to read an entire line - it could also set count to 2 and read individual characters. Those could not be "unread" anymore when a backspace occurs. Is there some magic that lets fgets() buffer characters internally in such a situation, or would the backspace/line editing functionality just be broken then?
[1] https://en.cppreference.com/w/c/io/fgets