I think you did a good job delineating top complaints people have about shell, but I have to reject the pro/con framing.

Shell operates quite naturally under stream-oriented data-centric design patterns. But that's an architectural familiar to almost nobody in our OOP- and FP-happy industry these days. Try using Python like a relational database language (e.g. SQL), and complex stuff gets messy, the syntax starts feeling hostile, logic becomes obscure, etc. The purported ickyness of shell stems not from shell per se but from trying to force it into a foreign design space.

Principles that work well with shell: Heavily normalize your data; Organize it into line-oriented tables with whitespace-separated fields; Use the filesystem as a blob store; Filenames can be structured data as well; positional parameters form a bona fide list.

> hard to escape everything correctly

IME, all of the really painful escape patterns can always be avoided. You see these a lot when people try to build commands and pass them off to exec. That's mostly unnecessary when you can just shove the arguments in the positional parameters via the set builtin: e.g.

    set -- "$@" --file 'filename with spaces.pdf'
    set -- "$@" 'data|blob with "dangerous" characters'
    set -- "$@" "$etc"
    some_command "$@"

which is functionally equivalent to

    some_command "$@" --file 'filename with spaces.pdf' \
        'data|blob with "dangerous" characters' \
        "$etc"

It also helps to rtfm and internalize the parsing steps that happen between line of input and the eventual execve call.

Pro - there is still not another language (generalizing to "shell" here) that allows you to easily write a series of commands and/or pipelines as concisely and understandably as shell.

This is the biggest reason to use shell - if your task is shaped like a command line session with a bit of looping or a few conditionals, shell is perfect.

If you start nesting loops or conditionals in shell, then start considering another language.