In my job in VFX we are dealing with sequences of files constantly, and sometimes being able to do things to parts of the frame range is extremely helpful, and awk is great for that!
Let's say U wanted to delete exr files named like filename.1055.exr where the frame number is not between 1001 and 1100, that could look like this:
This actually just prints the commands out which is nice to check that it looks reasonable, then one can redirect that back to the shell by adding "|sh" to the end to actually run them!
The -F "." bit sets the field separator to dot, so that $2 is the frame-number part of the filename.
This is just the tip of the iceberg! Once you're over the initial learning-hump, there's so much U can do with awk! Great stuff!
"|sh" is such a great thing I completely agree! I'm so glad to have opened your eyes to it! I use it all the time! (And it's not even specific to awk at all really! (xargs/sed/grep/tr/etc are all so helpful here! Heck.. make a python script that prints out commands and whack "|sh" on the end! that can work too! (very slow in comparison to awk tho!))
At work there are complex shell-environment cases where this type of redirection doesn't work because it needs to run in the actual-current-shell, not just a new shell spawned for your user which is what "|sh" does.
To work around that you can instead redirect the output to a file and source it into the current shell instead! It is a totally easy thing to do as long as u have somewhere you can write a temporary script file to! I use a 'tmp' dir in my home directory to do this (via an alias of course), like:
command > ~/tmp/bonza.sh; source ~/tmp/bonza.sh
Hopefully you won't have to worry about that in the short-term but I think it's good to keep in mind as an option when things don't work with "|sh" in more complex shell environments... generally I think it's really quite easy to generate a runnable script-file using redirection-to-awk, when u know how to use it! Very well worth the effort to learn!!
While serving time in HPC I saw a set of ETL jobs that were written using awk and xargs that beat the pants off a lot of in house code on both a performance and a simplicity standpoint. You lose a bit of "safety" when you take that approach (yes, I know you can do it in awk), but if you tightly control the inputs you don't need as many guard rails.
It's both unsurprising and amazing to me that a well crafted shell script can be highly performant.
Totally get your point about being able to go faster with tight specification of the expected inputs!
And yeah awk is renowned for being able to crunch through things (especially huge volumes of text) at quite an astounding rate for a scripting language!
Let's say U wanted to delete exr files named like filename.1055.exr where the frame number is not between 1001 and 1100, that could look like this:
ls -1 *.exr | awk -F "." '{if($2 < 1001 || $2 > 1100) print "rm -v "$0}'
This actually just prints the commands out which is nice to check that it looks reasonable, then one can redirect that back to the shell by adding "|sh" to the end to actually run them!
The -F "." bit sets the field separator to dot, so that $2 is the frame-number part of the filename.
This is just the tip of the iceberg! Once you're over the initial learning-hump, there's so much U can do with awk! Great stuff!