I learned Awk in 1988 before Perl was around (on our systems, anyway). It was super useful at the time. But if you know Perl and Perl is available on your system, there's certainly not a compelling need for writing standalone, multi-line Awk programs. But Awk is really, really useful for one-liners. As Larry Wall has said: "I still say awk '{print $1}' a lot."
Brian Kernighan himself, in this 2015 talk [1] on language design, states that Awk was primarily intended for one-liner usage (he mentions this at 20:43).
I don't have it in writing or video, but in 2008 at ACM Reflections/Projections at University of Illinois I was involved in a long conversation with Larry Wall and Al Aho. It was largely about the history and lineage of programming languages.
Al said that if Perl had existed first there wouldn't have been an awk. I pointed out that parts of Perl are inspired by awk and might have otherwise been inspired by SNOBOL or ICON, at which point everyone present seemed to agree means we're thankful for awk. I take it as high praise when Al Aho defers to your tool.
I was just reminiscing with Larry about that discussion last week at The Perl Conference in Arlington. He said he had fond memories of that conversation and that he and Al went for lunch the next day after that conversation, too. I'd have loved to be there for that.
Once I learned Perl I never used awk or sed again. Even for one liners with the -n -p -a options you can easily write one liners in Perl that are concise as those in Awk.
I've written reasonably complicated stuff in awk (like a page or two of code). Probably could have solved those problems more elegantly with another tool, but I never learned perl, and I find awk simple enough that the man page is all I need to refresh my memory. For text manipulation that's a one-off or likely to not need further maintenance, I think it's great.
Back at Netcraft for the data crunching pipelines for the surveys we tended to start off with sed+awk for expressivity/concision of operations and then rewrite in perl later for performance.
The nice thing about awk is that it's really only suitable for simple record processing. You therefore stop using fairly quickly when the problem reaches sufficient complexity to use a better suited language.
It's like a very neat, small domain specific language for use processing simple record based text files. And within those limits it's super useful.
>The nice thing about awk is that it's really only suitable for simple record processing.
Not exactly only simple processing, IMO. Record processing, yes - it is a domain specific language, and its core feature is the pattern-action thing, and it was (at least originally) meant mainly to operate on record-oriented text files (even if on lines, since a line is a record with one column). I don't have good examples off the top of my head, but can mention a few things:
The books The Unix Programming Environment by Kernighan and Pike, and either Programming Pearls or More Programming Pearls, by Jon Bentley, have some examples of advanced uses of awk - so it is not just for simple record processing. And I've read somewhere that just shell and awk and other Unix tools have even been used to create a DBMS of sorts. Plus, with later versions like GNU awk and so on, they've added more features to the language, including probably many more built-in functions, and also some network programming support [1].
According to an answer at the above link, an nroff-subset text formatter and Lisp subset interpreter have been written in awk. Those are more complex than simple record processing.
Right, I mean it's Turing complete, and you can do anything you want. But at least when I use it, I quickly reach for another language after using it for basic reformatting/simple calculations because using awk to do so would be too difficult and awkward.
Yeah it's basically "I know Perl and I don't know Awk". I'm surprised it got to the front page.
Obviously, the two languages overlap in functionality, and the one you know is going to be easier for the job (for you).
If you know neither, Awk is easier to learn, but that's of course because it does less. That may be good or bad, depending on what you are trying to do.
While I agree with the sentiment of the article, it misses a larger argument. Perl is reasonably good at "making common things easy and hard things possible". Meaning if the needs might vary, it's not a stretch to evolve the code to match the requirements.
And despite having a strong affinity for Perl, I don't think it's productive to just bannish Awk to the trash: part of being well-rounded is knowing many tools—which can overlap to some degree—and appreciating the sweet spot of each one. For some one-liners, Awk is just as (if not more) elegant as Perl.
Could you give some examples of one-liners you feel might be more elegant in Awk than in Perl? It might be fun to get a thread going where people try to give clean Perl versions.
Awk is not limited to $0 to $9; where did you get that??? Maybe you're confusing this with register substitutions in regex: \1 through \9.
The nice thing about the @F array is that @F[0] doesn't represent the whole record.
There is a class of Awk bugs whereby arithmetic is being done on the parameters using $N (where N is some calculated variable containing an integer). Everything is fine when N >= 1, but if there is a bug where N is accidentally zero, then $N refers to the whole record instead of throwing an out-of-bounds error.
And things are accidentally zero quite easily in awk, e.g.
awk '{print $X}' # X is undefined so serves as zero; this prints all lines
Yep, there is also /usr/xpg4/bin/awk vs /usr/ucb/bin/awk etc...
I've long since had to deal with Solaris but for about 10 years of Solaris I always treat awk with a lowest common denominator approach. Which means I tend to avoid it even to this day.
I think awk is great, i enjoy working with awk. The GNU version is indespensible because of multidimensional array support of course. Perhaps I enjoy the way it feels like driving an antique car, it really harkens back to a bygone age. Yet its fast.
I dont want to invest the time in Perl, when there are better general purpose languages out there. Perl certainly has a different look, I can understand why some would love it.
I find Perl extremely useful for making prototypes really quickly, and never learned how to use Awk because I only started with Perl in the first place a year or two ago. I'm sure it's just a matter of preference.
My time with Perl was wrestling with code that people wrote to show how good they were at Perl, and where maintainability was a secondary concern. This is a bit harder to do in Awk (but not impossible). I moved from Perl to Python in the 1.5 days in part because of this.
I believe this occurs with Java now too. Neither Perl nor Java are bad languages, they are both powerful and performant. However there are some architecture astronauts who seem to enjoy making life harder for the rest of us.
To me that question never really arose. I learned Perl before I knew the shell well enough to make use of awk or sed. By the time my shell scripts hit the complexity ceiling, I happily switched to Perl and haven't regretted it.
The shell scripts I write are usually just canned commands, maybe some tweaking with environment variables. Anything more complex I usually handle using Perl.
Shell scripts supposedly are more portable because the Bourne shell is always there, but that only is true with different Unix flavours. Once you run into a Windows box, (or more exotic systems like OS/400 (whatever IBM calls it these days), VMS, z/OS, BS2000), shell scripts won't help you. Perl will. ;-)
Last but not least: CPAN. There are perl modules available for nearly anything you could ask for. (Except talking to SharePoint, which I sadly have to do on a fairly regular basis. OTOH, having known its horrors, I can understand why the Perl community wants to stay away from that POS.)
There is no reason to deal with multiple flavors of Awk when you have Perl, especially when you are working with different operating systems. That and Perl's regex which is the defacto across multiple languages, made me completed do away with Awk.
Yes, Ruby was inspired quite a bit by Perl. Matz says so somewhere, IIRC. Even it's regex support (I mean the way it is integrated into the language, unlike in Python, not that fact that Ruby supports regex per se), and some language features, are inspired by Perl. Also either or both of Lisp and Smalltalk, I've read.
Ruby does not have $_ which you happen not to see in perl one liners because of its implicit nature, and that makes those one liners very clear and terse.
The author of this article makes himself appear bat-shit crazy for his claims about Perl syntax being better.
In Awk, you can define a function like this:
function add(a, m, n,
sum)
{
for (sum = 0; m < n; m++)
sum += a[m]
return sum
}
Named parameters (wow, there is a concept); and few dermatological problems: no damn sigils, or required statement-terminating semicolons. The array reference is a[m], the scalar is m and so on.
Using an extra parameter (which is not specified in the call) for the local variable sum is a massive design fail, but no worse than any of the Perl design fails.
Perl repeats most of the design flaws in Awk, with compounded interest.
When I need to do something like this I usually use sed. It's similar enough to vim that I don't have to remember too much. I've never liked perl at all and awk doesn't offer that much more than sed. Maybe that'd be different if this sort of wrangling was a steady diet but it isn't. If I was gonna use awk I'd just skip straight on through to python.
Not much; just, oh, stuff like floating-point math and trig functions; integer math, associative arrays, access to environment variables, named functions with arguments and full recursion; control and looping structures like if, for and while, file I/O and redirection, string literals with C escapes, ...
Holy double angle formula, Batman. Awk has command line trig functions? Riddle me this. Is the Google search bar just an Awk script and they won't tell you? That's just the sneaky kind of trick L+S would try.
The one huge thing that sed is particularly bad at (unless I don't know the particular trick yet) is to split fields on a pattern. I frequently have pipes where sed is used for most of the text-processing, but then cut is used to select one (or multiple) character-separated fields, or awk is used to select one (or multiple) whitespace-separated fields. I only go one step further (usually to Perl) when operations involve multiple lines, e.g. when columns need to be summed up.
I love awk, it's my go-to. Somehow learning perl sufficiently (yet another programming language) is more mental work I'd rather not undertake, what with its idiosyncracies and often implicit nature.
I find awk easy to understand when I read a script after not seeing it for a long time (though I have to google each time I'm writing string manipulation and use arrays) .
At work I grapple with an HPC job scheduler written as a set of concurrent Perl scripts coordinating through transactions in a Postgres database. What a disaster.
Still, it is good fortune that the scheduler was not written in awk.
I wondered at first because the text had some gaps in it, and thought that images were not loaded because of my strict uMatrix policy. After enabling the usual suspects (ajax.googleapis.com) didn't work, I inspected the gap and saw that it was just an ad container. And in fact, the same ad service is advertised below the article text.
Brian Kernighan himself, in this 2015 talk [1] on language design, states that Awk was primarily intended for one-liner usage (he mentions this at 20:43).
[1] https://youtu.be/Sg4U4r_AgJU?t=19m45s