Totally Awk-some
Please forgive the trite / hackneyed / banal title as well as any other annoying platitudes that may be "wearing a bit thin" contained in this post. It's the best I could do.
I've been messing around with gathering configuration / performance information from remote machines via SSH for processing by a product in some third party framework whose name I can't remember currently. I was running a command that produced a multi-column, tab delimited table of data. I wanted to get the number of occurrences of each distinct string value in one of the columns.
For example (this is not what I was doing, but it's a rather innocuous example), let's say I wanted to run ps -ef and get a count of how many processes belong to each UID along with a total of the number of processes. Pretty much just generate totals for occurrences of a key. For convenience's sake it would be nice if I could have done all of that on the remote machine, quickly and easily, rather than sucking everything back and marching in some heavyweight tool to do my bidding. No offense to any of you heavyweight tools in the audience.
Now, I've farted around with, and been very impressed by, awk in the past. I was pretty sure I could use it to get this sort of thing, but I didn't realize how gosh darn easy it would be.
In my contrived example, you could just do something like:
ps -ef | \
awk '{ processes[$1]++ processes["total"]++ } END \
{ for (user in processes) \
printf "%s\t%d\n", user, processes[user] }'
On my test machine this outputs something like:
lp 1
total 51
UID 1
root 47
daemon 2
Oh, but I meant all non-root processes. Wapow!
ps -ef | awk '$1 !~ /^root$/ \
{processes[$1]++ processes["total"]++ } END \
{ for (user in processes) \
printf "%s\t%d\n", user, processes[user] }'
Ok. If you're a *nix user you're probably saying something like "BFD" right about now, and rightly so. I'm just a poor old Windows user marvelling at the combination of power and syntactic terseness provided by awk (and default Unix tools in general). Man, that's some spicy meatball.










March 14th, 2006 at 8:56 pm
Yeah, you could pipe it to 'grep -v root' as well. Same difference. Simple little command line tools can do so much…
March 14th, 2006 at 10:30 pm
To paraphrase: Come to me, Superman! I defy you! Come and kneel before Sed! Sed!
March 14th, 2006 at 10:45 pm
I definitely think sed is ultra sweet (as is grep, uniq, wc, etc). It's enough to give knowledgeable Windows users serious shell envy. Luckily most of them (us?) toil in ignorance.
March 14th, 2006 at 10:48 pm
I might still have my old Sed/Awk book if you want it.
As Eric will attest to, I had a brief romance with Awk a few years ago.
September 2nd, 2008 at 3:48 pm
[...] Every so often I have a brief love affair with awk. Today I got curious about the file sizes beneath a directory. In particular I wanted to see the totals by file extension. I did a quick search but came up with nothing. I decided that even if there is something out there to do the job, it'd be a lot more fun to do it myself. Tada: ls -Rl | grep ^- | awk '{ split($9,e,"."); exts[e[length(e)==1?2:length(e)]]+=$5 } END { for (ext in exts) printf "%10d %sn", exts[ext], ext }' | sort [...]