Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Ask HN: Should I learn Awk or Perl? Or is that a bad question?
12 points by AnthOlei on July 10, 2024 | hide | past | favorite | 36 comments
I find myself being totally lost on the command line when I need to do more complex stuff - a series of pipes between greps and cuts and the works. Recently, I finally took some time to try to learn some awk and it was life changing. I’ve been doing some awk puzzles, but it occurred to me that if I’ll be deliberately spending time learning a command line text processing tool, I may as well go the extra mile and learn perl, which I’ve heard is far more expressive than gawk.

I’m also noticing a lack of literature on Perl on the command line, I.e “perl -nae <some script>”.

Is awk just better for this use case?



The case for Perl is just that it has no limits. You will never run out of tool if you go with Perl. That is what, I believe, makes the investment worthwhile.

Why Perl gets such a bad rap, then? I'll tell you.

I've used Perl for almost 30 years now, and I've never found any limits in the tool. This means every limit I found was _mine_. Many programmers feel bad when they find their own limits, but it doesn't have to be the case.

You should be OK with finding your own limits, and be willing to overcome them, and strive to become a Perl power user, just like the founder, Larry Wall.

If you choose this, to grow and improve and be better, and better, and better, then Perl is the best option.

It's up to you.


This is good insight. Limits are often built into programming languages. Java is all about the limits. TS's selling point is literally the limits. As well as every opinionated language.

I wonder if AI will make limitless languages easier to work with and maintain.


I'm not against limits in programming languages. They help to reduce the complexity that have to be managed.

Limited languages are suitable for those that are willing to manage only so much. That's why Java has no "goto".

But Perl does offer "goto". Why? Because Larry Wall built Perl for himself, and he wanted the most powerful tool that he could conceive. That's exactly what Perl is.

And I accept that, and I really want that. Every time I mess up with Perl, I don't blame my tool. I just laugh out loud and exclaim: "Thanks, Larry, for letting me know another way to not be up to my job".

Perl is the magic wand. If you don't want to be a magician, you don't need it.

But if you want to be a magician, you'll be thrilled to know Perl has three forms of "goto", not just one. This is what the textbook says about the third one: "The goto &NAME form is highly magical and sufficiently removed from the ordinary goto to exempt its users from the opprobium to which goto users are customarily subjected".


You can also break from a loop from a subroutine.

  sub test() {last unless $_}
  for (1, 2, 0, 3) {
    print "$_\n";
    test;
  }
  // 1 2 0
I miss it :(


To me, having to be the compiler and have 100% test coverage for typos is a limit. TS is a lame, misguided attempt to correct that to let people keep using such limited languages for things.


My reason to break up with Perl was the lack of garbage collection and then later the lack of types and then later 6.

I don’t think investing in it is wise when you can invest in something more relevant. Although there’s barely anything that covers the subj as much as Perl, which is a shame on us all. A programming language that requires an import or two to dig through a file is degenerate. Old languages understood well what a programmer needs.


There are two types of programming, and each has its own definition of "relevant": new programming, with its preference for ready-made components to reduce complexity, and old-time creative programming, which usually was about the creation of new things.

The "creation" part is key. Perl is a tool for "easy creation", for creators that want to be more productive. This is what Larry Wall wanted.

This is why Perl is good for prototyping. You "create" the prototype, and then you "translate" it to something else.

Perl is quite relevant for me, but I say this as a creator.


I'm a practical man, for me it's just n_loc difference and more useful imlicitness (and the listed downsides). Not sure if I can understand this "creation" part, even as a former Perl-ist. I think you can turn any language into Perl, it just happens that their core beliefs are incompatible with its ways.

For example, I can easily imagine something like this:

  import 'use-perl'

  while (await ls_lr('./*.txt')) {
    while (await read_lines($filename)) {
      print($ln, $line)
But it's not idiomatic as it uses awful implicit iterators and hideous globals. This conflict is purely dogmatic and doesn't need special syntax, for the most part.


The "creative" part is all decisions to be made before coding: architecture, design, modularization, algorithms, APIs, etc.

If I am in charge of those decisions, I can use Perl to build a prototype, to validate the solution I am creating.

But if I don't decide anything, my job is to code whatever. No chance for me to be creative.

Most quirks of Perl are there to make the construction of prototypes easier and faster.


To know more about creators and their tools, I recommend:

- "Programming is (should be) fun!", by Gerald Jay Sussman. https://m.youtube.com/watch?v=2MYzvQ1v8Ww

- "On Writing", by Stephen King. Just the chapter "Toolbox".


AWK is better suited to command line ad-hoc and shell script use. it is simple, straight-forward, and surprisingly powerful. most of the time sh+awk can replace sh+sed+cut+grep constructs. GNU awk (gawk) flavor is ubiquitous and it has a great manual [1]. and you have the AWK book [2] (not "free", but easily found).

perl is more of a completely replacement for sh+awk for a standalone script/program. it was the first scripting language i know of that could do sockets networking and other complex things you'd previously need to write in c. it has extremely powerful text processing capabilities, like awk. but it's a big complicated, unusual scripting language. if you want a big complex full-featured scripting language to make a complete tool these days, why not use Python? if you only need really fancy text manipulation, maybe Perl, but i don't see that use a lot.

[1] https://www.gnu.org/software/gawk/manual/html_node/index.htm... [2] https://awk.dev/


IMHO, Ruby is a much better Perl replacement than Python.

Ruby is heavily inspired by Perl and lets you do a lot of work with simple one liners embedded in Bash scripts.

It's also great for longer standalone programs and there is a larger community.

Lots of interesting Perl-isms are also doable in Ruby.


>noticing a lack of literature on Perl on the command line

I have written a book on Perl one-liners with plenty of examples and exercises [0]. I've also written books on CLI text processing tools like grep, sed, awk and coreutils [1]. If you prefer just solving exercises, check out my interactive TUI apps [2].

>Is awk just better for this use case?

It depends on the kind of tasks you'd need to solve. I generally prefer grep, sed and awk first. Perl helps if I need powerful regexp and other niceties like the huge number of built-in functions and access to third-party libraries.

[0] https://learnbyexample.github.io/learn_perl_oneliners/

[1] https://github.com/learnbyexample/scripting_course#ebooks

[2] https://github.com/learnbyexample/TUI-apps


Oh wow. I went though some of the perl book you linked and I was noticing the examples were really familiar; I then realized you were the same guy who wrote the awk book I have been going though.

Your work is excellent! Thank you, I’ll buy a copy soon.



this is amazing! I guess I was searing the wrong keywords. Thanks, I’ll evaluate these.


I highly recommend awk because you can learn it in 20 minutes [1]

So you can start by spending 20 minutes to learn awk, and then spend 20 years to learn Perl (and use awk in the mean time)

[1]: https://ferd.ca/awk-in-20-minutes.html


Having learned both at an intermediate level, I use awk for structured data handling, Perl for textual pattern matching.

Perl was inspired by awk, not the other way around. Perl is more general purpose.


this was my understanding as well. perl grew out of sed/awk/bash, so might as well learn perl.

that said, i found it gives a lot of 'foot-guns'.


>> i found it gives a lot of 'foot-guns'.

Most of those are for backward compatilibity with earlier versions.

Recommend you place the following at the top of your Perl scripts:

    use strict;
    use warnings;
These are recommended for all scripts: https://learn.perl.org/first_steps/


Perl is far more capable than awk and more portable.

The issue with awk is that there are multiple non-interoperable implementations:

https://en.wikipedia.org/wiki/AWK#Versions_and_implementatio...

https://stackoverflow.com/questions/40409632/what-is-the-dif...

This makes awk scripts less portable than other text processing tools.

awk is also not extensible--it can be awkward (heh) to adapt it to some problems and big scripts get difficult to wrangle.

awk is great if your text processing problem is small.

Sometimes small problems grow in which case Perl tends to be a better choice:

https://stackoverflow.com/questions/366980/what-are-the-diff...

People love to hate perl, but there is a reason why it was installed on all Unix and Linux systems by default and was so popular for web servers on the early Internet (e.g. Apache mod_perl https://perl.apache.org/ and perl mason https://www.perl.com/pub/2002/12/11/mason.html/ ).

Perl also popularized regular expressions as a standard component of programming languages so much that "Perl-compatible regular expressions" are probably the most widely-used regex flavor versus POSIX or other regex variants:

http://www.pcre.org/

https://en.wikipedia.org/wiki/Regular_expression


> but there is a reason why it was installed on all Unix and Linux systems by default

AWK has that one covered fair and square, even BusyBox has AWk.


If you're looking at learning Perl then you'll find the https://perldoc.perl.org/ site very useful (you'll probably also have its content available on the command line via the perldoc command).

If you want more information about running Perl on the command line then start with https://perldoc.perl.org/perlrun (or `perldoc perlrun` in your shell)


Maybe not so much a question of 'better' but perl is definitely bigger and multiheaded and does many, many more things (perl scientists were so preoccupied with whether or not they could, they didn't stop to think if they should). So you'll spend a lot more time (compared to awk) orienting yourself, beside the learning of the thing you want to know about. That could be a fun extra hobby or a waste of time, depending on your goals and inclinations.


Awk for sure! The pdf for the original awk book is somewhere around the web, i learned awk from that and it’s mostly still good. I heard they made a new edition some time ago? Really worth reading!

For perl… not sure. I learned a bit of perl in high school and never had a chance to use it, even at work where i occasionally see it being used from other people.


How much time do you want to invest ?

Perl is more powerful but learning and remembering all (or a sufficiently more powerful superset of awk's capabilities) of it is going to take time. The peculiarities of its syntax is certainly not a small set.

All of awk on the other hand needs about 2 hours, if you have done some programming before.


>> Perl is more powerful but learning and remembering all (or a sufficiently more powerful superset of awk's capabilities) of it is going to take time. The peculiarities of its syntax is certainly not a small set.

I disagree.

Perl's syntax is easy to learn for anyone who has used C-style syntax as found in C, C++, Java, Javascript, or C#:

https://perldoc.perl.org/perlintro

Perl's syntax is more similar to AWK and C-style languages than Python is.

That said, there is nothing wrong with Python or AWK. They work great and have their places. But it is silly how many people treat Perl like it is impossible to learn or read without ever having used it or seeing where it is used.


Its OK to disagree.

I am quite comfortable with C and C++ and still find it hard to hold all of Perl and its operators and naming rules in my head, to keep $s, @, % and .(dot) and quotes etc straight. Granted it makes types more obvious than say Python.

AWK is smaller. There are fewer things to remember.

Perhaps a good comparison would be the size of a Perl pocket guide and a AWK pocket guide. Perl and Python are much bigger subsets of AWK.


Perl for sure, because it's much better than awk. Also perl has a huge set of libraries, awk has nothing.


The tools have a lot of overlap. No need to choose one over the other, you can learn both. I don't write Perl but knowing how to read it comes in handy. AWK usually does what I need.


i feel like awk is better for simple pipings and the likes, so i guess it depends on what you need to do. I find myself using awk a lot expecially with other programs


use Python, so that other people can read it


But can they install the depencies and actually run it? :-)


If you just want to do what awk can do, then you do not really need other packages.


i wouldnt go for perl...id suggest the better question to be awk vs python, in which case id go for python


This is a saner answer than you might think.

In theory Python is the more verbose of these languages. But when I switched from Perl to Python way back when, my short scripts actually got shorter, were quicker and easier to write, and you could actually read them back and understand what they said!

Of course both Perl and Python have changed a lot since I last compared them, so YMMV these days somewhat. I'd be interested to hear from a modern perl-er!

(And: I do still use PCRE libraries sometimes :-P )




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: