81–90 of 128 posts
Earlier quoted context omitted.
Perl is Awk on steroids :)
Perl 6 is when they then took LSD and at a hang-up, they went to the clinic and came back a s two persons. One said: never again, the other: bring it on.
In case you didn't know: Perl 6 has been renamed to Raku, using the #rakulang tag on social media.
I learned perl in college in the 5.6 days, and did a lot of text processing with it for a time. At some point, I bit the bullet and learned awk, and I've mostly abandoned perl as a result. Why? awk is small enough that it fits in my head, or at least the bits I need every couple of months do. And if I forget, it only takes 20 minutes to put them back in. perl, by contrast, is far too large to fit in my head and comes…
What are you using Perl/awk for that it's needed so often? I've been a software engineer since 2005 and worked my way up to being a VP of Engineering currently and never had to use either Perl or awk (or similar). I often read about these tools on Hackernews and I find it quite mystifying as I manage to have written Java, Scala, C#, SQL, and so on for 15 years and happily never needed them. Is this a certain kind of…
To answer the serious question, it's often enough for a first order solution to any problem where you have line and field separated text. Pulling instances of "something weird" out of log files, for instance, is a great use for awk, especially if the fingerprint of "something weird" is scattered through fields in a line in a way that makes grep cumbersome. Or if it spans a couple of lines, especially if you're dealing with a log from a multi-threaded app where there might be irrelevant lines interleaved with the ones you care about.
I haven't written much production-grade awk, but I've often used it as a tool to understand a problem well enough to write a production-grade solution to a problem or fix bugs in application code.
Earlier quoted context omitted.
What are you using Perl/awk for that it's needed so often? I've been a software engineer since 2005 and worked my way up to being a VP of Engineering currently and never had to use either Perl or awk (or similar). I often read about these tools on Hackernews and I find it quite mystifying as I manage to have written Java, Scala, C#, SQL, and so on for 15 years and happily never needed them. Is this a certain kind of…
I'm sure use cases vary to some degree, but awk clicked for me when I had some data that wasn't formatted well. I didn't use it in a script, I just worked in the shell and kept iterating until it looked good, then redirected it into a text file. I use it for quick one-offs like this all the time now. I've also used awk to get button presses from an input stream on a MIDI controller. For me, I found that the up front…
I'm not proud to admit it, but I've used awk in a couple places where I should have properly used `expect' instead. Except, I don't know expect well and haven't gotten to the point where it was worth the investment in learning it.
Earlier quoted context omitted.
I've used it for this too, but things tend to go south pretty quick if you have escaped or quoted commas. That, unfortunately, is where I usually break down and pull out python.
hehe, I hit that bump in the road too, but maybe I was too stubborn. FWIW, you can get around it. I just took a peek because I couldn't remember how to do it off the top of my head. It looks like I had to use gawk -F ',' -v FPAT='([^,]+)|("[^"/]+")' to get the behavior you're looking for. Seems like I nabbed it from https://www.gnu.org/software/gawk/manual/html_node/Splitting... Agreed that this is the sane point at…
The risk of that seems to be if CSV allows embedded escaped quotes in a quoted string. Does it? I don't know. And CSV is pretty loosely defined. For most people it's probably "whatever Excel emits or ingests".
And I think that's why we're on the same page about pulling out python and using a module where somebody has explored what the corner cases are and dealt with them for us already.
Earlier quoted context omitted.
Awk '{ print $2 }' does LWSP gobbling.. cut -d' ' -f2 doesn't and this difference alone makes awk useful to me on a daily basis. Awk has a hash like perl which is very efficient. I use an awk expression to print uniq as they come in counted through the hash insert on new instead of uniq which prints at end. Awk count unique over 300,000,000 ips was as fast as perl and python and smaller memory footprint
If you just use awk for `{ print $2 }` then I would still prefer `tr -s ' ' | cut -d ' ' -f2`, since both are part of coreutils and with `awk` you would add a additional dependency to your script.
e.g. a regex or even a shell-style pattern match would be very useful. `cut -d ' +' -f2` would get rid of the need for 'tr' in your example.
Even just allowing >1 char delimiters would be helpful. A text list like 'foo, bar, foobar' is too much for cut, but it would be fine if cut accepted parameters like `cut -d ', '`
I learned perl in college in the 5.6 days, and did a lot of text processing with it for a time. At some point, I bit the bullet and learned awk, and I've mostly abandoned perl as a result. Why? awk is small enough that it fits in my head, or at least the bits I need every couple of months do. And if I forget, it only takes 20 minutes to put them back in. perl, by contrast, is far too large to fit in my head and comes…
What are you using Perl/awk for that it's needed so often? I've been a software engineer since 2005 and worked my way up to being a VP of Engineering currently and never had to use either Perl or awk (or similar). I often read about these tools on Hackernews and I find it quite mystifying as I manage to have written Java, Scala, C#, SQL, and so on for 15 years and happily never needed them. Is this a certain kind of…
I mention DevOps because it is hard to think of a situation where you’re just coding Java and C# all day that truly requires awk-or-similar. But just the other day I had a situation where I copied the contents of a CD-ROM to a web server — drag and drop on my Mac — and ended up with the files all in upper case. The program I was trying to run was failing because it requested everything with lower-case URLs. There were hundreds and hundreds of files in a bunch of directories, so I fixed it with Perl in about 30 seconds.
I’m curious if that’s the sort of problem you never encounter, or if it is, what you reach for.
I learned perl in college in the 5.6 days, and did a lot of text processing with it for a time. At some point, I bit the bullet and learned awk, and I've mostly abandoned perl as a result. Why? awk is small enough that it fits in my head, or at least the bits I need every couple of months do. And if I forget, it only takes 20 minutes to put them back in. perl, by contrast, is far too large to fit in my head and comes…
What are you using Perl/awk for that it's needed so often? I've been a software engineer since 2005 and worked my way up to being a VP of Engineering currently and never had to use either Perl or awk (or similar). I often read about these tools on Hackernews and I find it quite mystifying as I manage to have written Java, Scala, C#, SQL, and so on for 15 years and happily never needed them. Is this a certain kind of…
Awk/sed and the ETL stack that comes with every Unix-like OS (aka od, tr, cut, sort) and all that are superb data wrangling tools. Log file parsing, data cleaning, even actual data science at scale can be done with these tools. They're extremely efficient, as they're designed from an era when pretty much all interesting data was comparable to or much larger than memory size. As such, you can do a lot of stuff with them that most people don't imagine is even possible. FWIIW for high end data scientists; I don't consider knowledge of these tools to be optional at all. Anyone who hasn't used them in their career hasn't worked on serious problems, and is probably the kind of educated idiot who will suggest you do a job in a giant Hadoop cluster that you could easily do on one machine.
Earlier quoted context omitted.
What are you using Perl/awk for that it's needed so often? I've been a software engineer since 2005 and worked my way up to being a VP of Engineering currently and never had to use either Perl or awk (or similar). I often read about these tools on Hackernews and I find it quite mystifying as I manage to have written Java, Scala, C#, SQL, and so on for 15 years and happily never needed them. Is this a certain kind of…
There are hundreds of use-cases. It is not important what the use-cases are. - There are two types of developers: Those who use hacking to solve all kinds of tasks that need hundreds of manual steps. And those who never think of automating smaller steps. The second kind of devs can be very good software engineers, but they prefer IDEs instead of tweaking Vim or Emacs. The first kind of dev will look for possiblities…
I think it depends a lot on the sort of software one works with. A main use of awk and other unix tools for me is as-hoc data munging, looking through production logs, taking bits of data from different sources and comparing them. If you make eg gui applications and sell them or work as a contractor developing in-house applications for clients, you probably don’t see a lot of production logs like that or process them that way. Any data that you expect to process ought to go into a database and then you can use sql (where joins work much better than the unix join command and you can use the actual structure of the data instead of trying to tease it out with the smallest simplest code you can think of). You might be using a debugger or backtracks or reproduction in test to deal with/investigate production issues rather than starting with logs. On the other hand, many other companies will have big production systems that run on lots of different virtual machines and produce lots of logs and have issues that are hard to reproduce (or maybe the logs are easier), and have lots of data spread about random places such that it may be easier to do the hacky thing for one-off cases or to prototype or whatever rather than Doing things more thoroughly or properly. And this set up will also lead to tools that are designed to fit into the rest of the unix ETL tack by the way they output or input data.
On the second point, I think that even as a die-hard emacs user I wouldn’t recommend that anyone try to use emacs as an ide for something like java (or maybe c++). I’d probably try to use emacs a myself but I think my java-editing experience would be worse than for those people who do it with an IDE. More realistically, I’d try to avoid writing java if at all possible.