Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ruby: A great language for shell scripts

lucasoshiro.github.io

Post reply on HN
Print view View original

311–320 of 368 posts

Re: Ruby: A great language for shell scripts

#311

Post by lucasoshiro »

didip wrote: post #20

No it’s not a great language. Too many ways to do the same thing. Not packaged by default on most Linux. Monkey patching makes things even harder to debug.

You can say similar things for Bash or Python (or Perl, JS, etc):

> Too many ways to do the same thing.

In Bash you can use backticks, $(), and bash -c for the same thing. And also [ ], [[ ]], test. And so on.

> Not packaged by default on most Linux.

You can install it. Some distros (e.g. Alpine) even don't ship Bash. If you are restricted to only what comes by default in an operating system, then probably a simple script written in sh may be the solution.

> Monkey patching makes things even harder to debug.

Just because you can doesn't mean that you should. I won't label something as "hard to debug" for features that I don't need to use, I label as "hard to debug" for features that I _need_ to use. Then I can say: Bash is hard to debug.

Re: Ruby: A great language for shell scripts

#312

Post by lucasoshiro »

globular-toast wrote: post #229

Cool, I'm sold, added Ruby to my to learn list! I use Python a lot but I don't think it's as good for writing scripts as bash or Perl. Ruby looks like it fits that "better but not much harder" category much better.

Well, I'm happy that I you learnt something new here :-).

Ruby is quite similar to Python, perhaps you don't even need to study too much!

Re: Ruby: A great language for shell scripts

#313

Post by lucasoshiro »

Demiurge wrote: post #25

If Rails could be written in Python, I would probably prefer that. Ruby has too much of Perl. If I didn't choose to write a critical piece of software in Perl, which caused me numerous sleepless nights hunting for a missing quote or other weird character, I might have thought it was cool. By my experience with PHP and Perl is why I prefer Python. Clever is not what I ever want a language to be. Python > Lua > PHP > R…

> Python > Lua > PHP > Ruby > JavaScript > Perl > Bash

Those are different languages, with different characteristics. Perhaps the only two that can be directly compared as a 100% replacement to another are Ruby and Python.

Re: Ruby: A great language for shell scripts

#314

Post by lucasoshiro »

rednafi wrote: post #33

Ruby is slow and encourages an esoteric convention over configuration style of OO code. If you enjoy it, more power to you. However, Python is everyone's second favorite or least favorite language, and it runs laps around Ruby any day. Then there's Go if you need some extra oomph!

> Ruby is slow

We're talking about code that calls external commands here. If one wants performance it is already doing wrong by calling external commands. Don't think it is relevant here.

> encourages an esoteric convention over configuration style of OO code.

I can't see that. Its OO is not so different from other languages. Perhaps you are thinking about Rails (as I said in the first paragraph).

Re: Ruby: A great language for shell scripts

#315

Post by portpecos »

lucasoshiro wrote: post #301
xavdid wrote: post #129

Ruby's a great language- I've always enjoyed its ergonomics and clarity. But its editor tooling hasn't kept up with its one-time competitor, Python. I've mostly been in the Python ecosystem for the past few years and the LSP investment from Microsoft has really shown. Rich Python support in VSCode is seamless and simple. Coming back to Ruby after that caught me off guard - it feels like I'm writing syntax-highlighted…

For both Ruby and Python most of time I use Emacs without an LSP, and it works better with Ruby than Python. I never search _exactly_ why Ruby became so less popular than Python, but I think that at least two things: - Having its popularity too much dependent of Rails; - Not having any other killer app (e.g. Python is not only Django, but NumPy, TensorFlow, Pandas, Flask and so on); So, even if I love Ruby, Python is…

Even from a systems perspective there are more tools available in Python. Pyroute2 vs netlinkrb which was last updated 8 years ago and only does a few things. It’s sad because ruby had the potential, but ruby developers prefer to focus on the Rails ecosystem.

Re: Ruby: A great language for shell scripts

#316

Post by e12e »

RulerOf wrote: post #294
e12e wrote: post #283

Earlier quoted context omitted.

I'm currently working with 150MB worth of gzipped JSON - marshalling the full file from JSON to ruby hash eats up a lot of memory. One tweak that allows for easier lazy iteration over the file (while keeping temporary disk Io reasonable) is to pipe it through zcat, jq in stream mode to convert to ndjson, gzip again - for a temp file that ruby zlib can wrap for a stream convenient for lazy iteration per read_line...).…

Hmm. I don't typically mind throwing memory at a problem like that, but I can certainly see the issue. Is lazy marshalling something that other languages handle better?

Some do, some don't. JSON is a special case as a valid JSON file needs to be a single array or object literal - event driven (SaX style) parsing needs to be a hack (like jq stream mode). In theory json_streamer or yajl should help, but I couldn't get a combination to return a proper lazy iterator.

With file as ndjson it was easier, if a little sparsely documented (Zlib::new or #wrap?):

    my_it = Zlib::GzipReader.wrap(some_ndfile).lazy
    obs = my_it.each_line.lazy.map do |line|
    JSON.parse line
  end.first(4)
When we can get a line at a time marshalling the whole line isn't an issue.

My issue is more that it is tricky to nest ruby IO objects and return a lazy iterator - especially nesting custom filters along the way - at least more tricky than it should be.

Apparently there's a third party frame work that does seem promising:

https://iostreams.rocketjob.io/tutorial

Or manual lifting:

https://dev.to/bajena/streaming-gzipped-csv-files-from-ftp-i...

Or:

https://medium.com/smartly-io/streaming-data-with-ruby-enume...

https://github.com/lautis/piperator

I think something more like this should probably be built in, and readily available (for gzip, http, files etc). Maybe I'm greedy.

Btw the shell pipeline to convert a file would be something like this, and is fully streaming:

    # gzipped JSON to gzipped ndjson, stripping top level array:
    gzcat file.json.gz \
     | jq -cn --stream 'fromstream(inputs|(.[0]  |= .[1:]) | select(. != [[]]) )' \
      | gzip -9 \
      > file.ndjson.gzip

Re: Ruby: A great language for shell scripts

#317

Post by Qem »

shmerl wrote: post #114
asa400 wrote: post #108

Earlier quoted context omitted.

> I sometimes wonder why we don't see ruby used for shell stuff more often. The reason we don't see Ruby used more for shell stuff is because Python won this particular war. It's already installed on basically every Linux distribution out there, and this simple fact outweighs all other language considerations for probably >95% of people who are writing shell scripts in something that isn't Bash. Personally, I don't m…

How hard is to install it though? That doesn't sound like a reason not to use it.

Without a root account or inclusion in sudoers list, quite hard. There's millions of people that don't control the machines they work and spend most time with.

Re: Ruby: A great language for shell scripts

#318

Post by pilaf »

igouy wrote: post #297
pilaf wrote: post #63

Earlier quoted context omitted.

Ruby performance has improved a lot in the last few years, especially with the introduction of the JIT (YJIT), and it keeps improving with every new release. I think the notion of Python being the faster of the two may be outdated. You can see some comparisons in the Benchmarks Game [1] (ignore the reference to PHP and Erlang in the URL, seems to be a typo by the website's maintainer, although my link will break if t…

Thanks for the nudge: https://benchmarksgame-team.pages.debian.net/benchmarksgame/...

Haha, thanks! This is why I love HN.

Re: Ruby: A great language for shell scripts

#319

Post by derefr »

copirate wrote: post #203
derefr wrote: post #22

I love using Ruby for shell scripting, but there are also a ton of little nits I have to fix whenever I'm doing it. For example: Ruby has no built-in for "call a subprocess and convert a nonzero exit status into an exception", ala bash `set -e`. So in many of my Ruby scripts there lives this little helper: def system!(*args, **kwargs) r = system(*args, **kwargs) fail "subprocess failed" unless $?.success? r end And I…

> Ruby has no built-in for "call a subprocess and convert a nonzero exit status into an exception" Since Ruby 2.6 you can pass `exception: true` to `system` to make it behave like your `system!`. https://rubyreferences.github.io/rubychanges/2.6.html#system...

Didn't realize that! That's one snippet I can maybe eliminate now. (As to why I didn't know: the first thing in the RDoc for Kernel#system is still "see the docs for Kernel#spawn for options" — and then Kernel#spawn doesn't actually have that one, because it doesn't block until the process quits, and so returns you a pid, not a Process::Status. I stopped looking at the docs for Kernel#system itself a long time ago, just jumping directly to Kernel#spawn...)

But come to think of it, if Kernel#system is just doing a blocking version of Kernel#spawn → Process#wait, then shouldn't Process#wait also take an exception: kwarg now?

And also-also, sadly IO.popen doesn't take this kwarg. (And IO.popen is what I'm actually using most of the time. The system! function above is greatly simplified from the version of the snippet I actually use these days — which involves a DSL for hierarchical serial task execution that logs steps with nesting, and reflects command output from an isolated PTY.)

Re: Ruby: A great language for shell scripts

#320

Post by Alifatisk »

nunez wrote: post #254

I agree. Ruby is a _fantastic_ language for getting things done quickly whose credibility was unfairly maligned by Rails. Unbelievably easy to read, and, with rspec, it is stupid easy to write tests for. No need to fuss with interfaces like you do with Golang; yes, that is the right thing to do, but when you need to ship _now_, it becomes a pain and generates serious boilerplate quickly. I've switched to Golang for m…

Rspec changed my view on writing tests, it’s way to easy now!
Post reply on HN