Live data from Hacker News

Rewriting the Ruby parser

railsatscale.com

121–130 of 182 posts

Re: Rewriting the Ruby parser

#121
post #56

It feels to me like Shopify is single-handedly keeping Ruby alive. A little bit like Jane Street and O'Caml.

People forget Ruby is also massively popular in Japan to this day

>People forget Ruby is also massively popular in Japan to this day

So are fax machines.

Re: Rewriting the Ruby parser

#122

So Shopify spends millions of dollars in engineering salary to get a substandard performing language up to par with a dozen other contemporary better choices? The choice to stick so hard with Ruby just doesn't seem reasonable at all with their scale. It must be something dogmatic from the top.

Facebook did the same when they developed HipHop for PHP. I imagine the reasoning is the same in both cases: spend a few million dollars and a handful of engineers improving the language and tools, or spend hundreds of millions and years of time rewriting everything into a more performant language. I honestly think the math pencils and makes sense.

Re: Rewriting the Ruby parser

#123
"Rails at scale" is a funny title. Shopify had to rebuild the core parser. Github has a department dedicated to working off the bleeding edge of Rails (and has core Ruby and Rails maintainers on staff, Shopify probably does too). Both Shopify and Github spent literal engineering years upgrading Rails, and still for some reason think it's worthy of boasting about in engineering blog posts, despite the burnout, not-shipping-features, and turnover they suffered because of it.

If your company has enough engineering overhead and capacity to stop delivering features and try to fix your core language for a year, then you're in a different ballpark from most other tech companies. If you're NOT in that boat, then "Rails scaled for Github and Shopify" is not a message you should take home.

Re: Rewriting the Ruby parser

#124
post #43

Will this change impact Solargraph / Rubocop? They are painfully slow / unusable in their current state on large-ish projects

I made a Ruby LSP because of this problem. It's not perfect but incase it's helpful for you. It can parse a large project with all of its gems in a few minutes. That data is indexed in an in-memory db with Tantivy. https://github.com/pheen/fuzzy_ruby_server

Re: Rewriting the Ruby parser

#125

It’s little appreciated how much current parser generators are holding the industry back. That is, LISP maintains a lead in metaprogramming because it bypasses Chomskyism alltogether. It really should be a few lines of code to add an “unless(X) {}” statement to “if(!X) ()” to Java, Python, Ruby but the very idea that you could patch an existing grammar with a separate file is like technology that fell off a UFO. Also…

I think in most cases metaprogramming makes applications much harder to read. When taking over someone else's codebase, I don't want to also have to understand how they modified the grammar of the language.

Obviously, this isn't a blanket statement - metaprogramming has some amazing success stories (homebrew and vagrant both have excellent DSLs), but more often than not I appreciate code that's just "one layer" of abstraction.

Re: Rewriting the Ruby parser

#126
Hack?

I wonder if Shopify will ever give up on trying to fix Ruby and instead, do what FB did with PHP and just create their own derivative (Ruby) language that addresses their needs better.

Re: Rewriting the Ruby parser

#127

Hack? I wonder if Shopify will ever give up on trying to fix Ruby and instead, do what FB did with PHP and just create their own derivative (Ruby) language that addresses their needs better.

Why? Shopify was able to work with the core team and make the whole language better.

Facebook has far more employees and wasn’t going to get PHP improved to their liking.

Re: Rewriting the Ruby parser

#128

Hack? I wonder if Shopify will ever give up on trying to fix Ruby and instead, do what FB did with PHP and just create their own derivative (Ruby) language that addresses their needs better.

Why? Shopify was able to work with the core team and make the whole language better. Facebook has far more employees and wasn’t going to get PHP improved to their liking.

The irony is that PHP has improved massively in perf vs the relative gains of Ruby.

https://onlinephp.io/benchmarks

https://serpapi.com/blog/benchmarking-ruby-3-1-yjit-ruby-2-7...

Re: Rewriting the Ruby parser

#129
post #40

Earlier quoted context omitted.

I think ‘is Ruby dying?’ is little more than a meme that has stuck around for longer than it deserves. The continuing work on the language and its performance is impressive, but Ruby (and Rails) themselves have the honour of being stable, tried-and-tested solutions for rapid application development. Is it as exciting as the latest and greatest serverless lambda framework in Typescript? Not really. Is it a dependable…

I don't deny the utility of Rails for building a certain class of web site. But I think there is a difference between Rails and Ruby, and a difference between use of a technology and improvements to that technology. I think a technology must continue to change and adapt to remain relevant. I see Shopify as the main driver of current improvements to Ruby (the core features of the most recent release, and the upcoming…

Smaller companies can be happy with Ruby as it is while enjoying the incremental improvements. They're hardly going to invest in hiring C-proficient engineers to bolster the language - it's simply good enough to start a business from and the performance metric is low priority.

They're in a different class to heavy hitters like Shopify and Github, who will gain a lot more from investing in gradual improvements to Ruby's runtime at the scale they operate at.

I'll contrast it with JavaScript, which has tried to assimilate every language pattern under the sun over the past decade and is intensely difficult to maintain a stable stack with, even if it's better now than it used to be.

Re: Rewriting the Ruby parser

#130

As a devoted rubyist, it seems clear to me in retrospect that ruby's "pretty complex to parse is just fine" approach has been a real challenge to the language's success. For reasons related to discussion in OP. This project could be one approach to ameliorating that challenge, I'm very pleased to see shopify willing to invest in this, and hope it ends up effecting the ecosystem as hoped.

I disagree.

There are many languages that are hard to parse that are very successful. C and C++ are the obvious examples: there are many dark corners around the preprocessor, type annotations, etc. C++ is an absolute nightmare to parse.

C# is pretty complex too. Python has its weirdness around indentation.

Honestly, most popular languages end up acquiring a decent amount of grammatical complexity over the years and it doesn't seem to significantly hinder adoption. Humans are quite good at reading complex text and syntax.

I think the real tax on Ruby is its pervasive use of runtime metaprogramming. It's Ruby's most exciting strength and enables much of the joy and excitement that Ruby is known for. But it makes static analysis so hard and becomes less and less valuable as team and codebase size increases.

Post reply on HN