Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rewriting the Ruby parser

railsatscale.com

Post reply on HN
Print view View original

151–160 of 182 posts

Re: Rewriting the Ruby parser

#151

Post by skitter »

RhodesianHunter wrote: post #139
cout wrote: post #39

With the mention of the parser's performance, I am reminded of Rich Kilmer's 2004 RubyConf presentation. Rich used Ruby to test reliability of a distributed system with hundreds of Java VMs. Some of the serialized data was stored as XML (over 1M lines!), which was slow to parse and load. Rich modified the program to serialize the data as Ruby code, which loaded much faster ( https://www.infoq.com/news/2007/06/infoq-i…

>Some of the serialized data was stored as XML (over 1M lines!), which was slow to parse and load. Rich modified the program to serialize the data as Ruby code, which loaded much faster So he took a data format designed for human readability and converted it to a data format that's designed purely to be read by Ruby and people are surprised that it's faster?

I think a bigger difference is that the XML is parsed by a Ruby program and the generated Ruby code is parsed by a C program.

Re: Rewriting the Ruby parser

#152

Post by vidarh »

bbkane wrote: post #125
PaulHoule wrote: post #74

It’s little appreciated how much current parser generators are holding the industry back. That is, LISP maintains a lead in metaprogramming because it bypasses Chomskyism alltogether. It really should be a few lines of code to add an “unless(X) {}” statement to “if(!X) ()” to Java, Python, Ruby but the very idea that you could patch an existing grammar with a separate file is like technology that fell off a UFO. Also…

I think in most cases metaprogramming makes applications much harder to read. When taking over someone else's codebase, I don't want to also have to understand how they modified the grammar of the language. Obviously, this isn't a blanket statement - metaprogramming has some amazing success stories (homebrew and vagrant both have excellent DSLs), but more often than not I appreciate code that's just "one layer" of ab…

Classes and methods/functions alter the grammar just as much.

Re: Rewriting the Ruby parser

#153

Post by vidarh »

tiffanyh wrote: post #128
kayodelycaon wrote: post #127

Earlier quoted context omitted.

Why? Shopify was able to work with the core team and make the whole language better. Facebook has far more employees and wasn’t going to get PHP improved to their liking.

The irony is that PHP has improved massively in perf vs the relative gains of Ruby. https://onlinephp.io/benchmarks https://serpapi.com/blog/benchmarking-ruby-3-1-yjit-ruby-2-7...

Apart from measuring very different things, the PHP page compares PHP releases over something like 25 years vs. about 3 years of Ruby releases.

Look at PHP 7.2 onwards, which is a similar timeframe, and the improvements are fairly modest.

Re: Rewriting the Ruby parser

#155

Post by vidarh »

stevebmark wrote: post #123

"Rails at scale" is a funny title. Shopify had to rebuild the core parser. Github has a department dedicated to working off the bleeding edge of Rails (and has core Ruby and Rails maintainers on staff, Shopify probably does too). Both Shopify and Github spent literal engineering years upgrading Rails, and still for some reason think it's worthy of boasting about in engineering blog posts, despite the burnout, not-shi…

If you're not in that boat, odds are you don't need to care.

And they didn't stop delivering features for a year.

Re: Rewriting the Ruby parser

#156

Post by foldor »

RhodesianHunter wrote: post #139
cout wrote: post #39

With the mention of the parser's performance, I am reminded of Rich Kilmer's 2004 RubyConf presentation. Rich used Ruby to test reliability of a distributed system with hundreds of Java VMs. Some of the serialized data was stored as XML (over 1M lines!), which was slow to parse and load. Rich modified the program to serialize the data as Ruby code, which loaded much faster ( https://www.infoq.com/news/2007/06/infoq-i…

>Some of the serialized data was stored as XML (over 1M lines!), which was slow to parse and load. Rich modified the program to serialize the data as Ruby code, which loaded much faster So he took a data format designed for human readability and converted it to a data format that's designed purely to be read by Ruby and people are surprised that it's faster?

I would argue that an XML file that's "over 1M lines!" is no longer human readable.

Re: Rewriting the Ruby parser

#157

Post by michaelcampbell »

munificent wrote: post #138
nine_k wrote: post #134

Earlier quoted context omitted.

I'd say that C and C++ were successful despite their being hard to parse, and other unfortunate hacks, like null-terminated strings and include files instead if modules There was little choice for other features they offered (like Unix being written in C and bundling a C compiler) which.were the compelling reason to use them. JavaScript was a huge success not because it was a great language, but because everyone want…

I think you and I are saying essentially the same thing: grammatical complexity seems to have little negative impact on language success. > There was little choice for other features they offered (like Unix being written in C and bundling a C compiler) which.were the compelling reason to use them. Pascal was around at the same time, is grammatically infinitely more elegant... and is dead.

Shame, too; that was a really fun language.

Re: Rewriting the Ruby parser

#158

Post by draegtun »

PaulHoule wrote: post #74

It’s little appreciated how much current parser generators are holding the industry back. That is, LISP maintains a lead in metaprogramming because it bypasses Chomskyism alltogether. It really should be a few lines of code to add an “unless(X) {}” statement to “if(!X) ()” to Java, Python, Ruby but the very idea that you could patch an existing grammar with a separate file is like technology that fell off a UFO. Also…

>> It really should be a few lines of code to add an “unless(X) {}” statement to “if(!X) ()” to...

Or even one line :)

Factor...

  : unless ( ? quot -- res ) swap [ drop ] [ call ] if ; inline
Rebol/Red...

  unless: func [expr block] [either expr [] block]

Re: Rewriting the Ruby parser

#159

Post by nirvdrum »

tiffanyh wrote: post #146
nirvdrum wrote: post #144

Earlier quoted context omitted.

Your phrasing could be misconstrued by those not familiar with the history. Shopify hasn't been developing TruffleRuby for 10 years. The entire TruffleRuby project is 10 years old, including the first 12 - 18 months where it was a humble intern project. TruffleRuby (previously called JRuby+Truffle) was initially a research project for testing out optimizations, not a production-quality implementation suitable for dep…

> “getting a large Rails app booting ” What does ‘booting’ exactly mean in the context of a web app (Rails)?

The Rails initialization process is commonly called "booting", probably owing to the presence of the config/boot.rb file. It's the loading and execution of code necessary before a request can be processed. Beyond the code that Rails executes, it generally includes the loading of most, if not all, of your application's dependencies and any initialization they may require.

In my experience, the Rails boot stage accounts for most of the compatibility work. If you can't boot the app, you won't be able to serve requests and likely can't run tests. E.g., it was at this step that we learned we needed to support more of the native extension API to get memcached running (†). Once booting we did run into some other compatibility issues, but the lion's share came up during the Rails boot process.

† - As an aside, the memcached gem hasn't seen a release since 2014. Its C code attempts to detect the Ruby version and alter how it sets things up. It predates the availability of TruffleRuby so there isn't any TruffleRuby-specific detection logic in it. Our extension compatibility made it look like we supported CRuby's internal object model and that was causing the extension to try to allocate objects in a CRuby-specific way. A small change to a macro fixed the problem and the rest of the gem ran fine. It's one of those things that we can't fix until we see, but once we do we can fix it permanently.

[1] -- https://github.com/oracle/truffleruby/pull/2871

Re: Rewriting the Ruby parser

#160

Post by soulbadguy »

RhodesianHunter wrote: post #141
soulbadguy wrote: post #112

Earlier quoted context omitted.

> It is also a lie to say java is "quite slow" compare to c/c++ Why ?

Because it depends entirely on the use case. There are plenty of cases where Java will be faster than c++ purely due to the available libraries.

> Because it depends entirely on the use case.

Every thing depends on the use case, and one case always find special case as counter example and most engineering solution/choices.

Generalization are still useful and sometime "true". C++ (and native languages) are faster than java simply because it was designed this way. C++ chose speed at the price of complexity, safety and build time. Java on the other hand focused on simplicity and safety.

I understand the idea of wanted to bring nuance, but the"it depends" can also become an excuse for bad tech choices.

> There are plenty of cases where Java will be faster than c++ purely due to the available libraries.

Hard to believe, but in that case i would say that you are comparing libraries, not the languages.

Post reply on HN