Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Broken Promises of MRI/REE/YARV

timetobleed.com

Post reply on HN
Print view View original

21–30 of 75 posts

Re: The Broken Promises of MRI/REE/YARV

#21

Post by phillmv »

thibaut_barrere wrote: post #13

I do appreciate the technicality of the article, but I'm not sure to agree with the first point of conclusion: how does it makes MRI (and related) 'fatally flawed' though? (real question). What makes it 1/ irreversible and 2/ bad for today's users? EDIT: as well, I wouldn't stop using Ruby because of that; I would use JRuby or Rubinius or IronRuby (if I understand well, these ones are not affected?)

It's a bit hysterical.

Shit! MRI/YARV/REE are inherently fatally flawed! All that code I have running in production must be a FIGMENT OF MY IMAGINATION! SAVE YOURSELVES

Re: The Broken Promises of MRI/REE/YARV

#22

Post by wonnage »

Can someone dissect this a little more? My understanding is the pointer to str never gets written to the stack, and so str on the heap might get freed before zstream_append_input makes use of it. But how could the GC see this/what is the faulty assumption?

Re: The Broken Promises of MRI/REE/YARV

#23

Post by ssmoot »

judofyr wrote: post #9
[deleted] wrote: post #2

[deleted]

MRI - Matz' Ruby Interpreter (the default Ruby) REE - Ruby Enterprise Edition (unofficial branch of Ruby 1.8) YARV - Yet Another Ruby VM (the VM used in Ruby 1.8) (Although in this case I think he means MRI = 1.8 and YARV = 1.9)

Minor correction, YARV, written by ko1, has never been a part of the Ruby 1.8.x line to my knowledge. Some language features, (mostly stdlib I think) were back-ported in 1.8.7, but 1.8.x is and always has been an interpreter where 1.9.x has always been YARV (a VM).

Useless trivia: Once upon a time Ruby2 was going to be called "Rete" IIRC. Or maybe "Rite"? I doubt it's in any shape to be called a "formal" plan at this point, and who knows if it'll ever actually see the light of day. It was supposed to drop optional parens IIRC, it's even in the original Pickaxe I think, but I doubt that's still on the board. Don't remember what else.

Re: The Broken Promises of MRI/REE/YARV

#24

Post by jjore »

thibaut_barrere wrote: post #13

I do appreciate the technicality of the article, but I'm not sure to agree with the first point of conclusion: how does it makes MRI (and related) 'fatally flawed' though? (real question). What makes it 1/ irreversible and 2/ bad for today's users? EDIT: as well, I wouldn't stop using Ruby because of that; I would use JRuby or Rubinius or IronRuby (if I understand well, these ones are not affected?)

A fairly commonsensical approach is to just require all extension authors to annotate their code properly. At some basic level, this happens with Perl with its oft-maligned DSL for generating C code that happens to do all the right declarations. You might then end up writing your code using more macros. It's certainly not pretty but it is sound.

A plausible rewrite of that function in an XS for ruby would leave the function declaration and wrapper code up to your equivalent of xsubpp to execute your DSL and transform the wrapped code to fully functional C. If you build a C using extension from Perl, you'll find an XS file like http://cpansearch.perl.org/src/SIMON/Devel-Pointer-1.00/Poin... which during the `perl Makefile.PL && make` step is transformed via `xsubpp Pointer.xs > Pointer.c` and then compiled as normal C.

Re: The Broken Promises of MRI/REE/YARV

#25

Post by benblack »

phillmv wrote: post #21
thibaut_barrere wrote: post #13

I do appreciate the technicality of the article, but I'm not sure to agree with the first point of conclusion: how does it makes MRI (and related) 'fatally flawed' though? (real question). What makes it 1/ irreversible and 2/ bad for today's users? EDIT: as well, I wouldn't stop using Ruby because of that; I would use JRuby or Rubinius or IronRuby (if I understand well, these ones are not affected?)

It's a bit hysterical. Shit! MRI/YARV/REE are inherently fatally flawed! All that code I have running in production must be a FIGMENT OF MY IMAGINATION! SAVE YOURSELVES

I am running this code in production, hence it cannot have bugs. QED.

Yours in perpetual bogglement,

Lil' B

Re: The Broken Promises of MRI/REE/YARV

#26

Post by xpaulbettsx »

So, what this really seems to boil down to, is:

The Ruby C API is returning objects that are not correctly reference-counted for a short period of time and are incorrectly subject to GC.

This doesn't seem fatal to me, just not reasonably fixable from the GC side. It might be true, that a new API is needed to hold refs in the C side.

Re: The Broken Promises of MRI/REE/YARV

#27

Post by jjore »

jleader wrote: post #20
dlikhten wrote: post #16

Earlier quoted context omitted.

I fail to see how this is an all hands abandon ship issue. If its a critical issue in all 3 interpreters they should be fixed asap if possible. At worst with a flag. If rubinius/ironruby/jruby have no issues, this may become moot eventually as rubinius is gaining lots of traction recently and is becoming faster by the release outperforming standard ruby vms in many cases.

I think the author has a valid point that the "conservative" garbage collection approach has a flaw in its assumptions about the behavior of C compiler optimizations, and it doesn't sound like something easy to fix without a rewrite (i.e. switching to "accurate" GC). This sort of flaw will continue producing new surprising bugs, potentially any time the code is changed, or any time the compiler's optimizations change…

Right now, doesn't the GC traverse the entire heap and keep all objects where the memory's value looks like it might possibly be a pointer to some other object in memory?

This certainly isn't an awesome solution but couldn't the GC backtrace(3) the current process and look at %eax at all C stack frames to additionally include that value in the "pointers currently plausibly in flight" list?

Re: The Broken Promises of MRI/REE/YARV

#28

Post by eonwe »

wonnage wrote: post #22

Can someone dissect this a little more? My understanding is the pointer to str never gets written to the stack, and so str on the heap might get freed before zstream_append_input makes use of it. But how could the GC see this/what is the faulty assumption?

My understanding is that Ruby GC just runs through its heap of Ruby objects and sees which of them are reachable based on other objects in the Ruby heap and C-stack/registers.

Faulty assumption seems to be that counting references only to RVALUEs (Ruby objects in heap) is enough to determine if a part of memory can be freed. This breaks down in C-extensions where macros extract some part of the object or something pointed by it for use. In this case RSTRING_PTR extracts the C char-array used by str for zstream_append_input to use (lets call it arr).

If zstream_append_input or any calls underneath it tries to allocate a new Ruby object, GC may get called and str (and thus arr) may get freed because there are no references left to it anymore (no heap/stack/register because the register value was overwritten).

And this seems to require all Ruby C-extension writers to lock the objects they're using through macros with RB_GC_GUARD.

Edit: note that there are no references left to str

Re: The Broken Promises of MRI/REE/YARV

#29

Post by koudelka »

benblack wrote: post #25
phillmv wrote: post #21

Earlier quoted context omitted.

It's a bit hysterical. Shit! MRI/YARV/REE are inherently fatally flawed! All that code I have running in production must be a FIGMENT OF MY IMAGINATION! SAVE YOURSELVES

I am running this code in production, hence it cannot have bugs. QED. Yours in perpetual bogglement, Lil' B

The point was clearly not that it has no bugs, but that if something is working to spec, it's working.

Re: The Broken Promises of MRI/REE/YARV

#30

Post by benblack »

xpaulbettsx wrote: post #26

So, what this really seems to boil down to, is: The Ruby C API is returning objects that are not correctly reference-counted for a short period of time and are incorrectly subject to GC. This doesn't seem fatal to me, just not reasonably fixable from the GC side. It might be true, that a new API is needed to hold refs in the C side.

I am apparently in that foolish minority that believes language runtimes should not segfault/corrupt themselves while running correct code. That this problem requires significant effort just to hack around, while actually fixing it would take a major architectural change, is what elevates this from mere "lolwut?" to fatally flawed. There are good alternative runtimes for Ruby, such as the JVM and the CLR, that do not suffer from this problem. Y'all should use them.

Funktacularly yours,

Lil' B

Post reply on HN