Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Helix: Rust and Ruby, Without the Glue

blog.skylight.io

Post reply on HN
Print view View original

31–40 of 69 posts

Re: Helix: Rust and Ruby, Without the Glue

#31

Post by lobster_johnson »

aidenn0 wrote: post #18
x5n1 wrote: post #13

Earlier quoted context omitted.

huh? strings have encodings. rust strings are bytes encoded in utf-8. https://doc.rust-lang.org/book/strings.html

In Rust a string is a sequence of unicode scalar values. I personally find it unfortunate that they dictate the storage of it at the API level, but that is a necessary evil for presenting a consistent ABI with foreign code. I did not know that strings in Ruby have encodings. Is there a reason for that? I personally don't like mixing characters and opaque byte sequences as they are very different.

The reason is that Ruby supports non-Unicode encodings that are not subsets of Unicode. Not possible if your string is Unicode.

Re: Helix: Rust and Ruby, Without the Glue

#32

Post by steveklabnik »

MichaelGG wrote: post #21
aidenn0 wrote: post #18

Earlier quoted context omitted.

In Rust a string is a sequence of unicode scalar values. I personally find it unfortunate that they dictate the storage of it at the API level, but that is a necessary evil for presenting a consistent ABI with foreign code. I did not know that strings in Ruby have encodings. Is there a reason for that? I personally don't like mixing characters and opaque byte sequences as they are very different.

Ruby's Japanese heritage is probably why it handles encodings like that - I think there were multiple encs it had to deal with at once or something. Also Unicode doesn't completely handle all kanji in that there's some that have an old style not available in Unicode. But maybe that's not relevant.

Ruby encoding stuff changed a lot over its history; it was one of the big changes from 1.8 to 1.9.

Re: Helix: Rust and Ruby, Without the Glue

#33

Post by steveklabnik »

twelvechairs wrote: post #6

Very excited to see the development of the Ruby/Rust space. The two languages together would seem a joy of a workflow from concept through to maintenance, and the wealth of important Ruby personalities currently involved in Rust (as well as others) encourages that that this will be a well trodden and well documented workflow sooner rather than later. I think this could help Ruby regain some of its early excitement, a…

There's a _lot_ of stuff in this space: mrusty, ruru, Helix. It's exciting stuff.

Re: Helix: Rust and Ruby, Without the Glue

#34

Post by steveklabnik »

MichaelGG wrote: post #4

But the Rust version of the blank check doesn't handle any encoding but UTF-8. Helix could wrap up a char iterator for it I suppose, one that calls rb_enc_codepoint_len? And isn't there some common C lib that exposes Unicode functions like is_whitespace? Granted, using a cargo crate is easier than finding and adding a .h, and far easier than getting and linking another lib.

Akira Matsuda actually suggested at RailsConf that maybe Rails handling non-UTF-8 encodings was not necessary, and maybe phasing it out was a good idea.

I wasn't present for the talk, just saw his slides.

Re: Helix: Rust and Ruby, Without the Glue

#35

Post by donpdonp »

pzh wrote: post #19

Hmm. Kind of makes you wonder why you're using Ruby at all, which is slow as hell. It appears that once they port all Rails libraries to Rust, it may not take that much effort to create Rust on Rails and get rid of Ruby.

I find a great answer in this area is Crystal. I consider it 'go in ruby clothing'. Like go, Crystal is a typed language and outputs binary executables. They kept the beautiful ruby syntax and made better internals. http://crystal-lang.org/

Re: Helix: Rust and Ruby, Without the Glue

#37

Post by kibwen »

twelvechairs wrote: post #29
aidenn0 wrote: post #18

Earlier quoted context omitted.

In Rust a string is a sequence of unicode scalar values. I personally find it unfortunate that they dictate the storage of it at the API level, but that is a necessary evil for presenting a consistent ABI with foreign code. I did not know that strings in Ruby have encodings. Is there a reason for that? I personally don't like mixing characters and opaque byte sequences as they are very different.

Its a better way of doing things - you can handle things in their native format rather than have to arbitrarily convert to UTF8 (which is an 'encoding' itself). [edit] I remember a talk where Matz was asked this specific question and tried to explain it clearly but seemed confused as to how the questioner could have such a poor grasp of unicode (the difference between monolingual americans and japanese i guess)

String is just a typedef for Vec with some extra convenience functions for working with UTF-8. There's nothing stopping anyone from just using Vec to handle non-UTF-8 data in their native format, nor stopping anyone from writing convenience types like String for other encodings.

Re: Helix: Rust and Ruby, Without the Glue

#38

Post by technion »

pzh wrote: post #19

Hmm. Kind of makes you wonder why you're using Ruby at all, which is slow as hell. It appears that once they port all Rails libraries to Rust, it may not take that much effort to create Rust on Rails and get rid of Ruby.

Plenty of gems use native C to provide best possible performance. This doesn't mean everyone is planning to rewrite their Ruby applications in C.

Rust provides a safer option - which is worthwhile. There have been several examples of gems with memory issues. It doesn't change anything, unless this is a general "Ruby is too slow" sort of rant.

Re: Helix: Rust and Ruby, Without the Glue

#39

Post by chancancode »

MichaelGG wrote: post #22
wycats wrote: post #17

Earlier quoted context omitted.

The Rust version does a type coercion from Ruby VALUE to Rust String. The type coercions are defined generically using Rust traits (see the Helix README) so once somebody defines RubyString -> String once everyone benefits. In this case, the coercion needs to ask Ruby for the encoding tag and ask Ruby to validate the encoding (which is does often enough that it's often cached) but after that we can safely coerce dire…

I was just pointing out that the C and Rust versions provided weren't quite equivalent.

You are definitely correct, this is definitely a bug.

Helix is setup to do the right thing – it already goes through a coercion protocol, we can easily add the encoding check there. We just missed that detail when porting the code, will fix it soon.

I suppose that echoes my point about how system programming in is hard to get right, there are just too many details you have to remember!

This is why having a shared solution like Helix is beneficial. By moving all the unsafe code into a common library, it's more likely that someone will notice the problem and fix it for everyone.

This actually touches on an interesting point I would like to elaborate on. When we say {Helix/Rust/Ruby} is safe, there is an important caveat – {Helix/Rust/Ruby} themselves could of course have bugs. I have definitely experienced segfaults on Ruby myself.

While true, this caveat is not particularly interesting. It is not a slight of hand. Moving code around doesn't magically remove human errors, that's not the point. It's about establishing clear boundaries for responsibility. (This is why unsafe blocks in Rust is great.)

When you get a segfault on Ruby, you know for certain that your code is not the problem. Sure, you might be something weird, but it is part of the contract that the VM is not supposed to crash no matter what you do. As a result, memory safety is just not a thing you have to constantly worry about when programming in Ruby.

It is the same thing as saying JavaScript code on a website "cannot" crash the browser, segfaults in user-space code "cannot" cause a kernel panic or malicious code "cannot" fry your chip. All of these could of course (and do) happen – but from the programmer's perspective, you can work with the assumption that they are not going to happen (and when they do, it's someone else's fault). It's not "cannot" in the "mathematically proven" sense, but it's just a useful abstraction boundary.

Re: Helix: Rust and Ruby, Without the Glue

#40

Post by placeybordeaux »

hobs wrote: post #24
bjz_ wrote: post #11

Earlier quoted context omitted.

For something that is in memory and throw-away, why go through a DB?

I was taking the point of "Why is the problem taking so long when it seems like a simple thing to calculate?" which was a little orthonogal from theme of the total post, so I can see why there would be some confusion. The article talked about 30 minutes to run through a "is this set of items in this other set of a bunch of items". While it would probably make sense to do this efficiently in memory the reality is the…

Why hit the network/a central resource when you could just do it locally?
Post reply on HN