Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Helix: Rust and Ruby, Without the Glue

blog.skylight.io

Post reply on HN
Print view View original

21–30 of 69 posts

Re: Helix: Rust and Ruby, Without the Glue

#21

Post by MichaelGG »

aidenn0 wrote: post #18
x5n1 wrote: post #13

Earlier quoted context omitted.

huh? strings have encodings. rust strings are bytes encoded in utf-8. https://doc.rust-lang.org/book/strings.html

In Rust a string is a sequence of unicode scalar values. I personally find it unfortunate that they dictate the storage of it at the API level, but that is a necessary evil for presenting a consistent ABI with foreign code. I did not know that strings in Ruby have encodings. Is there a reason for that? I personally don't like mixing characters and opaque byte sequences as they are very different.

Ruby's Japanese heritage is probably why it handles encodings like that - I think there were multiple encs it had to deal with at once or something. Also Unicode doesn't completely handle all kanji in that there's some that have an old style not available in Unicode. But maybe that's not relevant.

Re: Helix: Rust and Ruby, Without the Glue

#22

Post by MichaelGG »

wycats wrote: post #17
MichaelGG wrote: post #4

But the Rust version of the blank check doesn't handle any encoding but UTF-8. Helix could wrap up a char iterator for it I suppose, one that calls rb_enc_codepoint_len? And isn't there some common C lib that exposes Unicode functions like is_whitespace? Granted, using a cargo crate is easier than finding and adding a .h, and far easier than getting and linking another lib.

The Rust version does a type coercion from Ruby VALUE to Rust String. The type coercions are defined generically using Rust traits (see the Helix README) so once somebody defines RubyString -> String once everyone benefits. In this case, the coercion needs to ask Ruby for the encoding tag and ask Ruby to validate the encoding (which is does often enough that it's often cached) but after that we can safely coerce dire…

I was just pointing out that the C and Rust versions provided weren't quite equivalent.

Re: Helix: Rust and Ruby, Without the Glue

#23

Post by rattray »

pzh wrote: post #19

Hmm. Kind of makes you wonder why you're using Ruby at all, which is slow as hell. It appears that once they port all Rails libraries to Rust, it may not take that much effort to create Rust on Rails and get rid of Ruby.

Ruby is phenomenally easy to write. This post essentially describes a way to make Ruby less slow, which would reduce the incentive to leave the language.

Once they port all Rails libraries to Rust, you can still use the extremely-ergonomic Ruby programming language, but it won't be "slow as hell" anymore. And ruby performance is "tolerable in most circumstances" as-is.

I certainly agree that it could be awesome to have Rust on Rails for those who would find even Rails on Rust performance intolerable, and Iron's abstractions inadequate.

Re: Helix: Rust and Ruby, Without the Glue

#24

Post by hobs »

bjz_ wrote: post #11
hobs wrote: post #10

Isnt a set containment problem a really simple problem to solve with a database, eg an inner join and a count?

For something that is in memory and throw-away, why go through a DB?

I was taking the point of "Why is the problem taking so long when it seems like a simple thing to calculate?" which was a little orthonogal from theme of the total post, so I can see why there would be some confusion.

The article talked about 30 minutes to run through a "is this set of items in this other set of a bunch of items".

While it would probably make sense to do this efficiently in memory the reality is the company mentioned are probably doing it inefficiently in memory; asking some tool which can do this type of work effectively and is fairly well known was my basic suggestion.

Re: Helix: Rust and Ruby, Without the Glue

#25

Post by rattray »

> With Rails reaching version 5.0, there are plenty of APIs that are heavily used and extremely feature-stable. My goal with Helix is to eventually make it possible to reimplement a lot of these APIs as an native extension gem.

Sounds extremely exciting!

Re: Helix: Rust and Ruby, Without the Glue

#26

Post by danso »

pzh wrote: post #19

Hmm. Kind of makes you wonder why you're using Ruby at all, which is slow as hell. It appears that once they port all Rails libraries to Rust, it may not take that much effort to create Rust on Rails and get rid of Ruby.

How many platforms and frameworks have been completely ported over to a better language? There are many languages that have arguably succeeded PHP, and yet WordPress and Facebook remain on PHP, which would belie your assumption that "it may not take that much effort to [re]create [some framework] and get rid of [that framework's original language]"

Re: Helix: Rust and Ruby, Without the Glue

#27

Post by aidenn0 »

MichaelGG wrote: post #21
aidenn0 wrote: post #18

Earlier quoted context omitted.

In Rust a string is a sequence of unicode scalar values. I personally find it unfortunate that they dictate the storage of it at the API level, but that is a necessary evil for presenting a consistent ABI with foreign code. I did not know that strings in Ruby have encodings. Is there a reason for that? I personally don't like mixing characters and opaque byte sequences as they are very different.

Ruby's Japanese heritage is probably why it handles encodings like that - I think there were multiple encs it had to deal with at once or something. Also Unicode doesn't completely handle all kanji in that there's some that have an old style not available in Unicode. But maybe that's not relevant.

Unicode now handles all the Kanji in JIS. I wouldn't be surprised if Ruby predated that. It almost certainly predates good library support for all the Kanji in JIS.

Re: Helix: Rust and Ruby, Without the Glue

#28

Post by burntsushi »

aidenn0 wrote: post #18
x5n1 wrote: post #13

Earlier quoted context omitted.

huh? strings have encodings. rust strings are bytes encoded in utf-8. https://doc.rust-lang.org/book/strings.html

In Rust a string is a sequence of unicode scalar values. I personally find it unfortunate that they dictate the storage of it at the API level, but that is a necessary evil for presenting a consistent ABI with foreign code. I did not know that strings in Ruby have encodings. Is there a reason for that? I personally don't like mixing characters and opaque byte sequences as they are very different.

> In Rust a string is a sequence of unicode scalar values.

The representation of a Rust String in memory is guaranteed valid UTF-8. To me, a "sequence of Unicode scalar values" is an abstract description, because it could be implemented via UTF-8, UTF-16 or UTF-32.

> I personally find it unfortunate that they dictate the storage of it at the API level

It is extraordinarily convenient and provides a very transparent way to analyze the performance of string operations.

For transcoding, there is the in-progress `encoding` crate: https://github.com/lifthrasiir/rust-encoding

I note that Go does things very similarly (`string` is conventionally UTF-8) and it works famously for them. They have a much more mature set of encoding libraries, but they work the same as the equivalent libraries would work in Rust: transcode to and from UTF-8 at the boundaries. See: https://godoc.org/golang.org/x/text

Re: Helix: Rust and Ruby, Without the Glue

#29

Post by twelvechairs »

aidenn0 wrote: post #18
x5n1 wrote: post #13

Earlier quoted context omitted.

huh? strings have encodings. rust strings are bytes encoded in utf-8. https://doc.rust-lang.org/book/strings.html

In Rust a string is a sequence of unicode scalar values. I personally find it unfortunate that they dictate the storage of it at the API level, but that is a necessary evil for presenting a consistent ABI with foreign code. I did not know that strings in Ruby have encodings. Is there a reason for that? I personally don't like mixing characters and opaque byte sequences as they are very different.

Its a better way of doing things - you can handle things in their native format rather than have to arbitrarily convert to UTF8 (which is an 'encoding' itself).

[edit] I remember a talk where Matz was asked this specific question and tried to explain it clearly but seemed confused as to how the questioner could have such a poor grasp of unicode (the difference between monolingual americans and japanese i guess)

Re: Helix: Rust and Ruby, Without the Glue

#30

Post by steveklabnik »

hetman wrote: post #14
aidenn0 wrote: post #8

Earlier quoted context omitted.

The Rust version works on strings not on bytes. Strings don't have encodings.

What do you mean? All Rust strings are UTF-8 encoded, and all Ruby strings have an associated encoding.

All rust String and &strs are UTF-8 encoded, there are also other string types.
Post reply on HN