Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a JPEG Decoder in Rust – Part 2: Implementation I

mht.technology

Post reply on HN
Print view View original

81–86 of 86 posts

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#81

Post by Veedrac »

eddyb wrote: post #57
pcwalton wrote: post #22

Earlier quoted context omitted.

The biggest difference between Scala and Rust here is ".iter()", which is necessary mainly because you need to specify whether the iterator iterates over references (".iter()") or passes results by value (".into_iter()"). Because Java doesn't have value types and has no concept of move semantics, it has no need for the distinction, while Rust does. In effect, the "elegance" of Scala here is really the product of its…

The example can be written like this though: let codes: Vec = data_table.iter() .zip(&code_lengths) .zip(&code_table) .map(|((&value, &length), &code)| { HuffmanCode { length: length, code: code, value: value, } }) .collect();

In fact `code_lengths` and `code_table` are never used again so should just be passed by value, which saves four more `&`s.

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#82

Post by Veedrac »

nercury wrote: post #18
lmm wrote: post #16

Earlier quoted context omitted.

It's not, but they're both ML-family languages. I had hoped that Rust would be able to offer a similar level of elegance to Scala.

Looking at the example, I can't find a thing in there that is not required (except the obvious `:Vec ` type that is not required). We need `.iter` to tell that iterator is imutable (there is mutable version), we need `.collect` to actually run the iteration. I also don't think objects should have default constructor functions. It may be possible to implement `.zipped` on tuples though.

Actually the last two `.iter()` aren't needed at all because of IntoIter. All three can all be removed if you use Itertools' izip!

    let codes: Vec =
        izip!(&data_table, code_lengths, code_table)
            .map(|(&value, length, code)| {
                HuffmanCode {
                    length: length,
                    code: code,
                    value: value,
                }
            })
            .collect();
(If HuffmanCode was a tuple type, this could even be

    #![feature(fn_traits)]

    let codes: Vec =
        izip!(data_table.iter().cloned(), code_lengths, code_table)
            .map(|args| (&HuffmanCode).call(args))
            .collect();
but now I'm just playing around.)

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#83

Post by Veedrac »

epidemian wrote: post #25
nercury wrote: post #18

Earlier quoted context omitted.

Looking at the example, I can't find a thing in there that is not required (except the obvious `:Vec ` type that is not required). We need `.iter` to tell that iterator is imutable (there is mutable version), we need `.collect` to actually run the iteration. I also don't think objects should have default constructor functions. It may be possible to implement `.zipped` on tuples though.

Yep, but it's still an unfortunately big difference in expressiveness between the two languages, where one might think that they would be more or less on par, given the languages' similarities (both are more or less contemporary in their design, statically typed, with similarly powerful type systems, etc). My question is then: is there something fundamentally preventing Rust from achieving similar levels of expressiv…

Actually Rust already has izip and IntoIterator that covers the early differences. The type annotation `Vec` is also optional since it can be inferred from the context.

The `map` is uglier in Rust because

* the comparison was against a constructor with positional, rather than named, arguments

* the Scala code didn't need to dereference any arguments, and

* and Scala's `Zipped` is a special type with a special `map` function that takes three arguments, unlike a normal iterator.

The first and last points could be easily copied in Rust: you'd build a constructor for HuffmanCode and augment iterators of tuples with with a starmap method (that can be done in a library). The middle point can be done before the zipping. The result would be

    let codes =
        izip!(data_table.iter().cloned(), code_lengths, code_table)
            .starmap(HuffmanCode::new)
            .collect();
Rust's collect is never implicit, like Scala's CanBuildFrom. This prevents accidental collects, which helps writing fast code, but in principle I don't see why it couldn't be implicit - it would just require the whole standard library to be overhauled.

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#84

Post by pcwalton »

wahern wrote: post #76
nnethercote wrote: post #73

Earlier quoted context omitted.

> Firefox will all pretty much just crash if malloc() fails It's not nearly as simple as that. A general approach we try to use is the following. - Small allocations are infallible (i.e. abort on failure) because if a small allocation fails you're probably in a bad situation w.r.t. memory. - Large allocations, especially those controlled by web content, are fallible. The distinction between small and large isn't clea…

That approach may make sense on the desktop, but it doesn't make sense for network servers. On a network server allocations tend to be small. Your 10,001st request might fail not because of a huge allocation but because it just pushed you over the line. Aborting all 10,000 requests provides really poor QoS. You can get away with that if you're Google, because 10,000 requests is still miniscule relative to your total…

So, to be clear, you're implying that you shouldn't write network services in Python, Ruby, PHP, Go, Java in practice [1], Perl, C++ without exceptions, JavaScript, etc. etc.?

[1]: http://stackoverflow.com/questions/1692230/is-it-possible-to...

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#85

Post by jakub_h »

pslam wrote: post #53

If you scan through the source (on github), the most notable thing is the lack of the "unsafe" keyword. I've seen too many people basically transliterate from C to Rust, along with all the unsafe operations. This one is pretty much unoptimized, but still seems to be performant enough, and doesn't do anything "unsafe". That's not to say you could just throw this into the back-end of a public-facing website. It's still…

Isn't it the case that at least some unsafe features in C actually prevent compiler optimizations? Being safe should not necessarily mean being slower. Sometimes restrictions are enablers for better compilers.

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#86

Post by EugeneOZ »

technion wrote: post #74
pslam wrote: post #53

If you scan through the source (on github), the most notable thing is the lack of the "unsafe" keyword. I've seen too many people basically transliterate from C to Rust, along with all the unsafe operations. This one is pretty much unoptimized, but still seems to be performant enough, and doesn't do anything "unsafe". That's not to say you could just throw this into the back-end of a public-facing website. It's still…

Given the history of vulnerabilities in tools like ImageMagick, that panic might still be a better position than an exploitable memory bug.

Given the fact you can avoid panic and exit safely, all panicking is just laziness ;)
Post reply on HN