Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Writing a JPEG Decoder in Rust – Part 2: Implementation I

mht.technology

Post reply on HN
Print view View original

61–70 of 86 posts

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#61

Post by cbreeden »

pcwalton wrote: post #23
steveklabnik wrote: post #21

Earlier quoted context omitted.

> (except the obvious `:Vec ` type that is not required) It is required, otherwise, `collect()` doesn't know what type of collection to collect into. > It may be possible to implement `.zipped` on tuples though. This is impossible without varargs, no?

> This is impossible without varargs, no? Nah, you'd just implement it as an impl over and over on tuples of reasonable size (up to 16 or so). It wouldn't be elegant, but it'd work in practice.

Which is what itertools does I believe https://bluss.github.io/rust-itertools/doc/itertools/struct....

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#62

Post by vertex-four »

wahern wrote: post #60
vertex-four wrote: post #43

Earlier quoted context omitted.

The vast majority of programs that you run on a day to day basis will abort when they run out of memory. On a default Linux system, there's not even any way to prevent that - you need to faff with the config to make it actually return errors when allocating memory.

That's all irrelevant for a language which is supposed to be a systems programming language, however that term is defined.

I don't think it is - less, bash, gcc, systemd, most interpreters, and Firefox will all pretty much just crash if malloc() fails - they're varying levels of things you'd want a "systems language" for. Unless you're trying to define systems language to mean one used for kernel or bare-metal embedded development (both cases where even in C, you skip the standard library and write your own memory allocation functions etc), there's really very few cases where you (a) are allowed to allocate memory and (b) need to fail gracefully somehow if you can't.

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#63

Post by elktea »

steveklabnik wrote: post #36
grayrest wrote: post #34

Earlier quoted context omitted.

Since Rust people are reading this. I find myself wanting to walk an iterator pairwise regularly for stuff like this and I'm using: for (cur, next) in (& vec).iter().zip(vec.iter().skip(1)) { encoded_data.push(cur); if cur == 0xff && next == 0x00 { // ... } } Is there a better way to write this? What I'd really like is the equivalent to Clojure's partition[1]. I know that the stdlib has windows `(partition n (dec n)…

I'm not 100% sure, but https://crates.io/crates/itertools might have what you're looking for.

I think http://bluss.github.io/rust-itertools/doc/itertools/struct.Z... would fit their use case?

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#64

Post by __s »

steveklabnik wrote: post #24
pcwalton wrote: post #23

Earlier quoted context omitted.

> This is impossible without varargs, no? Nah, you'd just implement it as an impl over and over on tuples of reasonable size (up to 16 or so). It wouldn't be elegant, but it'd work in practice.

Oh right, I see now.

This technique is used a lot by specs: https://github.com/slide-rs/specs/blob/master/src/join.rs#L5...

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#65

Post by steveklabnik »

wahern wrote: post #59
Manishearth wrote: post #19

Earlier quoted context omitted.

Rust has provided ways to not abort on oom for a while now.

If by "a while" you mean several months. But, yes, the recent addition of catch_unwind answered one of my biggest gripes with Rust. https://doc.rust-lang.org/std/panic/fn.catch_unwind.html Does anybody know if the standard library allocator aborts the process or throws a catchable exception? When they added catch_unwind they officially split panics into two types: abort panics and unwind panics. Code can choose one o…

To be clear, _every_ panic has to have the same implementation; you can't choose unwind vs abort for some panics vs others.

The default allocator crate always does an abort, both before and after this change.

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#66

Post by Tyr42 »

gmfawcett wrote: post #50
cfallin wrote: post #48

Earlier quoted context omitted.

Not at all -- there's a lot of commonality in the language designs. Core language features: both have algebraic datatypes like ML ("case classes" in Scala, enums in Rust), and a 'match' statement that makes working with them easy. Both have pervasive destructuring that works with match arms, 'let' bindings, etc. Data structures and mutation: both encourage a pragmatic version of immutability -- use values and provide…

I'm aware that Rust's bootstrap compiler was written in Ocaml. And yes, there are lots of ML inspirations in the language -- but not enough to make an ML language. Where are the module functors, for example? Many would consider that an essential element of an ML language, and it is a stretch to call a language an ML family member without them. I don't want this to devolve into a No True Scotsman debate, I just feel t…

I mean, Haskell doesn't have module functors, and I'd certainly claim Haskell is in the ML family. If Haskell isn't, what would be?

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#67

Post by wahern »

vertex-four wrote: post #62
wahern wrote: post #60

Earlier quoted context omitted.

That's all irrelevant for a language which is supposed to be a systems programming language, however that term is defined.

I don't think it is - less, bash, gcc, systemd, most interpreters, and Firefox will all pretty much just crash if malloc() fails - they're varying levels of things you'd want a "systems language" for. Unless you're trying to define systems language to mean one used for kernel or bare-metal embedded development (both cases where even in C, you skip the standard library and write your own memory allocation functions et…

I'm not saying that aborting on OOM never makes sense. I'm saying there are many situations where you shouldn't abort on OOM, and those situations overlap considerably with the situations in which a systems language is ideal[1]. So a systems language that doesn't allow for robust and fine-grained OOM handing isn't much of a systems language.

Example: Basically any highly concurrent network daemon that multiplexes many clients on the same thread. In that case, you want much more control over where to put your recovery point. Even if the process doesn't abort, if the recovery point is beyond the scope of the kernel thread (i.e. a controller thread, which was the recommended solution before catch_unwind), that can be really inconvenient, and also requires a lot of unnecessary passing of mutable state between threads, which is usually something you try to avoid.

[1] Lua has robust support for OOM recovery, which is noteworthy because there can be situations where you both want to handle OOM but where a scripting language is more preferable. Example: An image manipulation program with scriptable filters, where you don't want an operation that can't complete to take down your process or thread.

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#68

Post by [deleted] »

steveklabnik wrote: post #65
wahern wrote: post #59

Earlier quoted context omitted.

If by "a while" you mean several months. But, yes, the recent addition of catch_unwind answered one of my biggest gripes with Rust. https://doc.rust-lang.org/std/panic/fn.catch_unwind.html Does anybody know if the standard library allocator aborts the process or throws a catchable exception? When they added catch_unwind they officially split panics into two types: abort panics and unwind panics. Code can choose one o…

To be clear, _every_ panic has to have the same implementation; you can't choose unwind vs abort for some panics vs others. The default allocator crate always does an abort, both before and after this change.

[deleted]

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#69

Post by [deleted] »

steveklabnik wrote: post #65
wahern wrote: post #59

Earlier quoted context omitted.

If by "a while" you mean several months. But, yes, the recent addition of catch_unwind answered one of my biggest gripes with Rust. https://doc.rust-lang.org/std/panic/fn.catch_unwind.html Does anybody know if the standard library allocator aborts the process or throws a catchable exception? When they added catch_unwind they officially split panics into two types: abort panics and unwind panics. Code can choose one o…

To be clear, _every_ panic has to have the same implementation; you can't choose unwind vs abort for some panics vs others. The default allocator crate always does an abort, both before and after this change.

[deleted]

Re: Writing a JPEG Decoder in Rust – Part 2: Implementation I

#70

Post by wahern »

steveklabnik wrote: post #65
wahern wrote: post #59

Earlier quoted context omitted.

If by "a while" you mean several months. But, yes, the recent addition of catch_unwind answered one of my biggest gripes with Rust. https://doc.rust-lang.org/std/panic/fn.catch_unwind.html Does anybody know if the standard library allocator aborts the process or throws a catchable exception? When they added catch_unwind they officially split panics into two types: abort panics and unwind panics. Code can choose one o…

To be clear, _every_ panic has to have the same implementation; you can't choose unwind vs abort for some panics vs others. The default allocator crate always does an abort, both before and after this change.

When building an app, does changing the default allocator effect all imported libraries? And is that allocator used for boxing, when boxing requires dynamic allocation?
Post reply on HN