Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory safe ‘curl’ for a more secure internet

daniel.haxx.se

Post reply on HN
Print view View original

61–70 of 210 posts

Re: Memory safe ‘curl’ for a more secure internet

#61

Post by no_wizard »

svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

I've recently taken a different stance on this. the issue isn't number of dependencies, its the reliability of said dependencies to be the following

- Secure. Does this contain malicious code, exploits or otherwise known bugs that could be fixed but aren't? This of course is hard, and will never be perfect. There are static security scans though, espcially in a language like rust, that should be able to verify what the code does before its being published for consumption via carog. NPM is trying to do something similiar. This isn't foolproof and more sophisicated exploits always will exist, but getting low hanging and mid-level fruit should be within reach, which is a net win

- Is it quality? This is beyond just secure, but does it provide real utility value? One thing we always hear is DRY, which may mean sometimes you consume alot of dependencies, since the problem space you work in involves a lot of things, so why re-invent every single fix if something exists and you can glue together to start making impact in your problem domain? I don't think this is an issue espcially if #1 is true

So I don't know, I think its fine to have a lot of dependencies, I think the justification for those dependencies is often related to the complexity of the work involved. I'd expect a cURL replacement to have quite a few dependencies, since its complicated software with lots of edge cases, so for instance not re-inventing a HTTP parser is a great idea.

Now if only we all shared as developers this sentiment in terms of upstreaming contributions back too. The more we contribute and share with each other the more productive we can be.

Of course, sometimes package managers do terrible jobs at ensuring some sort of base quality, around security or otherwise, and thats never great. So its important to be aware of the trade offs you make when you source your dependencies.

By no means does this excuse developers from not understanding their dependency tree either. Its really the opposite.

Re: Memory safe ‘curl’ for a more secure internet

#62

Post by nindalf »

When I saw the issue (https://github.com/hyperium/hyper/issues/2265) for adding support for a C API to Hyper I was puzzled - couldn't you just use curl for that?

As a user of software, it makes me happy to know that folks are investing in making the nuts and bolts safer and more secure.

Re: Memory safe ‘curl’ for a more secure internet

#63

Post by k__ »

steveklabnik wrote: post #38
svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

https://wiki.alopex.li/LetsBeRealAboutDependencies > These complaints are valid, but my argument is that they’re also not NEW, and they’re certainly not unique to Rust ... The only thing new about it is that programmers are exposed to more of the costs of it up-front.

"It has always been that way." doesn't seem to be a helpful statement here.

Re: Memory safe ‘curl’ for a more secure internet

#64

Post by svnpenn »

steveklabnik wrote: post #38
svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

https://wiki.alopex.li/LetsBeRealAboutDependencies > These complaints are valid, but my argument is that they’re also not NEW, and they’re certainly not unique to Rust ... The only thing new about it is that programmers are exposed to more of the costs of it up-front.

Hm, this response essentially says "other languages have this problem too, so deal with it".

While thats true, it completely misses the point. The point is not that dependencies exist, or even that a package might have many dependencies.

The point is, Rust (and NPM) I have found many times dont care or even consider the impact of a large amount of dependencies, and often take no steps to mitigate or reduce that number.

As others said, some features could be split off into other crates. Maybe someone only needs HTTP, or maybe they need HTTPS but no Async. Or maybe they dont need logging. With Hyper and others you just have to build everything whether you want it or not.

Re: Memory safe ‘curl’ for a more secure internet

#65

Post by steveklabnik »

k__ wrote: post #63
steveklabnik wrote: post #38

Earlier quoted context omitted.

https://wiki.alopex.li/LetsBeRealAboutDependencies > These complaints are valid, but my argument is that they’re also not NEW, and they’re certainly not unique to Rust ... The only thing new about it is that programmers are exposed to more of the costs of it up-front.

"It has always been that way." doesn't seem to be a helpful statement here.

I mean, in some sense you're right, but in another, the point is that it's not a regression.

Re: Memory safe ‘curl’ for a more secure internet

#66

Post by steveklabnik »

svnpenn wrote: post #64
steveklabnik wrote: post #38

Earlier quoted context omitted.

https://wiki.alopex.li/LetsBeRealAboutDependencies > These complaints are valid, but my argument is that they’re also not NEW, and they’re certainly not unique to Rust ... The only thing new about it is that programmers are exposed to more of the costs of it up-front.

Hm, this response essentially says "other languages have this problem too, so deal with it". While thats true, it completely misses the point. The point is not that dependencies exist, or even that a package might have many dependencies. The point is, Rust (and NPM) I have found many times dont care or even consider the impact of a large amount of dependencies, and often take no steps to mitigate or reduce that numbe…

> As others said, some features could be split off into other crates.

I agree with this, but that's not what your original post says. Or at least, it’s not what I understood from reading it. :)

> The point is, Rust (and NPM)

and C, in many real-world cases, which is why the above post matters.

Re: Memory safe ‘curl’ for a more secure internet

#67

Post by k__ »

steveklabnik wrote: post #65
k__ wrote: post #63

Earlier quoted context omitted.

"It has always been that way." doesn't seem to be a helpful statement here.

I mean, in some sense you're right, but in another, the point is that it's not a regression.

True.

Also, I coud imagine Rust's type-system raises the bar of dependencies that can be mangaged before everything breaks down. So a lib with 100 deps in NPM isn't the same as a lib with 100 deps in Cargo.

Re: Memory safe ‘curl’ for a more secure internet

#68

Post by svnpenn »

timdorr wrote: post #41
svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

I count 14: https://github.com/hyperium/hyper/blob/master/Cargo.toml#L22... bytes, futures-core, futures-channel, futures-util, http, http-body, httpdate, httparse, h2, itoa, tracingfeatures, pin-project, tower-service, tokio, want

Here is my script:

https://github.com/nu8/cove/tree/master/rust-deps

Re: Memory safe ‘curl’ for a more secure internet

#69

Post by ComputerGuru »

afwe wrote: post #36
tialaramex wrote: post #33

Earlier quoted context omitted.

Historically there was a long period where this didn't do what you expect, which is very bad. What this looks like it does, and indeed does today (modulo bugs some of which could be prevented using Rust) is: Ask totally-not-evil.example.com for this install.sh resource and then run that as root as a Bash script. This is no worse than if you were to have totally-not-evil.example.com give you the bash script on a flopp…

The website could detect whether you are using a regular browser or curl itself to download the .sh file and return something different. So inspecting the .sh using your browser before you run that line would not protect you.

Inspect it by piping to less or vim instead?

Re: Memory safe ‘curl’ for a more secure internet

#70

Post by LockAndLol »

svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

C/C++ suffers from severe wheel reinvention due to a lack of package management, but do you see people cautioning the use of programs written in that language for that reason?

The STL package contains lots of things, but most of them are pretty ugly to use and quite a few of them are quite complicated to use because they are generalized for as many use cases as possible. There are so many authors out there that decide they can do memory management manually, write their own thread or process pools, implement sorting in a different way, rewrite basic algorithms for list operations like mapping, filtering, zipping, reducing...

Simply counting the number of dependencies isn't a great indicator for dependency bloat. There are extremes on both ends : no deps --> I know everything better and reimplemented the world, and thousands of deps --> I put a one liner in a package ma! One should not judge too quickly.

Post reply on HN