Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory safe ‘curl’ for a more secure internet

daniel.haxx.se

Post reply on HN
Print view View original

141–150 of 210 posts

Re: Memory safe ‘curl’ for a more secure internet

#141

Post by simias »

burntsushi wrote: post #111
simias wrote: post #73

Earlier quoted context omitted.

You could argue the same thing for many big monolithic C projects though. How many of the original authors/maintainers are left in OpenSSL or the Linux kernel? My main worry about Rust dependencies is not so much the number, it's that it's still a fairly young ecosystem that hasn't stabilized yet, packages come and go fairly quickly even for relatively basic features. For instance for a long time lazy_static (which i…

I switched ripgrep to clap 4 years ago. And that was well after clap had already become the popular "go to" solution. Some parts of the ecosystem are more stable than others. That's true. And it takes work to know which things are stable and which aren't. And yet, some things just take a longer time to improve. lazy_static has been stable and unchanged for a very long time and it works just fine. You don't need to sw…

Interesting that I missed clap when I wrote that program a few years ago then. In my defence "argparse" is a lot more explicit than "clap" for a such a library. Also argparse's last update was 2 years ago, so there's been quite a bit of overlap.

I guess what I'm saying is that it's an other problem with the current package ecosystem: you often end up finding multiple packages purporting to do what you need, and it can be tricky to figure out which one you want. As an example, if you want a simple logging backend the log crate currently lists 6 possibilities: https://crates.io/crates/log

I picked "simple_logger" basically at random.

Re: Memory safe ‘curl’ for a more secure internet

#142

Post by CUViper »

nindalf wrote: post #126
nicoburns wrote: post #50

Earlier quoted context omitted.

Part of this is just crates being broken up more in Rust. For example the `http` crate only contains trait (interface) definitions. They break down like so: Platform integration: libc, winapi, winapi-build, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, ws2_32-sys, fuchsia-zircon, fuchsia-zircon-sys, kernel32-sys, redox_syscall Primitive algorithms: itoa, memchr, unicode-xid Proc macro / pinning utilities:…

Why does hyper pull in hashbrown? Isn't it identical to std::collections::HashMap?

The dependency goes hyper -> h2 -> indexmap -> hashbrown, and indexmap is built on hashbrown::raw::RawTable.

Re: Memory safe ‘curl’ for a more secure internet

#143

Post by steveklabnik »

cle wrote: post #135
steveklabnik wrote: post #91

Earlier quoted context omitted.

> with a standard that is rarely matched by third party libraries I mean, you can go both ways with this. Standard libraries are significantly more difficult to work on than third party libraries, and I've seen a lot of code in standard libraries that is objectively worse than ecosystem equivalents because of it.

I agree, there's no silver bullet, only tradeoffs and our appetites for them. From looking at the list of crates above, I see a lot that are often part of the stdlib of other languages, such as HTTP, concurrency/nio primitives, and logging. My perception of Rust's not including such fundamental primitives in the standard library is that Rust is still very much experimental, and the ecosystem values tinkering and expe…

HTTP libraries are a prime example of where many, many standard libraries are considered old and crufty, and there are much better ecosystem libraries that end up being wildly used more.

You may have that perception, and that is fine, but it's not likely to be a thing that changes significantly, even when Rust is quite old. There's just not a lot of advantage to being in the standard library, and numerous downsides.

Re: Memory safe ‘curl’ for a more secure internet

#144

Post by steveklabnik »

ncmncm wrote: post #136

Switching immediately to building with C++, and then migrating incrementally to safe forms in C++, would provide much more value per unit effort. It would also enable engagement by the orders-of-magnitude more available skilled C++ programmers, who could also pick up new skills writing modern, safe C++ to apply in other migrations. It is not an either/or proposition. Certain, select modules could be recoded in Rust b…

I think you may have misunderstood what the post is saying they're going to do. It is significantly more in line with your suggestion than you seem to think.

Re: Memory safe ‘curl’ for a more secure internet

#145

Post by jandrese »

svnpenn wrote: post #64
steveklabnik wrote: post #38

Earlier quoted context omitted.

https://wiki.alopex.li/LetsBeRealAboutDependencies > These complaints are valid, but my argument is that they’re also not NEW, and they’re certainly not unique to Rust ... The only thing new about it is that programmers are exposed to more of the costs of it up-front.

Hm, this response essentially says "other languages have this problem too, so deal with it". While thats true, it completely misses the point. The point is not that dependencies exist, or even that a package might have many dependencies. The point is, Rust (and NPM) I have found many times dont care or even consider the impact of a large amount of dependencies, and often take no steps to mitigate or reduce that numbe…

> other languages have this problem too, so deal with it

This is the primary reason I try to avoid projects built with npm. Fucking dependency hell. If the project hasn't been actively maintained in the last 3 months your chances of getting it to work drop precipitously.

Re: Memory safe ‘curl’ for a more secure internet

#146

Post by jandrese »

steveklabnik wrote: post #66
svnpenn wrote: post #64

Earlier quoted context omitted.

Hm, this response essentially says "other languages have this problem too, so deal with it". While thats true, it completely misses the point. The point is not that dependencies exist, or even that a package might have many dependencies. The point is, Rust (and NPM) I have found many times dont care or even consider the impact of a large amount of dependencies, and often take no steps to mitigate or reduce that numbe…

> As others said, some features could be split off into other crates. I agree with this, but that's not what your original post says. Or at least, it’s not what I understood from reading it. :) > The point is, Rust (and NPM) and C , in many real-world cases, which is why the above post matters.

When I find a project that is a handful of .c files and a Makefile they almost always compile and run. Sometimes with warnings because the features used in the code are depreciated but usually without too much fanfare.

Re: Memory safe ‘curl’ for a more secure internet

#147

Post by monsieurbanana »

simias wrote: post #141
burntsushi wrote: post #111

Earlier quoted context omitted.

I switched ripgrep to clap 4 years ago. And that was well after clap had already become the popular "go to" solution. Some parts of the ecosystem are more stable than others. That's true. And it takes work to know which things are stable and which aren't. And yet, some things just take a longer time to improve. lazy_static has been stable and unchanged for a very long time and it works just fine. You don't need to sw…

Interesting that I missed clap when I wrote that program a few years ago then. In my defence "argparse" is a lot more explicit than "clap" for a such a library. Also argparse's last update was 2 years ago, so there's been quite a bit of overlap. I guess what I'm saying is that it's an other problem with the current package ecosystem: you often end up finding multiple packages purporting to do what you need, and it ca…

> it's an other problem with the current package ecosystem: you often end up finding multiple packages purporting to do what you need, and it can be tricky to figure out which one you want

I'm trying to remember the last language I've used where people didn't say that.

Hmm... clojure? Nop.

Javascript? Nop nop nop.

Python? Hahaha I can't even remember all the package managers: virtualenv, venv, pipenv, poetry, ...

Re: Memory safe ‘curl’ for a more secure internet

#148

Post by dpc_pw »

monadic2 wrote: post #81
nicoburns wrote: post #50

Earlier quoted context omitted.

Part of this is just crates being broken up more in Rust. For example the `http` crate only contains trait (interface) definitions. They break down like so: Platform integration: libc, winapi, winapi-build, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, ws2_32-sys, fuchsia-zircon, fuchsia-zircon-sys, kernel32-sys, redox_syscall Primitive algorithms: itoa, memchr, unicode-xid Proc macro / pinning utilities:…

Thinking about crates this way is a revelation to me. Is there any tooling to make analyzing dependencies as you did easier?

cargo-crev is one

Re: Memory safe ‘curl’ for a more secure internet

#149

Post by dpc_pw »

jandrese wrote: post #146
steveklabnik wrote: post #66

Earlier quoted context omitted.

> As others said, some features could be split off into other crates. I agree with this, but that's not what your original post says. Or at least, it’s not what I understood from reading it. :) > The point is, Rust (and NPM) and C , in many real-world cases, which is why the above post matters.

When I find a project that is a handful of .c files and a Makefile they almost always compile and run. Sometimes with warnings because the features used in the code are depreciated but usually without too much fanfare.

> they almost always compile and run

Same in Rust. Actually, it's quite better than in C. The only time Rust projects fail to compile is when they pull in some C library and something there (like configure.ac) messed up. :D

And if this C project does anything interesting it pulls in bunch of C libraries that came precompiled with your OS, and might be stale and contain unpatched security vulns.

C/C++ developers pointing at other languages about dependency hell is a curiosity.

Re: Memory safe ‘curl’ for a more secure internet

#150

Post by bagder »

skohan wrote: post #139
ordu wrote: post #122

Earlier quoted context omitted.

I think that it is the right tool. 1. CURL without https seems insufficient nowadays. 2. CURL could be improved by running multiple downloads at once. I'm not sure that curl command line utility could do it, but certainly libcurl.so has this ability, it allows client code to work with multiple connections. 3. Any application having UI could benefit from async: input/output and main task are async by nature. For examp…

1. So as to your first point, I totally agree CURL needs to support HTTPS. My point is that Hyper needs a runtime for HTTPS, and it doesn't necessarily make sense for CURL to have a runtime. 2. I'm not sure that CURL should necessarily support multiple concurrent downloads. It could also be argued it's more UNIX-y to make it just do one thing and allow the caller to run multiple CURL processes at the same time 3. You…

Tadaa: curl already supports --parallel to download many URLs simultaneously...
Post reply on HN