Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory safe ‘curl’ for a more secure internet

daniel.haxx.se

Post reply on HN
Print view View original

131–140 of 210 posts

Re: Memory safe ‘curl’ for a more secure internet

#131

Post by benecollyridam »

steveklabnik wrote: post #59
benecollyridam wrote: post #56

> Hyper is a fast and safe HTTP implementation Well.. Hyper does rely on unsafe blocks (14 at first glance[2]), so I don't know if we can just assume that it's safe. When Sergey Davidoff did their big smoke test of popular Rust HTTP implementations they found a couple of bugs[1] (through Reqwest). I love the idea of a safer cURL, but I don't think you should take this as a magical answer to all of cURL's problems. [1…

> I don't think you should take this as a magical answer to all of cURL's problems. Is anyone actually suggesting this?

Well kinda but people are saying that it is memory safe, which is not a guarantee with unsafe code.

Re: Memory safe ‘curl’ for a more secure internet

#133

Post by dralley »

kej wrote: post #132

I was under the impression that curl worked on more platforms than Rust and LLVM. It will be interesting to see what happens to curl support on those platforms going forward.

Libcurl supports multiple compile-time backends for http support, encryption, and so forth. This will be no different. Hyper will be just one option among many, as will Rustls.

Re: Memory safe ‘curl’ for a more secure internet

#134

Post by masklinn »

kej wrote: post #132

I was under the impression that curl worked on more platforms than Rust and LLVM. It will be interesting to see what happens to curl support on those platforms going forward.

As the article indicates, it would be but one of dozens of existing backends, although it'd be one where few alternative backends currently exist (HTTP/1 and HTTP/2).

For instance libcurl can use any of 13 different TLS backends (one of which is already in Rust), or 3 different HTTP/3 backend (one of which is in rust).

Re: Memory safe ‘curl’ for a more secure internet

#135

Post by cle »

steveklabnik wrote: post #91
cle wrote: post #83

Earlier quoted context omitted.

There’s a third option if the package is so important: put it in the standard library. Technically it’s still a dependency but the standard library is maintained with a standard that is rarely matched by third party libraries, and can dramatically simplify the ecosystems’ dependency graph.

> with a standard that is rarely matched by third party libraries I mean, you can go both ways with this. Standard libraries are significantly more difficult to work on than third party libraries, and I've seen a lot of code in standard libraries that is objectively worse than ecosystem equivalents because of it.

I agree, there's no silver bullet, only tradeoffs and our appetites for them.

From looking at the list of crates above, I see a lot that are often part of the stdlib of other languages, such as HTTP, concurrency/nio primitives, and logging.

My perception of Rust's not including such fundamental primitives in the standard library is that Rust is still very much experimental, and the ecosystem values tinkering and experimenting with new ideas. The cost of that is increased dependency hell, especially over time. There's obviously huge value in experimentation for the industry, but it makes me hesitate to use Rust for projects where I just need to get stuff done, and stay done.

Re: Memory safe ‘curl’ for a more secure internet

#136

Post by ncmncm »

Switching immediately to building with C++, and then migrating incrementally to safe forms in C++, would provide much more value per unit effort. It would also enable engagement by the orders-of-magnitude more available skilled C++ programmers, who could also pick up new skills writing modern, safe C++ to apply in other migrations.

It is not an either/or proposition. Certain, select modules could be recoded in Rust by particularly motivated Rust coders, leaving the huge amount of other code, for which there are too few Rust enthusiasts to work on, to be modernized in C++, and still able to call into the Rust code.

Re: Memory safe ‘curl’ for a more secure internet

#137

Post by bluGill »

qchris wrote: post #84
count wrote: post #49

Earlier quoted context omitted.

This is true today, will it remain true?

It's also worth pointing out that once a version is published to crates.io, it can't be altered, specifically to prevent social engineering attacks. If you're worried about it, that means you can audit the frozen codebase for any given version from a top-level crate down through the dependencies, and once that trust is established, it can't be leveraged for a silent dependency change later on, which can only happen t…

What if I audit something down a few levels and find it lacking - how do I force update everything to not use the bad version?

Re: Memory safe ‘curl’ for a more secure internet

#138

Post by geodel »

ori_b wrote: post #85
steveklabnik wrote: post #38

Earlier quoted context omitted.

https://wiki.alopex.li/LetsBeRealAboutDependencies > These complaints are valid, but my argument is that they’re also not NEW, and they’re certainly not unique to Rust ... The only thing new about it is that programmers are exposed to more of the costs of it up-front.

Saying "The people complaining are right, but they've been right a long time" isn't a great endorsement of the situation.

It seems to me Rust folks have developed this habit of deflecting blame by pointing to shallow commentary. One of good examples is compiler slowness reasons, sometimes its LLVM, or it is lot of optimizations, or it is not really slow compare to C++ and so on.

They could have said it straight "Guys highly optimized, safe compilation of medium size project will be in range of 20-30 min". And that would great and honest way to deal. Instead we get oh, we have reduced compile times from 26 minutes to 21 minutes so it is 19% improvement in just one year and there is more to come. Now this is hard work and great but I am sure Rust committers understand when people say fast compile times they most likely comparing to Go etc which would be under a minute for most mid size projects. And this is very well not going to happen.

Same is now to Cargo dependency situation. Cargo and NPM are ideologically in agreement that people in general should just pull it from package manager instead of rewriting even little bit of code. And again instead of owning it there will be list of shallow reasoning: others do it too, reusing is better that rewriting, cargo is so awesome that pulling crate is much easier and so on.

Re: Memory safe ‘curl’ for a more secure internet

#139

Post by skohan »

ordu wrote: post #122
skohan wrote: post #101

Earlier quoted context omitted.

I also wonder if Hyper is really the right tool for the job here. When I was looking into HTTP/S crates, I decided against Hyper because it seemed to require bringing in a runtime for HTTPS, and the async nature of hyper did not seem necessary for my totally synchronous CLI tool. For something like CURL it seems like you would just want the leanest, simplest synchronous HTTP implementation possible

I think that it is the right tool. 1. CURL without https seems insufficient nowadays. 2. CURL could be improved by running multiple downloads at once. I'm not sure that curl command line utility could do it, but certainly libcurl.so has this ability, it allows client code to work with multiple connections. 3. Any application having UI could benefit from async: input/output and main task are async by nature. For examp…

1. So as to your first point, I totally agree CURL needs to support HTTPS. My point is that Hyper needs a runtime for HTTPS, and it doesn't necessarily make sense for CURL to have a runtime.

2. I'm not sure that CURL should necessarily support multiple concurrent downloads. It could also be argued it's more UNIX-y to make it just do one thing and allow the caller to run multiple CURL processes at the same time

3. You wouldn't necessarily need async to achieve these goals. You could easily have a synchronous http implementation which allows for printing to the console between receiving chunks of data from the network. And if you really didn't want blocking to have a "spinning" activity indicator or something, you could still achieve it with threads.

I think ultimately you'd have to decide based on the relative cost of including an entire runtime vs. just launching a second thread.

Re: Memory safe ‘curl’ for a more secure internet

#140

Post by OJFord »

bluGill wrote: post #137
qchris wrote: post #84

Earlier quoted context omitted.

It's also worth pointing out that once a version is published to crates.io, it can't be altered, specifically to prevent social engineering attacks. If you're worried about it, that means you can audit the frozen codebase for any given version from a top-level crate down through the dependencies, and once that trust is established, it can't be leveraged for a silent dependency change later on, which can only happen t…

What if I audit something down a few levels and find it lacking - how do I force update everything to not use the bad version?

https://doc.rust-lang.org/edition-guide/rust-2018/cargo-and-...
Post reply on HN