Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Memory safe ‘curl’ for a more secure internet

daniel.haxx.se

Post reply on HN
Print view View original

121–130 of 210 posts

Re: Memory safe ‘curl’ for a more secure internet

#121

Post by steveklabnik »

edflsafoiewq wrote: post #120
steveklabnik wrote: post #58

Earlier quoted context omitted.

> The overall result is that everyone pays the cognitive cost of exception (spelled "unwind" in Rust) safety Very, very few people have to think about unwind safety, because it really only comes into play when you're writing unsafe code, and relying on the ability for panics to be caught. Many folks aren't writing any unsafe, and many who are are doing it explicitly in a panic=abort environment. And most don't rely o…

> It's one character. No it isn't. With exceptions: fn foo() -> i32 { let x = bar(); x + 1 } vs fn foo() -> Result { let x = bar()?; Ok(x + 1) }

This is fair, thank you for the correction.

I do think it's not quite what you're saying though, due to Rust's design, we would have checked exceptions, and so the Result part would be there with exceptions, it's the Ok() part that would change.

In theory you could make a language without checked exceptions that would make it be like your example.

Re: Memory safe ‘curl’ for a more secure internet

#122

Post by ordu »

skohan wrote: post #101
svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

I also wonder if Hyper is really the right tool for the job here. When I was looking into HTTP/S crates, I decided against Hyper because it seemed to require bringing in a runtime for HTTPS, and the async nature of hyper did not seem necessary for my totally synchronous CLI tool. For something like CURL it seems like you would just want the leanest, simplest synchronous HTTP implementation possible

I think that it is the right tool.

1. CURL without https seems insufficient nowadays. 2. CURL could be improved by running multiple downloads at once. I'm not sure that curl command line utility could do it, but certainly libcurl.so has this ability, it allows client code to work with multiple connections. 3. Any application having UI could benefit from async: input/output and main task are async by nature. For example, curl might want to show progress/status to a terminal despite of a stalled connection.

So maybe Hyper is too much of a code for a curl, but it is arguable that it is not.

Re: Memory safe ‘curl’ for a more secure internet

#123

Post by aliceryhl »

InfiniteRand wrote: post #112
simias wrote: post #76

Earlier quoted context omitted.

Most applications are not supposed to survive a panic!(), to the point where I personally tend to use "panic = 'abort'" for any code where performance matters. As such it's not really a mental overhead, and definitely not the Rust way of doing error handling (in the same way than using "assert" and a signal handler is not the way you're supposed to do error handling in C++). The Rust way of dealing with errors is to…

How does Rust handle errors coming out of library operations? Is the main application able to override the panic handling behavior of a library?

Yes, the binary crate determines the behaviour of panics. Libraries do typically not use panics for errors.

Re: Memory safe ‘curl’ for a more secure internet

#124

Post by hirako2000 »

weinzierl wrote: post #108
hirako2000 wrote: post #96

Earlier quoted context omitted.

That. And then we become responsible for yet more possible vulns in all those deps. Current number of cURL deps: 14 I wish it relied only on OpenSSL.

Not depependin on OpenSSL is kind of the point of the original post if I understand it correctly. Also I find the dependency on OpenSSL one major pain in my Rust projects. When you want to build a statically linked binary you need to supply a statically built OpenSSL and if your distro doesn't come with one (like Ubuntu) you are on your own. Yes, there is a Docker container that comes with all the prerequisites but I…

Containers is becoming the way to build stuff, partly for that reason.

I think any dependency adds a level of burden, but some things are better delegated to library. I think crypto is a good case, btw I think OpenSSL is not the only one lib for TLS with curl.

Re: Memory safe ‘curl’ for a more secure internet

#125

Post by aliceryhl »

hpb42 wrote: post #21

I've heard a lot about Rust's "safety" things. But what are they? How does it compare with modern C++?

Rusts type system is able to carry a lot of information it can use to verify the memory safety of programs at compile time.

For example, the type system includes a piece called the borrow-checker, which is able to guarantee that pointers are still valid when you use them, which eliminates use-after-free and buffer overflows.

In a similar vein, the type system includes information about in which ways types may be shared across threads, and by using this information, the compiler can guarantee that there are no data races whatsoever in multi-threaded programs.

Re: Memory safe ‘curl’ for a more secure internet

#126

Post by nindalf »

nicoburns wrote: post #50
svnpenn wrote: post #32

I like this idea, but I dont know if Hyper is the best package to go with. Hyper occupies part of the Rust ecosystem that I think suffers from package bloat, like much of NPM. For example, currently Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futures-util, h2, hashbrown, http, http-body, httparse, htt…

Part of this is just crates being broken up more in Rust. For example the `http` crate only contains trait (interface) definitions. They break down like so: Platform integration: libc, winapi, winapi-build, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, ws2_32-sys, fuchsia-zircon, fuchsia-zircon-sys, kernel32-sys, redox_syscall Primitive algorithms: itoa, memchr, unicode-xid Proc macro / pinning utilities:…

Why does hyper pull in hashbrown? Isn't it identical to std::collections::HashMap?

Re: Memory safe ‘curl’ for a more secure internet

#127

Post by steveklabnik »

nindalf wrote: post #126
nicoburns wrote: post #50

Earlier quoted context omitted.

Part of this is just crates being broken up more in Rust. For example the `http` crate only contains trait (interface) definitions. They break down like so: Platform integration: libc, winapi, winapi-build, winapi-i686-pc-windows-gnu, winapi-x86_64-pc-windows-gnu, ws2_32-sys, fuchsia-zircon, fuchsia-zircon-sys, kernel32-sys, redox_syscall Primitive algorithms: itoa, memchr, unicode-xid Proc macro / pinning utilities:…

Why does hyper pull in hashbrown? Isn't it identical to std::collections::HashMap?

I imagine that it's there because it is older than that happening. HashBrown's README says

> Since Rust 1.36, this is now the HashMap implementation for the Rust standard library. However you may still want to use this crate instead since it works in environments without std, such as embedded systems and kernels.

I don't know if that use case is important to Hyper or not.

Re: Memory safe ‘curl’ for a more secure internet

#128

Post by staticassertion »

reificator wrote: post #95
tialaramex wrote: post #33

Earlier quoted context omitted.

Historically there was a long period where this didn't do what you expect, which is very bad. What this looks like it does, and indeed does today (modulo bugs some of which could be prevented using Rust) is: Ask totally-not-evil.example.com for this install.sh resource and then run that as root as a Bash script. This is no worse than if you were to have totally-not-evil.example.com give you the bash script on a flopp…

> This is no worse than if you were to have totally-not-evil.example.com give you the bash script on a floppy disk or something. In practice, perhaps. But detecting whether the file is being piped to bash (and not cat/less/grep/etc) is pretty straightforward. https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b... https://code.moparisthebest.com/moparisthebest/curl_bash

Yeah, and it still doesn't matter. Because if you run that command you are already trusting that site with RCE on your machine, so if the premise of the attack is "the site is bad" you were owned anyways.

Re: Memory safe ‘curl’ for a more secure internet

#129

Post by gspr »

brundolf wrote: post #80
woodrowbarlow wrote: post #77

Earlier quoted context omitted.

the binary `curl` is just a CLI frontend for `libcurl`. curl can do a lot more than HTTP. it can transfer data over 20-odd different protocols, including real-time streaming media. it's huge in embedded software.

Oh I see, I didn't realize its core was exposed as a library. That makes more sense!

Once you go looking, you'll find libcurl everywhere. I wouldn't be surprised if your grandma interacts with libcurl multiple times per day without knowing.

Re: Memory safe ‘curl’ for a more secure internet

#130

Post by weinzierl »

hirako2000 wrote: post #124
weinzierl wrote: post #108

Earlier quoted context omitted.

Not depependin on OpenSSL is kind of the point of the original post if I understand it correctly. Also I find the dependency on OpenSSL one major pain in my Rust projects. When you want to build a statically linked binary you need to supply a statically built OpenSSL and if your distro doesn't come with one (like Ubuntu) you are on your own. Yes, there is a Docker container that comes with all the prerequisites but I…

Containers is becoming the way to build stuff, partly for that reason. I think any dependency adds a level of burden, but some things are better delegated to library. I think crypto is a good case, btw I think OpenSSL is not the only one lib for TLS with curl.

> Containers is becoming the way to build stuff, partly for that reason.

This sits wrong with me, but thinking about it: I rewrote the sentence about containers in my comment three times before posting it and it still doesn't sound compelling. Maybe you have a point here.

Post reply on HN