Memory safe ‘curl’ for a more secure internet • Hacker News Board

Re: Memory safe ‘curl’ for a more secure internet

#201

Post by johnisgood » Sun, Oct 11, 2020, 3:23 AM UTC

Measter wrote: post #195

johnisgood wrote: post #191

Earlier quoted context omitted.

Moderately low-level crates require some unsafe blocks. Rust is supposed to be a systems language from what I heard, so if you are going to write something larger, and/or complicated, and/or low-level, you will have to resort to unsafe blocks, I think . Hyper requires 52 packages: autocfg, bitflags, bytes, cfg-if, fnv, fuchsia-zircon, fuchsia-zircon-sys, futures-channel, futures-core, futures-sink, futures-task, futu…

One thing to bear in mind is that the libc and winapi crates are bindings to the libc and Windows C headers, so every single function signature in there will be marked unsafe because it's FFI, so those will be heavily impacting your results. You could also avoid some false positives in comments by searching for "unsafe fn" and "unsafe {", rather than just the word "unsafe", as those are the only tokens (to my knowled…

Without rust/library/, libc, and winapi-rs:

    $ grep -irn 'unsafe {\|unsafe fn' . | wc -l
    1780

Without rust/library/:

    $ grep -irn 'unsafe {\|unsafe fn' . | wc -l
    1937

With rust/library/ (and libc, and winapi-rs):

    $ grep -irn 'unsafe {\|unsafe fn' . | wc -l
    5612

Re: Memory safe ‘curl’ for a more secure internet

#202

Post by dllthomas » Sun, Oct 11, 2020, 7:02 AM UTC

staticassertion wrote: post #128

reificator wrote: post #95

Earlier quoted context omitted.

> This is no worse than if you were to have totally-not-evil.example.com give you the bash script on a floppy disk or something. In practice, perhaps. But detecting whether the file is being piped to bash (and not cat/less/grep/etc) is pretty straightforward. https://www.idontplaydarts.com/2016/04/detecting-curl-pipe-b... https://code.moparisthebest.com/moparisthebest/curl_bash

Yeah, and it still doesn't matter . Because if you run that command you are already trusting that site with RCE on your machine, so if the premise of the attack is "the site is bad" you were owned anyways.

Moreover, the site could be legitimate and ship you something legitimate, but if it's truncated for any reason it could still be a "valid" bash script that now does incorrect things. Consider various prefixes of `rm -rf /tmp/thisscript.working/...`

Re: Memory safe ‘curl’ for a more secure internet

#203

Post by Kednicma » Sun, Oct 11, 2020, 1:53 PM UTC

kingkilr wrote: post #7

Kednicma wrote: post #5

You can't get memory safety by picking a language with `unsafe` blocks. I appreciate the sentiment but the implementation details are important here; I'd take this more seriously if they picked something like OCaml instead of Rust.

Sure you can. First, in a philosophical sense: pointers and x86 CPUs are real, ultimately any safe abstraction must be built on unsafe primitives. The ability and need to do that aren't specific to memory unsafety, we do that all over software engineering. Second, empirically, my experience has been that the design of these abstractions can be safe, but moreover that the cordoning off of unsafe blocks makes 3p auditi…

A TCB should be dozens of lines, not thousands. More code means more places for more bugs to hide.

My experience in Safe Haskell was that, if you have to ask each module individually whether it has a safety property, then you've already created too much work for yourself. Instead, require every module to structurally encode the desired invariant.

Or, in fewer words: If you want memory safety, don't have `unsafe` blocks.

Re: Memory safe ‘curl’ for a more secure internet

#204

Post by tracker1 » Sun, Oct 11, 2020, 7:08 PM UTC

a1369209993 wrote: post #186

tracker1 wrote: post #182

Earlier quoted context omitted.

It depends. If it's in a github repo and there isn't a massive backlog of issues for a software that hasn't been updated in a while, I might think that. One good thing about stat counters for packages combined with GitHub for issue tracking of you can kind of tell. It does take some level of die diligence and isn't easy. But neither is anything relying on say system installed libraries in C projects. I'd rather have…

Can you give even a single example of: - a significant (eg, at least as complex as wget) software project, - that has been unmaintained (no updates, code has the same MD5/etc hash), - with a significant userbase (not sure exactly how to define that one), - for a significant amount of time (at least five years), - which is generally regarded as finished and bug-free (not in need of further development) rather than aba…

5 years is a relatively rough one... in terms of libraries, I come across a lot that are 2+ years old that are feature complete and work. In terms of applications, there are a couple other responses in this thread, but the specific focus in reference was really on libraries themselves, which shouldn't be as complex as wget in general.

Re: Memory safe ‘curl’ for a more secure internet

#205

Post by Groxx » Sun, Oct 11, 2020, 10:38 PM UTC

steveklabnik wrote: post #192

Groxx wrote: post #178

Earlier quoted context omitted.

I assume library authors can specialize code to either semantic as well, and carefully target both in a single library? E.g. with cfg attributes/macros.

I actually don't think so, but I'm also not sure what use case would require you to do this. Generally, you're agnostic, and only in very specific circumstances would you require unwind. I'm not sure when a library would require abort.

I can't come up with a case for requiring abort, but perhaps there are some performance optimizations possible when not unwinding? I assume the compiler does plenty already, but e.g. there would be no need to hang on to any data for making better error messages from a panic, since you can't capture it.

Re: Memory safe ‘curl’ for a more secure internet

#206

Post by baby » Mon, Oct 12, 2020, 10:21 PM UTC

baby wrote: post #98

monadic2 wrote: post #81

Earlier quoted context omitted.

Thinking about crates this way is a revelation to me. Is there any tooling to make analyzing dependencies as you did easier?

Https://Www.Github.com/mimoo/dephell

Sorry www.github.com/mimoo/cargo-dephell

Re: Memory safe ‘curl’ for a more secure internet

#207

Post by ccktlmazeltov » Mon, Oct 12, 2020, 10:23 PM UTC

the8472 wrote: post #104

cle wrote: post #83

Earlier quoted context omitted.

There’s a third option if the package is so important: put it in the standard library. Technically it’s still a dependency but the standard library is maintained with a standard that is rarely matched by third party libraries, and can dramatically simplify the ecosystems’ dependency graph.

Rust's standard library is intentionally kept small since it's initially not always obvious what the best solution is and the stability guarantee makes it the wrong place for evolving, diverse or opinionated APIs. E.g. the async ecosystem offers you to pick between runtimes of different complexity and tradeoffs. Std only picked up the essential traits that allow other crates to interoperate with each other and the la…

hopefully at some point they start including more batteries

Re: Memory safe ‘curl’ for a more secure internet

#208

Post by ccktlmazeltov » Mon, Oct 12, 2020, 10:23 PM UTC

steveklabnik wrote: post #91

cle wrote: post #83

Earlier quoted context omitted.

There’s a third option if the package is so important: put it in the standard library. Technically it’s still a dependency but the standard library is maintained with a standard that is rarely matched by third party libraries, and can dramatically simplify the ecosystems’ dependency graph.

> with a standard that is rarely matched by third party libraries I mean, you can go both ways with this. Standard libraries are significantly more difficult to work on than third party libraries, and I've seen a lot of code in standard libraries that is objectively worse than ecosystem equivalents because of it.

Go disagrees with you

Re: Memory safe ‘curl’ for a more secure internet

#209

Post by jdright » Tue, Oct 13, 2020, 6:46 PM UTC

asveikau wrote: post #185

jdright wrote: post #173

Earlier quoted context omitted.

Try it. If you do not use any "optional" dependency it becomes pretty limited, almost useless for anything serious (eg. zlib, ssl)

Zlib is pretty small. TLS implementions are often giant hairballs but one that many things depend on. You can think of it as a somewhat "system level" dependency.

Being small is not in the question. Being there is. Otherwise we wouldn't be here discussing all hyper dependencies in detail as some of these are a lot smaller than zlib.

Re: Memory safe ‘curl’ for a more secure internet

#210

Post by asveikau » Tue, Oct 20, 2020, 12:19 AM UTC

jdright wrote: post #209

asveikau wrote: post #185

Earlier quoted context omitted.

Zlib is pretty small. TLS implementions are often giant hairballs but one that many things depend on. You can think of it as a somewhat "system level" dependency.

Being small is not in the question. Being there is. Otherwise we wouldn't be here discussing all hyper dependencies in detail as some of these are a lot smaller than zlib.

I am late to reply. What I mean to say is zlib is a small self contained dependency, suitable for static linking, and no major dependency of its own, just math. A lot of more modern libraries and languages tend to have their small dependencies pull in a massive hydra of dependencies.