Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Love C, hate C: Web framework memory problems

alew.is

Post reply on HN
Print view View original

91–100 of 218 posts

Re: Love C, hate C: Web framework memory problems

#91

Post by acidx »

One thing to note, too, is that `atoi()` should be avoided as much as possible. On error (parse error, overflow, etc), it has an unspecified return value (!), although most libcs will return 0, which can be just as bad in some scenarios.

Also not mentioned, is that atoi() can return a negative number -- which is then passed to malloc(), that takes a size_t, which is unsigned... which will make it become a very large number if a negative number is passed as its argument.

It's better to use strtol(), but even that is a bit tricky to use, because it doesn't touch errno when there's no error but you need to check errno to know if things like overflow happened, so you need to set errno to 0 before calling the function. The man page explains how to use it properly.

I think it would be a very interesting exercise for that web framework author to make its HTTP request parser go through a fuzz-tester; clang comes with one that's quite good and easy to use (https://llvm.org/docs/LibFuzzer.html), especially if used alongside address sanitizer or the undefined behavior sanitizer. Errors like the one I mentioned will most likely be found by a fuzzer really quickly. :)

Re: Love C, hate C: Web framework memory problems

#92

Post by MathMonkeyMan »

acidx wrote: post #91

One thing to note, too, is that `atoi()` should be avoided as much as possible. On error (parse error, overflow, etc), it has an unspecified return value (!), although most libcs will return 0, which can be just as bad in some scenarios. Also not mentioned, is that atoi() can return a negative number -- which is then passed to malloc(), that takes a size_t, which is unsigned... which will make it become a very large…

Unspecified, really? cppreference's [C documentation][1] says that it returns zero. The [OpenGroup][2] documentation doesn't specify a return value when the conversion can't be performed. This recent [draft][3] of the ISO standard for C says that if the value cannot be represented (does that mean over/underflow, bad parse, both, neither?), then it's undefined behavior.

So three references give three different answers.

You could always use sscanf instead, which tells you how many values were scanned (e.g. zero or one).

[1]: https://en.cppreference.com/w/c/string/byte/atoi.html

[2]: https://pubs.opengroup.org/onlinepubs/9799919799/functions/a...

[3]: https://www.open-std.org/jtc1/sc22/wg14/www/docs/n2310.pdf

Re: Love C, hate C: Web framework memory problems

#93

Post by acidx »

MathMonkeyMan wrote: post #92
acidx wrote: post #91

One thing to note, too, is that `atoi()` should be avoided as much as possible. On error (parse error, overflow, etc), it has an unspecified return value (!), although most libcs will return 0, which can be just as bad in some scenarios. Also not mentioned, is that atoi() can return a negative number -- which is then passed to malloc(), that takes a size_t, which is unsigned... which will make it become a very large…

Unspecified, really? cppreference's [C documentation][1] says that it returns zero. The [OpenGroup][2] documentation doesn't specify a return value when the conversion can't be performed. This recent [draft][3] of the ISO standard for C says that if the value cannot be represented (does that mean over/underflow, bad parse, both, neither?), then it's undefined behavior. So three references give three different answers…

The Linux man page (https://man7.org/linux/man-pages/man3/atoi.3.html#VERSIONS) says that POSIX.1 leaves it unspecified. As you found out, it's really something that should be avoided as much as possible, because pretty much everywhere disagrees how it should behave, especially if you value portability.

sscanf() is not a good replacement either! It's better to use strtol() instead. Either do what Lwan does (https://github.com/lpereira/lwan/blob/master/src/lib/lwan-co...), or look (https://cvsweb.openbsd.org/src/lib/libc/stdlib/strtonum.c?re...) at how OpenBSD implemented strtonum(3).

For instance, if you try to parse a number that's preceded by a lot of spaces, sscanf() will take a long time going through it. I've been hit by that when fuzzing Lwan.

Even cURL is avoiding sscanf(): https://daniel.haxx.se/blog/2025/04/07/writing-c-for-curl/

Re: Love C, hate C: Web framework memory problems

#94

Post by davemp »

yipikaya wrote: post #78

As an aside, it's amusing that it took 25 years for C coders to embrace the C99 named struct designator feature: HttpParser parser = { .isValid = true, .requestBuffer = strdup(request), .requestLength = strlen(request), .position = 0, }; All the kids are doing it now!

I’m still regularly getting on projects and moving C89 variable declarations from the start of functions to where they’re initialized, but I guess it’s not the kids doing it.

Re: Love C, hate C: Web framework memory problems

#95

Post by ambicapter »

1718627440 wrote: post #77
lelanthran wrote: post #54

Earlier quoted context omitted.

> Good C code will try to avoid allocations as much as possible in the first place. I've upvoted you, but I'm not so sure I agree though. Sure, each allocation imposes a new obligation to track that allocation, but on the downside, passing around already-allocated blocks imposes a new burden for each call to ensure that the callees have the correct permissions (modify it, reallocate it, free it, etc). If you're doing…

To reduce the amount of allocation instead of: struct parsed_data * = parse (...); struct process_data * = process (..., parsed_data); struct foo_data * = do_foo (..., process_data); you can do parse (...) { ... process (...); ... } process (...) { ... do_foo (...); ... } It sounds like violating separation of concerns at first, but it has the benefit, that you can easily do procession and parsing in parallel, and al…

How testable is this, though?

Re: Love C, hate C: Web framework memory problems

#96

Post by dgfitz »

LexiMax wrote: post #87
jacquesm wrote: post #43

Earlier quoted context omitted.

Yawn. Really, if you have nothing to say don't do it here.

Gotcha hypocrisy might be a really cheap thing to point out, but they're not wrong. I have noticed my fair share of Rust Derangement Syndrome in C++ spaces that seems completely outsized from the series of microaggressions that they eventually point out when asked "Why?"

It’s interesting, over the past 15 years I’ve had occasion to work with other c/c++ devs on various contracts, probably 50ish distinct different companies. Not once has rust even come up in casual conversation.

Re: Love C, hate C: Web framework memory problems

#97

Post by obviouslynotme »

lelanthran wrote: post #54
bluetomcat wrote: post #21

Good C code will try to avoid allocations as much as possible in the first place. You absolutely don’t need to copy strings around when handling a request. You can read data from the socket in a fixed-size buffer, do all the processing in-place, and then process the next chunk in-place too. You get predictable performance and the thing will work like precise clockwork. Reading the entire thing just to copy the body o…

> Good C code will try to avoid allocations as much as possible in the first place. I've upvoted you, but I'm not so sure I agree though. Sure, each allocation imposes a new obligation to track that allocation, but on the downside, passing around already-allocated blocks imposes a new burden for each call to ensure that the callees have the correct permissions (modify it, reallocate it, free it, etc). If you're doing…

The most important pattern to learn in C is to allocate a giant arena upfront and reuse it over and over in a loop. Ideally, there is only one allocation and deallocation in the entire program. As with all things multi-threaded, this becomes trickier. Luckily, web servers are embarrassingly parallel, so you can just have an arena for each worker thread. Unluckily, web servers do a large amount of string processing, so you have to be careful in how you build them to prevent the memory requirements from exploding. As always, tradeoffs can and will be made depending on what you are actually doing.

Short-run programs are even easier. You just never deallocate and then exit(0).

Re: Love C, hate C: Web framework memory problems

#98

Post by accelbred »

messe wrote: post #5

> Another interesting choice in this project is to make lengths signed: There are good reasons for this choice in C (and C++) due to broken integer promotion and casting rules. See: "Subscripts and sizes should be signed" (Bjarne Stroustrup) https://open-std.org/jtc1/sc22/wg21/docs/papers/2019/p1428r0... As a nice bonus, it means that ubsan traps on overflow (unsigned overflows just wrap).

If using C23, _BitInt allows for integer types without promotion.

Re: Love C, hate C: Web framework memory problems

#99

Post by cogman10 »

lock1 wrote: post #48
bluetomcat wrote: post #21

Good C code will try to avoid allocations as much as possible in the first place. You absolutely don’t need to copy strings around when handling a request. You can read data from the socket in a fixed-size buffer, do all the processing in-place, and then process the next chunk in-place too. You get predictable performance and the thing will work like precise clockwork. Reading the entire thing just to copy the body o…

Why does "good" C have to be zero alloc? Why should "nice" javaesque make little sense in C? Why do you implicitly assume performance is "efficient problem solving"? Not sure why many people seem fixated on the idea that using a programming language must follow a particular approach. You can do minimal alloc Java, you can simulate OOP-like in C, etc. Unconventional, but why do we need to restrict certain optimization…

> Why should "nice" javaesque make little sense in C?

Very importantly, because Java is tracking the memory.

In java, you could create an item, send it into a queue to be processed concurrently, but then also deal with that item where you created it. That creates a huge problem in C because the question becomes "who frees that item"?

In java, you don't care. The freeing is done automatically when nobody references the item.

In C, it's a big headache. The concurrent consumer can't free the memory because the producer might not be done with it. And the producer can't free the memory because the consumer might not have ran yet. In idiomatic java, you just have to make sure your queue is safe to use concurrently. The right thing to do in C would be to restructure things to ensure the item isn't used before it's handed off to the queue or that you send a copy of the item into the queue so the question of "who frees this" is straight forward. You can do both approaches in java, but why would you? If the item is immutable there's no harm in simply sharing the reference with 100 things and moving forward.

In C++ and Rust, you'd likely wrap that item in some sort of atomic reference counted structure.

Re: Love C, hate C: Web framework memory problems

#100

Post by goalieca »

simonask wrote: post #46
uecker wrote: post #30

Earlier quoted context omitted.

I think the correct comparison is a sharp knife. It is extremely useful and while there is a risk it is fully acceptable. The idea that we should all use plastic knifes because there are often accidents with knifes is wrong and so is the idea that we use should abandon C because of memory safety. I follow computer security issues for several decades, and while I think we should have memory safety IMHO the push and ar…

I’m sorry, but there is an incredible amount of hard data on this, including the number of CVEs directly attributable to memory safety bugs. This is publicly available information, and we as an industry should take it seriously. I don’t mean to be disrespectful, but this cavalier attitude towards it reads like vaccine skepticism to me. It is not serious. Programming can be inconsequential, but it can also be national…

CVE are important but there’s also a lot of theatre there. How many are known exploitable? Most aren’t if you follow threat intel. Most of the Internet infrastructure is running c/c++ and is very safe.
Post reply on HN