How can C Programs be so Reliable? (2008) • Hacker News Board

Re: How can C Programs be so Reliable? (2008)

#131

Post by dsego » Fri, Dec 06, 2013, 4:10 AM UTC

dllthomas wrote: post #14

"[Pointers are] arguably the trickiest concept in low-level languages, having no simple real-world analogy[.]" I'm not sure how "address" is not a good, "simple real-world analogy" for pointers. It may be that I've just internalized enough of the behavior of pointers that I'm glossing over important differences, though... I struggled with pointers long enough ago that I don't remember struggling with pointers.

I think pointer syntax in C, not pointers themselves, is what causes confusion for most beginner programmers. This post explains it well: http://objcsharp.wordpress.com/2013/08/14/the-great-pointer-...

Re: How can C Programs be so Reliable? (2008)

#132

Post by daeken » Fri, Dec 06, 2013, 4:25 AM UTC

MichaelGG wrote: post #116

vidarh wrote: post #94

Earlier quoted context omitted.

I grew up using the Amiga, when having memory allocation fail was routine (a standard Amiga 500 for example, came with 512KB RAM, and was rarely expanded to more than 1MB, so you would run out of memory). What you do when malloc() fails depends entirely on your application: If a desktop application on the Amiga would shutdown just because a memory allocation failed, nobody would use it. The expection was you'd gracef…

Isn't fork the real offender, which requires Linux to overcommit by default? Disabling swap shouldn't affect that, right? Just makes your problem happen later, in a somewhat non-deterministic way. Without fork, what reason do you not disable swap? I can only think of an anonymous mmap where you want to use the OS VM as a cache system. But that's solved easily enough by providing a backing file, isn't it?

> Isn't fork the real offender, which requires Linux to overcommit by default?

Maybe I'm missing something, but how does fork require overcommitment? When you fork, you end up with COW pages, which share underlying memory. They don't guarantee that physical memory would be available if every page were touched and required a copy; they just share underlying physical memory. Strictly speaking, very little allocation has to happen for a process fork to occur.

Re: How can C Programs be so Reliable? (2008)

#133

Post by tracker1 » Fri, Dec 06, 2013, 4:27 AM UTC

vidarh wrote: post #84

RyanZAG wrote: post #19

Earlier quoted context omitted.

It depends on the problem you're trying to solve, I think. Let's consider a command line application that fetches a URL, like wget. Without exceptions, you would check the return code of all the system functions you call (dns, sockets, etc), and if any of them fail you can't really recover. You just write out an error message and exit. With exceptions, you could wrap the whole thing and if any system function throws…

> Without exceptions this is tougher as you need to unroll everything manually in order to retry. I don't get why you think this is a problem. You can trivially "unroll" it by explicitly writing your functions to be single-exit and "unrolling" at the end before returning whatever your result is. "Exception handling" without "real" exceptions is then also trivial: just check the appropriate error code and take whateve…

This is why I really like the callback pattern that node.js tends to use for all it's IO api calls... the callback always has the option for an error first.. generally if its' a nested callback, bubble up early. Internally, the C code can just check the state and call the callback method early if an error state arises.

Re: How can C Programs be so Reliable? (2008)

#134

Post by kansface » Fri, Dec 06, 2013, 4:27 AM UTC

vidarh wrote: post #96

nitrogen wrote: post #60

Earlier quoted context omitted.

If malloc() fails now, it might succeed again later. So you can just go on doing everything else you were doing, then try the memory-hungry operation again in the future. For example, this could be important in systems where you might be controlling physical hardware at the same time as reading commands from a network. It's probably a good idea to maintain control of the hardware even if you don't have enough memory…

This is a pet peeve of mine with modern applications: So many of them just throw their metaphorical hands in the air and give up. Prior to swap and excessive abuse of virtual memory this was not an option: If you gave up on running out of memory, your users gave up on your application. On the Amiga, for example, being told an operation failed due to lack of memory and given the chance to correct it by closing down so…

Conversely, if app developers wrote the same code that embedded systems programmers do, we'd never have any apps to use. Its just not worth the trade off- moreover, telling a user to free memory is a losing battle.

Re: How can C Programs be so Reliable? (2008)

#135

Post by majika » Fri, Dec 06, 2013, 5:05 AM UTC

vidarh wrote: post #96

nitrogen wrote: post #60

Earlier quoted context omitted.

If malloc() fails now, it might succeed again later. So you can just go on doing everything else you were doing, then try the memory-hungry operation again in the future. For example, this could be important in systems where you might be controlling physical hardware at the same time as reading commands from a network. It's probably a good idea to maintain control of the hardware even if you don't have enough memory…

This is a pet peeve of mine with modern applications: So many of them just throw their metaphorical hands in the air and give up. Prior to swap and excessive abuse of virtual memory this was not an option: If you gave up on running out of memory, your users gave up on your application. On the Amiga, for example, being told an operation failed due to lack of memory and given the chance to correct it by closing down so…

Doesn't it duplicate effort to put the responsibility of checking for available memory on individual applications?

I think, in most computing environments, it should be the operating system's responsibility to inform the user that free memory is running out. Applications should be able to assume that they can always get memory to work with.

I think the swap is an extremely sensible solution, in that executing programs slowly is (in most environments, including desktop) better than halting programs completely. It provides an opportunity for the user to fix the situation, without anything bad happening. Note that the swap is optional anyway, so don't use it if you don't like it.

Comparing modern computing environments to the Amiga is laughable. It's not even comparable to modern embedded environments, because they serve(d) different purposes.

I'm a hobbyist C application/library developer who assumes memory allocation always works.

Re: How can C Programs be so Reliable? (2008)

#136

Post by flogic » Fri, Dec 06, 2013, 5:48 AM UTC

mfenniak wrote: post #128

flogic wrote: post #125

Earlier quoted context omitted.

I disagree. Go errors aren't even error codes. The error interface is just one method that returns a string. The only advantage Go has over C is that the errors are no longer inband with normal return values. Once you get one though, you're back in the perl5 land of reasoning about your error by scraping a string. I'm completely baffled as to why they didn't attempt to bundle more machine readable information into th…

Go errors are machine-readable data. Most modules expose variables with the naming convention Err... that represent the errors that they can return. The errors can be compared with those values (eg. err == os.ErrPermission). Some modules also define error types that provide additional error data, such as net.OpError; a type-cast can be used to check if the error is of that type, and then access the data within. As a…

I stand corrected. I'd still prefer normal exception handling but whatever.

Re: How can C Programs be so Reliable? (2008)

#137

Post by drdaeman » Fri, Dec 06, 2013, 5:50 AM UTC

C programs are so damn reliable I spent last three weeks mostly trying to debug others' code (from multiple software projects - I was lucky enough they decided they'd all start hitting obscure bugs at approximately same time) and waiting for it to crash again.

Just imagine my feelings when a critical server that ran just fine for 3 years gets a bit more load and starts getting assertion failures. That was a known and already fixed bug, but after version bump to next stable those had changed to SIGSEGVs - and it's not trivial NULL pointer dereferencing but a weird concurrency issue.

C has neither [relatively] strong guarantees of OCaml and Haskell (so I could be at least sure there're no trivial bugs with that weird macros doing crazy pointer juggling), nor dynamic flexibility of Python and Erlang (so I could hack working runtime live, letting it fail and hot-swapping code at will).

Re: How can C Programs be so Reliable? (2008)

#138

Post by army » Fri, Dec 06, 2013, 6:20 AM UTC

daeken wrote: post #132

MichaelGG wrote: post #116

Earlier quoted context omitted.

Isn't fork the real offender, which requires Linux to overcommit by default? Disabling swap shouldn't affect that, right? Just makes your problem happen later, in a somewhat non-deterministic way. Without fork, what reason do you not disable swap? I can only think of an anonymous mmap where you want to use the OS VM as a cache system. But that's solved easily enough by providing a backing file, isn't it?

> Isn't fork the real offender, which requires Linux to overcommit by default? Maybe I'm missing something, but how does fork require overcommitment? When you fork, you end up with COW pages, which share underlying memory. They don't guarantee that physical memory would be available if every page were touched and required a copy; they just share underlying physical memory. Strictly speaking, very little allocation ha…

The problem is once you touch those memory pages, total memory usage increases even if you don't call malloc.

Re: How can C Programs be so Reliable? (2008)

#139

Post by nitrogen » Fri, Dec 06, 2013, 6:27 AM UTC

majika wrote: post #135

vidarh wrote: post #96

Earlier quoted context omitted.

This is a pet peeve of mine with modern applications: So many of them just throw their metaphorical hands in the air and give up. Prior to swap and excessive abuse of virtual memory this was not an option: If you gave up on running out of memory, your users gave up on your application. On the Amiga, for example, being told an operation failed due to lack of memory and given the chance to correct it by closing down so…

Doesn't it duplicate effort to put the responsibility of checking for available memory on individual applications? I think, in most computing environments, it should be the operating system's responsibility to inform the user that free memory is running out. Applications should be able to assume that they can always get memory to work with. I think the swap is an extremely sensible solution, in that executing program…

Most computing environments don't have a user to speak of, and the correct response of an application to an out of memory error could range from doing nothing to sounding an alarm.

As a user, I find it incredibly frustrating when my old but indispensable music software runs out of address space (I have plenty of RAM) and, instead of canceling the current operation (e.g. processing some large segment of audio), just dies with a string of cryptic error dialogs. The best thing for the user in this case is to hobble along without allocating more memory, not to fail catastrophically by assuming that memory allocation always works.

Swap is not a good solution because when there's enough swap to handle significant memory overuse, the system becomes unresponsive to user input since the latency and throughput to swap are significantly slower than RAM.

Re: How can C Programs be so Reliable? (2008)

#140

Post by MichaelGG » Fri, Dec 06, 2013, 6:30 AM UTC

daeken wrote: post #132

MichaelGG wrote: post #116

Earlier quoted context omitted.

Isn't fork the real offender, which requires Linux to overcommit by default? Disabling swap shouldn't affect that, right? Just makes your problem happen later, in a somewhat non-deterministic way. Without fork, what reason do you not disable swap? I can only think of an anonymous mmap where you want to use the OS VM as a cache system. But that's solved easily enough by providing a backing file, isn't it?

> Isn't fork the real offender, which requires Linux to overcommit by default? Maybe I'm missing something, but how does fork require overcommitment? When you fork, you end up with COW pages, which share underlying memory. They don't guarantee that physical memory would be available if every page were touched and required a copy; they just share underlying physical memory. Strictly speaking, very little allocation ha…

If there's no overcommit, each of those COW pages needs some way of making sure it can actually be written to. Isn't that literally the point of overcommit? Giving processes more memory than they can actually use on the assumption they probably won't use it? And Windows takes the different approach of never handing out memory unless it can be serviced (via RAM or pagefile).

What am I missing? (I know you know way more about this than I do.)