A significant amount of programming is done by superstition • Hacker News Board

Re: A significant amount of programming is done by superstition

#161

Post by danbruc » Thu, Mar 26, 2015, 7:31 AM UTC

kazinator wrote: post #126

danbruc wrote: post #109

Earlier quoted context omitted.

I obviously do not and can not trace the entire code path from a print statement through the class library, the operating system, the display driver and the hardware, some things I will still take for granted. But if there is a really critical aspect to what I am doing, say some thread synchronization, I will definitely not just follow some sample code, but read the relevant parts of the memory model of the language…

The computer doesn't care which code you consider to be in the scope of your concern, and which you take for granted. If you examine why you're concerned about some code more, you will probably come to the realization that you're less confident in it because it is less old and mature, has fewer users, and has had fewer developer eyes on it and such. I.e. whether explicitly or not, you're focusing your understanding w…

It's not about the quality or maturity of the code I am relying on, it's about the quality of my code. I want my code to do the right thing in the simplest and most elegant way possible and part of this is knowing really well how everything works.

Re: A significant amount of programming is done by superstition

#162

Post by roel_v » Thu, Mar 26, 2015, 8:12 AM UTC

dandelany wrote: post #20

rjbwork wrote: post #11

I think a better title might be "A vast majority of programming is done at a level of abstraction that this author is slightly uncomfortable with". If you work in Java or Ruby or Python or C#, or even C++ all day with say, an IoC container for argument's sake, you're certainly not constantly thinking "okay this interface will be resolved to this type, which has these dependencies which will be resovled to these types…

I don't think this is about abstraction. Abstraction implies having a deep understanding of the problem domain and encapsulating it in such a way that the implementation details can be ignored and the domain can be reasoned about at a higher level. I think a better title would be "A significant amount of (shipped, so-called production-quality) code is not fully understood by its authors".

"I think a better title would be "A significant amount of (shipped, so-called production-quality) code is not fully understood by its authors"."

Well duh. Not a single program shipped today, no matter how small, is "fully" understood by its authors, because to do so, one would have to understand every implementation nuance and bug of OS layers, libraries used, other parts it interacts with, ... And even for less strict definitions of "fully" - for any program that took longer than say 2 weeks to write, how can anyone claim they really know the details of everything, at the same time? Including the nuanced differences between various of OS calls, and the effect some environment variable might have on your localization code, just to name one thing?

Re: A significant amount of programming is done by superstition

#163

Post by marak830 » Thu, Mar 26, 2015, 9:20 AM UTC

anon4 wrote: post #14

I'll have to admit something about myself -- I've never been able to learn how to do anything from just reading its first principles. I need to have a working example that I can then adapt and change and observe changes in the result. This has been especially true in programming. I've been writing programs in some capacity for 18 years now and not once in that time have I been able to simply read the reference manual…

Thankyou! Also to everyone else who replied. I thought i was a bad programmer. I am cobstantly in a cycle of read, try, adjust, repeat until it works, rebuild it now i understand better how it works.

Re: A significant amount of programming is done by superstition

#164

Post by fsloth » Thu, Mar 26, 2015, 9:32 AM UTC

ecnahc515 wrote: post #124

avmich wrote: post #81

Earlier quoted context omitted.

> Experimentation is the key to science. Computer science is a branch of mathematics. Here you build models, reason about them, find properties, then prove them, then rely on them. Tests - a.k.a. experiments - are usually not exhaustive and don't provide guarantees of correctness. Having said that, I agree that experiment is of crucial importance - how else you would validate your models in the first place? However,…

Physics of also a branch of mathematics. But experimentation is key to proving correctness.

I would restate this so that software engineering and physics are fields concerned about building practical models. They are similar in a way that they both employ and invent models communicated through a mathematical formalism. "Model" is the key here, not mathematics.

Re: A significant amount of programming is done by superstition

#165

Post by fsloth » Thu, Mar 26, 2015, 9:34 AM UTC

prawn wrote: post #91

Gravityloss wrote: post #59

Earlier quoted context omitted.

Gall's law is related to this. I think it's reasonable that people want to create software components that have been proven to work and then just forget about their internal details and copy-pas... apply them. It's also what Bret Victor has been getting at. Very often the component's input-output mapping is much more relevant than implementation details. And you can get a feel for that by fiddling with the components…

Gall's Law for anyone who didn't know or couldn't remember it: "A complex system that works is invariably found to have evolved from a simple system that worked. A complex system designed from scratch never works and cannot be patched up to make it work. You have to start over with a working simple system." – John Gall (1975)

Thanks for the reference, this is the best quote I've found in a long while. Strangely, I was unfamiliar with this.

Re: A significant amount of programming is done by superstition

#166

Post by Rapzid » Thu, Mar 26, 2015, 9:52 AM UTC

gdrulia wrote: post #151

shpx wrote: post #120

Earlier quoted context omitted.

I think for a lot of people reading man pages is not as fun and satisfying as just making things and watching them work. It's a form of instant gratification.

from my own personal experience I can tell that most of the time I prefer stackoverflow instead of man pages simply because I need to achieve one task and stackoverflow contains answer of how to do it, man pages can teach me how to do all sorts of a tasks, then I have to choose the right one from those that I have learned. On top of that, to be able to use man page, I need to know command in the first place, while go…

I think it really depends.. I was at a job where it was like "Hey, these monthly md checks are causing IO issues let us disable them." So I read the man page for md and yeah... Let's not disable them.

This is an anecdote sure, but an important one. Not everyone needs to understand what's going on with everything but I can tell you the ones who seek out the information and really try to grok the systems are rare and can be well compensated for it.

Another anecdote: trying to use ZFS, and in particular de-dupe, without understanding it. De-dupe has its own trail of tears laid out from people who did not understand the memory requirements of it.

Hello, I'm Rapzid. I'm 30 years old, and I'm a man page reader. Sue me.

Re: A significant amount of programming is done by superstition

#167

Post by olavk » Thu, Mar 26, 2015, 10:17 AM UTC

avmich wrote: post #81

GauntletWizard wrote: post #46

Earlier quoted context omitted.

Almost nobody can. Some people think they can, and charge forward with it, but rarely do they produce something that actually works. Usually, they bang on it and test it and cargo-cult it until it works, then convince themselves that it's from 'first principles'. Even if it were a common ability to produce engineering work from written design principles, I'm not certain that's what we want. Experimentation is the key…

> Experimentation is the key to science. Computer science is a branch of mathematics. Here you build models, reason about them, find properties, then prove them, then rely on them. Tests - a.k.a. experiments - are usually not exhaustive and don't provide guarantees of correctness. Having said that, I agree that experiment is of crucial importance - how else you would validate your models in the first place? However,…

Then most programming is not computer science.

Re: A significant amount of programming is done by superstition

#168

Post by olavk » Thu, Mar 26, 2015, 10:19 AM UTC

coldnebo wrote: post #135

Swizec wrote: post #68

Earlier quoted context omitted.

Anyone can build a bridge that stands up, but only an engineer can build a bridge that barely stands up. Good Enough is our main design goal. Anything more than good enough means you're wasting resources.

This sounds like Engineering, but be aware that in mechanical engineering the requirements are well-defined for materials and load. Consider the famous bridge resonance problem and you will see why wind resonance is now a required subject for structural engineering. ME has an advantage over CSE in this regard because in CS the requirements are usually very poorly stated. About the best and most rigorous requirements…

The web might have been created by amateurs, but it works incredibly well. Sure, we have browser incompatibilities and whatnot, but this is an effect of having multiple independent implementations of the standards, which is part of what makes the web work in the first place.

Re: A significant amount of programming is done by superstition

#169

Post by feskmldms » Thu, Mar 26, 2015, 10:25 AM UTC

bostik wrote: post #153

krick wrote: post #106

That's right, but I guess we're missing the real problem here. The real gold are these words: > You've really read the LSB specification for init scripts and your distribution's distro-specific documentation? If so, you're almost certainly a Debian Developer or the equivalent specialist for other distributions. Many of us would agree, that it's true. But why is it true? Shouldn't that make us suspicious in the first…

There is a balance to be struck. Progress depends on humans being lazy. We have inventions and innovations precisely because humans are so good at finding ways to make their (own) lives easier - usually by eliminating repetitive tasks via some kind of work-around or alternate approach. There is a well known quote about insanity [0] and blind repetition that occasionally gets associated with idiocy too. If everyone wa…

I don't think laziness accounts for that behavior. Need for efficiency and business might, since most inventions are motivated by yet other endeavors which hardly counts as lazy. I think that the most basic ideas involved in hunting, farming and fighting are motivated by the ultimate need to get some rest.

OTOH, rest shouldn't be confused with laziness, which has a negative connotation, because some see idleness as an ill when it is assigned the highest priority before other desires/needs.

Re: A significant amount of programming is done by superstition

#170

Post by lmm » Thu, Mar 26, 2015, 10:31 AM UTC

This is specific to shell scripting and to the history of nix standardization. No-one follows or cares about the LSB[1]; certification is expensive and based on buggy test programs that practically force you to "code to the test" and not to the specification. And when so many implementations ignore the spec, it would be more superstitious to code to the magical spec and assume it's going to work, rather than writing something and testing it empirically.

When I'm programming (in Python or Scala) I can and do check the language specification if something is confusing, and I don't think I'm alone in this. It may be a "significant" amount of programming, but there are definitely areas of programming with more rigour.

[1] I blame Debian, usually quixotic in its adherence to standards like the FHS - but when it came to the LSB they prefer to pretend that "alien" counts as RPM support. That set the tone for how much other distros tend to care about the spec.