Live data from Hacker News

I'm Sure It Will Only Take You A Few Days To Code

danshipper.com

111–117 of 117 posts

Re: I'm Sure It Will Only Take You A Few Days To Code

#111
post #58

Earlier quoted context omitted.

Sounds like mathematicians defining their assumptions - it's up to them where they draw the line as to what they will work on themselves, and what they'll accept as a given. i.e. it's arbitrary. Taking it back to turing machines, you can simulate any specific design/implementation of a turing machine on some another turing machine, given a program of constant size (an emulator for the first machine). This can be appl…

Here's why I'm interested. The most important thing in making good software is to eschew unnecessary complexity. A program's size is the best measure of its complexity. Therefore we should try to write shorter programs. Much, much shorter programs. (See Alan Kay's VPRI project etc. etc.) It's easy enough to measure the size of a program: LOC or something like it. But how do we define what a program is ? Surprisingly,…

I think my bracketed aside covers this: by using a probability density, it's not affected by how many people are actually using it, or whether that changes. Of course, it's only a conceptual help, since you can't actually know this density: it amounts to knowing all possible tasks now and in future (a practical impossibility) and also knowing all the solutions (a theoretical impossibility). It wraps all that up in two words.

I think defining whether A and B are part of the same program is a red herring. If you think of their calls to each other taking place through an interface, they don't need to know about each other - they just expect something that fulfills the contract of the interface. Again, it's like a mathematician arbitrarily making assumptions: we'll take a component that does this as a given. So, A and B aren't part of the same program. They aren't related at all (except that each happens to fulfill some interfaces needed by the other). This perspective is similar to taking probability density, in that it replaces a concrete actual program with an abstract representation.

But how then does one measure complexity? My view is you can't, in practice. But we can have a concept that can guide us. Of course, when staying entirely within a single program it's clear that we should factor out commonalities to reduce length. But what about factoring those things out into libraries? I think an approach that works is to think about being in the business of selling those libraries - but not with the perspective of making money for yourself, but for improving the world. [again, this is an abstract concept, so we don't actually need to sell them and can ignore the real-world issue of marketing] So, there's questions like: is there a need for this, in many programs? Can it be reused generally? (or is it specific to program A - shades of Brooks program-vs-product here) Is it a clean abstraction (i.e. can it be used without knowing its details - this reduces complexity in the code that uses it). How much effort/LoC is saved by it? Factoring out code into a library that is broadly needed, is simple to use, saves a lot of code, is simple to use (doesn't require lots of code to use; solution doesn't create more problems - not the cure is worse than the disease) would be needed broadly.

Good libraries save the world!

I agree that in practice, it becomes a social (or community) issue, and whether it's adopted is important. You might have a great product that no one ever uses; or a crappy one that everyone does. It's messy, in that it's affected by non-programming issues like the particular alternatives existing at this moment, the need for it at this moment. [But again, a probability density circumvents all this]

Maybe it's a bit like whether the scientific method requires the communication of the results: you haven't done "science" until you tell someone. It also seems absurd - how can communication affect what you've already done? - but if you being with science being a community effort (like language, commerce and law), then it does make sense: you can't do science alone. And our established science was a community effort (notwithstanding that we wouldn't know of the other stuff). This is contentious - I mention it because it seems like a similar contention.

Finally, some caveats. Though I think complexity is central (Occam's razor) and even did a masters around it, it's not everything. For example, in English communication, redundancy helps by giving listeners a way to verify understanding, and different perspectives in case they didn't grasp it, or if the message was ambiguous and the speaker didn't realize it. and raw repetition in case they weren't listening or the message was corrupted/obscured. Some of these were significant in Shannon's original work on information theory. In communication in general, a constant background helps convey a changing foreground - as simple as the lines on a page of printed text; or looking at a file that is laid out neatly, so you can quickly orient yourself, and also notice irregularities.

In code, some repetitive concrete structure can be helpful to the humans reading it. Related to this is the concept of "accidental redundancy". This is when two pieces of code are identical, so you factor them out - but it turns out that they change on independent bases, and it was just a coincidence that they happened to appear identical syntactically. i.e. their meaning was different. You can't tell this just by looking at the code, nor by measuring the length. You have to understand the model beneath the code, and even more importantly, understand the problem being solved (though there might be no way to know the two things differ, until they do differ).

My over-idealization of short code eventually made me see the benefits of seemingly over-concrete representations. :-) But short code is still central for me, as it seems to be in scientific discovery. The simpler hypothesis/code is likely closer to the truth. And it's happened to me many times that simpler code generalizes in just the right way - before I realized that it would need to generalize that way. :-) Also, the Dirac/antimatter story is nice on this point: http://arachnoid.com/is_math_a_science/index.html (near the end)

Re: I'm Sure It Will Only Take You A Few Days To Code

#112
post #111

Earlier quoted context omitted.

Here's why I'm interested. The most important thing in making good software is to eschew unnecessary complexity. A program's size is the best measure of its complexity. Therefore we should try to write shorter programs. Much, much shorter programs. (See Alan Kay's VPRI project etc. etc.) It's easy enough to measure the size of a program: LOC or something like it. But how do we define what a program is ? Surprisingly,…

I think my bracketed aside covers this: by using a probability density, it's not affected by how many people are actually using it, or whether that changes. Of course, it's only a conceptual help, since you can't actually know this density: it amounts to knowing all possible tasks now and in future (a practical impossibility) and also knowing all the solutions (a theoretical impossibility). It wraps all that up in tw…

You make some good points, but it seems to me that each time we come close to the practical questions, the conversation veers away from them again. The one practical detail I understand and agree with you about is that sometimes when one factors to remove duplication, the change makes things worse by blocking the evolution of one or more of the original passages of code. They were "false friends" as the French say, and the abstraction you thought you found was spurious. It takes a while to get a feel for when this is the case. That being said, it's not that hard and I don't think it has much bearing on the big question, which is: how can we minimize the overall complexity of a system? I believe this is (1) deeper, (2) harder, (3) more important, and (4) more doable than (as far as I can tell) you give it credit for.

defining whether A and B are part of the same program is a red herring

Do you accept that minimizing complexity is critical and that program length is the best indicator of complexity? That makes program length critical, and you can't measure that without knowing what counts as part of the program. What part of this argument is wrong?

I'm beginning to think it boils down to whatever a particular team or organization has to maintain. If you're Microsoft then you pay for the length of Windows, otherwise you don't have to count it. If this is true then it's entirely a social question, and what we should be considering is something like lines of code per maintainer.

Re: I'm Sure It Will Only Take You A Few Days To Code

#113
post #111

Earlier quoted context omitted.

I think my bracketed aside covers this: by using a probability density, it's not affected by how many people are actually using it, or whether that changes. Of course, it's only a conceptual help, since you can't actually know this density: it amounts to knowing all possible tasks now and in future (a practical impossibility) and also knowing all the solutions (a theoretical impossibility). It wraps all that up in tw…

You make some good points, but it seems to me that each time we come close to the practical questions, the conversation veers away from them again. The one practical detail I understand and agree with you about is that sometimes when one factors to remove duplication, the change makes things worse by blocking the evolution of one or more of the original passages of code. They were "false friends" as the French say, a…

The spurious abstraction issue was a caveat. The central problem is the hardest possible problem, IMHO; my suggestions are just what we can do about it.

how can we minimize the overall complexity of a system?

Nice clear statement. If we're minimizing complexity of an overall system that includes A and B, it doesn't matter whether A and B are distinct programs. OTOH, if we exclude one from the system, we can't measure complexity effectively - unless we're comparing code that uses the same environment e.g. A vs. A', and B is a library.

If MS provides an extensive framework, that doesn't quite suit your project, you may distort your project (i.e. add unnecessary complexity) to take advantage of it.

I'm still not fully clear on your question/purpose (though minimize overall complexity seems close). You want something you can use as a guiding value/principle for your company; also something practical. "Do you accept that minimizing complexity is critical..." Critical, for what? [FWIW, I think it's critical for finding truth, generalizing well, and also for beauty/rightness/elegance]

I'm now going to have a beer after all this! :-) Cheers!

Re: I'm Sure It Will Only Take You A Few Days To Code

#114
post #53

Is Pachelbel's Canon considered complicated as far as music performance goes? Reading that knocked me right out of the flow of this article. What's he thinking of? Maybe someone's complicated improvisations on top of the (really simple) canon? On topic: the post didn't really cover where the layperson's base estimate comes from -- a layperson sees X pages, or X features, and multiplies that by... what? The reference…

Is Pachelbel's Canon considered complicated as far as music performance goes? Consider the more complicated violin part.

Still, you hear grade schoolers play it.

And wow, if you've ever been forced to play the part of anything other than violin -- I played the viola part innumerable times in HS orchestra, and the cello part was even worse -- it's almost a punishment.

Re: I'm Sure It Will Only Take You A Few Days To Code

#115
post #113

Earlier quoted context omitted.

You make some good points, but it seems to me that each time we come close to the practical questions, the conversation veers away from them again. The one practical detail I understand and agree with you about is that sometimes when one factors to remove duplication, the change makes things worse by blocking the evolution of one or more of the original passages of code. They were "false friends" as the French say, a…

The spurious abstraction issue was a caveat. The central problem is the hardest possible problem, IMHO; my suggestions are just what we can do about it. how can we minimize the overall complexity of a system? Nice clear statement. If we're minimizing complexity of an overall system that includes A and B, it doesn't matter whether A and B are distinct programs. OTOH, if we exclude one from the system, we can't measure…

[also, reducing tedium]

Re: I'm Sure It Will Only Take You A Few Days To Code

#116
post #115
post #113

Earlier quoted context omitted.

The spurious abstraction issue was a caveat. The central problem is the hardest possible problem, IMHO; my suggestions are just what we can do about it. how can we minimize the overall complexity of a system? Nice clear statement. If we're minimizing complexity of an overall system that includes A and B, it doesn't matter whether A and B are distinct programs. OTOH, if we exclude one from the system, we can't measure…

[also, reducing tedium]

Yes, beer reduces tedium :)

I meant critical for building great software systems sustainably over time. That's also my purpose, if you expand "system" to include the team that's building it.

I agree with Alan Kay that the software industry has become a reductio of wildly inflated complexity and that it's possible to do orders of magnitude better. Presumably, anyone who figures out how to do that will be able to do things other companies can't. That is how the world gets changed. Not by persuading others to do it but by just doing it.

For a long time, smart people thought it was a matter of writing in more powerful languages, but it seems clear that it isn't that alone. It's not just the medium, but how we're using it. The 64 billion dollar question is: how can we use it better? And how will we know?

My hypothesis (not that I invented it, but I subscribe to it) is that we will know when we are able to build valuable systems with drastically less code. Obviously that requires knowing how much code there actually is. I'm fine with LOC as a metric (some HNers convinced me of that) but you still have to know what code you're measuring and what (e.g. libraries, language implementations, OS) you're not.

Interestingly, Kay's VPRI project doesn't have this conundrum because they're building everything up from hardware.

Re: I'm Sure It Will Only Take You A Few Days To Code

#117
post #115

Earlier quoted context omitted.

[also, reducing tedium]

Yes, beer reduces tedium :) I meant critical for building great software systems sustainably over time. That's also my purpose, if you expand "system" to include the team that's building it. I agree with Alan Kay that the software industry has become a reductio of wildly inflated complexity and that it's possible to do orders of magnitude better. Presumably, anyone who figures out how to do that will be able to do th…

Striking some chords, I've thought on these issues too.

software systems sustainably over time

Aside: the related idea of business sustainability is a business issue, not a technical issue (to do with customer needs and competitors both changing over time). Clayton Christensen sees high-tech as going through phases favouring performance, reliability, convenience then price over time - what wins one phase won't necessarily win the next. Warren Buffet sticks with low-tech (like a brick maker) because slow change enables advantages to be more sustainable over time.

reductio of wildly inflated complexity and that it's possible to do orders of magnitude better

Yes, their performance sucks, and also distorts the right way to do things (like the MS framework example). But does it depend on which "better" we mean? If better means "time to market", then the present stack is better... I respect much of what Alan Kay has said and done but note he's never had business success.

Not by persuading others to do it but by just doing it.

Yes, I think so too. :-)

It's not just the medium, but how we're using it.

I think it's specific abstractions, that suit the problems people are facing - and I think we're making steady progress. From arithmetic, function invocation, namespacing, way up to standard libraries (consider in sequence those of: C, C++, Java, Python) and now web-based APIs. Like scientific progress, it's not one single theory, but a great many specific facts and theories. Each a mysterious journey and heroic victory in itself.

BTW: There's that idea of using the analogue behaviour of transistors directly, instead of going through circuits, microcode, compilers and programs to simulate them... amazing performance, but hard to manage.

build valuable systems with drastically less code

I think libraries. e.g. we can make a valuable website very little code with sinatra (or PHP). That means we're there already :-). I know you're concerned with how to measure the length, and should it include libraries, but I think that if the abstraction suits your problem, it does not introduce unnecessary complexity into your code - and so I'm happy to consider it as part of the platform and ignore its actual cost. I'm not sure how to justify that, but it seems intuitively right to me (like not worrying how long code is in the implementation of arithmetic - I understand the IEEE fp spec is huge). Perhaps because that's the kind of library I'd write if I did the whole thing from scratch.

I think this is the nub of our impasse: I'm fixated on clean abstractions, and you're concerned about measuring program length. I think clean abstractions don't contaminate your code with complexity from another level - so you can pretend to start fresh in a new world. Ray Kurzweil suggests this happens with technological innovation, and even in biology e.g. tweaking DNA, cells, multicellular body plans, behaviour etc rather than inventing alternatives from scratch. Similar to SOTSOG.

A source of facts: ARM is a new stack compared to x86; and iOS is a new stack compared with desktop OS. Is there progress? They seem better (more efficient, smaller), but using the same fundamentals. Those Forth chips are totally different though.

BTW: do you have a link for the ideas about complexity from Kay at VPRI? I looked at their website, but couldn't see anything directly on it - and didn't want to download a bunch of pdfs to search for it. :-)

Post reply on HN