Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Suffering-oriented programming

nathanmarz.com

Post reply on HN
Print view View original

31–40 of 64 posts

Re: Suffering-oriented programming

#31

Post by nathanmarz »

3pt14159 wrote: post #28

"With Storm, I distilled the realtime computation problem domain into a small set of abstractions: streams, spouts, bolts, and topologies. I devised a new algorithm for guaranteeing data processing that eliminated the need for intermediate message brokers, the part of our system that caused the most complexity and suffering." As someone who has gone through the storm source in very fine detail, let me tell you how he…

Not quite. Every edge in the dependency tree is assigned a random 64-bit id, and when a tuple is acked it sends the xor of all the incoming and outgoing edges to the acker.

Re: Suffering-oriented programming

#32

Post by Deestan »

scott_s wrote: post #25
Deestan wrote: post #10

Earlier quoted context omitted.

> put your code on Github or the like and others will add any features that you're missing. No they won't, because he hadn't even started on the "make it beautiful". When I want to solve a new-ish problem, I can't imagine grabbing some barely working cowdung from some guy's github repo. If he hasn't even tried to make it clean or readable, it'll take me more time to make sense of the mess than just rebuild it myself.

It depends on just how useful, and how ugly, it is. See: http://dreamsongs.com/WorseIsBetter.html

WiB is more about appreciating barebone low-level tools, like C/Unix in the posted example. These tools are crude but technically polished, otherwise no-one would bother to use them.

Re: Suffering-oriented programming

#33

Post by nathanmarz »

ced wrote: post #30
3pt14159 wrote: post #28

"With Storm, I distilled the realtime computation problem domain into a small set of abstractions: streams, spouts, bolts, and topologies. I devised a new algorithm for guaranteeing data processing that eliminated the need for intermediate message brokers, the part of our system that caused the most complexity and suffering." As someone who has gone through the storm source in very fine detail, let me tell you how he…

Isn't it problematic that 11, 01 and 10 XOR together to get 00? And why not use a simple counter?

Random 64 bit ids are used in the process. So the probability of accidentally completing a tuple is very, very small (1 / 2^64 for every ack).

Counters don't work because of the asynchronous nature of Storm. For example, consider a topology that looks like this:

A -> B -> C

      \-> D
Let's say A emits 2 tuples (+2 differential), B processed those and emits 2 to C and 3 to D (+3 differential), C processes 2 tuples (-2 differential), and D processes 3 tuples (-3 differential).

Everything's asynchronous, so the acker could receive the acks in this order: A, C, B, D

The counter would then look like this: 2, 0, 3, 0

So it would think that the tuple was complete before it actually was, which means the counter algorithm doesn't work.

Re: Suffering-oriented programming

#34

Post by duggan »

A few other commenters have suggested that they don't get a chance to get past "First, make it possible." to making things beautiful, then fast. The person writing the checks doesn't care if the implementation is horrible, as long as it works, etc.

Thing is, both points of view are correct, in different ways.

The kind of long development cycle Nathan alludes to here is not something a lot of people can do, but is one of the risks you can take / benefits you gain if you're your own boss. I wonder if it's more common in larger companies, who can afford to polish some projects before putting them into production? With a startup, it's more likely that there's one or two projects, and both are P1-CRITICAL.

For people wanting to explain it to a boss, I think it's best put in terms of technical debt[1]. This project will be expensive, but by spending more time/resources up front, there is a much lower cost to adding features or fixing bugs in the future; this includes getting new coders up to speed on the codebase, turnaround of critical features, etc. The cost is initial time to market - it will be longer before you see version 1 going out to people who can use it.

Taking on technical debt is a perfectly valid business decision under the right circumstances. If time-to-market is critical and you're planning on having enough money when it's successful to pay off that debt in the future, then maybe you want to throw out making it beautiful / performant just to get something in front of people.

Planned technical debt is a business decision. It's the folks who buy up front and ignore their debt that end up with problems, but that much is something we're all familiar with. One way or the other, that debt is paid eventually, whether with cash or time.

Even if you're in the unfortunate position of turning out software that sucks for people who don't care though, there's light at the end of the tunnel! The best thing about programming is that the more you do it the better you get at it. The faster you get at it. This means you can build refactoring into your projects and estimations without making a big hoo-hah (as it were) about it.

Open source projects are another outlet people have to scratch their perfectionist itch; if you start a project, you decide the timelines, features, level of polish, etc.

[1] http://www.codinghorror.com/blog/2009/02/paying-down-your-te...

Re: Suffering-oriented programming

#35

Post by jt2190 »

Suffering-oriented programming rejects that you can effectively anticipate needs you don't currently have.

I do anticipate, however, I never do it in code. Rather, I find that just thinking through some scenarios helps me to see where the code might start evolving, and to make sure that that part of the code is isolated enough to change without having to change everything else, if the time ever comes.

(Perhaps Nathan includes this in his "make it beautiful" phase, but I felt it deserved to be made explicit. I've too often heard YAGNI used as an excuse to not even think about a problem, let alone code for it.)

Re: Suffering-oriented programming

#36

Post by reinhardt »

Well written, I especially loved the definition: Suffering-oriented programming can be summarized like so: don't build technology unless you feel the pain of not having it.

Having said that, it would be cool to hear from a "devil's advocate". I am particularly thinking of cases where "making it possible" gives an 80% solution but a fundamental flaw or limitation makes the remaining 20% prohibitive, basically requiring a complete rewrite to move forward. Is this a problem in practice or are 80% solutions usually "good enough"?

Re: Suffering-oriented programming

#37

Post by scott_s »

Deestan wrote: post #32
scott_s wrote: post #25

Earlier quoted context omitted.

It depends on just how useful, and how ugly, it is. See: http://dreamsongs.com/WorseIsBetter.html

WiB is more about appreciating barebone low-level tools, like C/Unix in the posted example. These tools are crude but technically polished , otherwise no-one would bother to use them.

It depends on what you mean by "technically polished." Unix handles interrupted signal calls by returning an error code that means "I was interrupted." This technique bunts on the hard problem of rolling back OS operations. It is not "technically polished" in that it does not solve all of the hard problems in front of them. But it's still useful, and it caught on because people used it.

Gabriel's insight is not about low-level tools. It's about at what point can you bunt on the hard problems, have an ugly work-around, but still be useful enough that no one will use "the right thing" when it eventually comes about?

Re: Suffering-oriented programming

#38

Post by shadowfiend »

alttag wrote: post #23
shadowfiend wrote: post #3

This is really an extension of the advice to make sure you build a product that scratches an itch you have. Extend it to developing frameworks, and you have “suffering-oriented programming”. Of course, the real problem is that there are certain spaces that wouldn't really be serviced if that's all we did. Education is an excellent example: the people who really feel the pains of education, students, won't really star…

I'm not sure I (fully) agree with your example of education. In education the scratchable "itch" is felt by educators and parents in their offering of a service (education). The extent to which we as a society expend resources providing tools/resources for educators and allowing them to build/purchase solutions demonstrates the extent of (or lack of) our societal valuation of education. The itch in this is scratched…

The trouble is that the scratchable itches educators and parents feel are not the same ones that the students feel. You need to be able to address all three itches in many educational products, but it's fairly difficult to be in all three roles to gain the understanding you need.

Re: Suffering-oriented programming

#40

Post by losethos »

C:\LoseThos\www.losethos.com\text\HUCKFINN.TXT

me and shook hands with the king and thanked him and talked to him; and then they shook hands with the duke and didn't say nothing, but just kept a-smiling and bobbing their heads like a passel of sapheads whilst he made all sorts of signs with his hands and said "Goo-goo--goo-goo-goo" all the time, like a baby that can't talk.

So the king he blattered along, and managed to inquire about pretty much everybody and dog in town, by his name, and mentioned all sorts of little things that happened one

Post reply on HN