Live data from Hacker News

Drop millions of allocations by using a linked list

github.com

21–30 of 158 posts

Re: Drop millions of allocations by using a linked list

#21
post #6
post #3

Ruby people discovering algorithms ducks

Really. Every time I whine publicly how web programming people aren't familiar with even basic CS, I get a slap. But really, I should move to web programming. I'll be an expert computer scientist there, probably.

Slap. Stop whining.

Re: Drop millions of allocations by using a linked list

#23
post #17
post #6

Earlier quoted context omitted.

Really. Every time I whine publicly how web programming people aren't familiar with even basic CS, I get a slap. But really, I should move to web programming. I'll be an expert computer scientist there, probably.

Annnndddd.... what reaction do you expect? "Oh, please come and do web development so we can bask in the glow of your self-righteousness and infinite knowledge of computers." /snark (apologies for being offensive, but good lord, what a silly statement - unless I missed the joke) I stand before you as a counterpoint to your foolish generalisation, and guess what, I know plenty of other people that don't fit your stere…

The statement was a little strong, but in my experience it contains some truth. People who has never used a lower-level programming language can't often recognize these performance issues.

Re: Drop millions of allocations by using a linked list

#24
post #16

I'm not very familiar with Ruby; where exactly did the duplication occur in the original code?

I believe it's in the Gem::Specification.traverse method here: http://ruby-doc.org/stdlib-1.9.3/libdoc/rubygems/rdoc/Gem/Sp... The 'trail' parameter, an array, was implicitly duplicated by applying the '+' operator on each recursion through a dependency.

    trail = trail + [self]
That + operator looks so innocent, so seductively simple..

Re: Drop millions of allocations by using a linked list

#25
post #19
post #17

Earlier quoted context omitted.

Annnndddd.... what reaction do you expect? "Oh, please come and do web development so we can bask in the glow of your self-righteousness and infinite knowledge of computers." /snark (apologies for being offensive, but good lord, what a silly statement - unless I missed the joke) I stand before you as a counterpoint to your foolish generalisation, and guess what, I know plenty of other people that don't fit your stere…

I keep repeating myself on Hacker News, but once more, we've found the difference between Software Engineer and Computer Scientist. One makes things work, the other is a mathematician. Why do we keep conflating the two?

Because you cant be good at either without having an element of the other.

Ideally, they overlap.

Re: Drop millions of allocations by using a linked list

#26
post #3

Ruby people discovering algorithms ducks

Java had the same kind of issue for years.

In Java and Ruby, the standard library uses an object `+` operator to mean concatenation (not numeric addition), and the implementation did immediate data copies, rather than doing reference linking and copy-on-write (or immutability).

Re: Drop millions of allocations by using a linked list

#27

Earlier quoted context omitted.

"The First Rule of Program Optimization: Don't do it. The Second Rule of Program Optimization (for experts only!): Don't do it yet." — Michael A. Jackson

I prefer full quote from Knuth: "Programmers waste enormous amounts of time thinking about, or worrying about, the speed of noncritical parts of their programs, and these attempts at efficiency actually have a strong negative impact when debugging and maintenance are considered. We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass…

ok, now i have to rephrase my statement: 1. Make it work 2. Measure 3. Optimize

What people often miss is the measuring part.

Re: Drop millions of allocations by using a linked list

#28
post #17

Earlier quoted context omitted.

Annnndddd.... what reaction do you expect? "Oh, please come and do web development so we can bask in the glow of your self-righteousness and infinite knowledge of computers." /snark (apologies for being offensive, but good lord, what a silly statement - unless I missed the joke) I stand before you as a counterpoint to your foolish generalisation, and guess what, I know plenty of other people that don't fit your stere…

I completely agree that software development is such a vast field that it is completely impossible to be proficient in everything. I am kind of a lamer in web development, for example. But fundamental CS is a different thing. "Annnndddd.... what reaction do you expect?" Admitting your shortcomings and try to fix them is not an option at all? When some central web frameworks do lamest mistakes like making an O(n^2) qu…

Almost everyone does do lamest mistakes at some point, e.g.:

goto:fail - https://news.ycombinator.com/item?id=7281378

shellshock - https://news.ycombinator.com/item?id=8365110

I'm sure whoever wrote this would feel a little chagrin when coming back to their code and seeing how inefficient it was (though it got the job done when the lists were small), and they probably would admit their shortcomings, why wouldn't they?

Condescending snark is really easy, and it's easy to say in retrospect and with time to reflect that most useful code has flaws and point them out - software is never finished, and there are a lot of different levels of experience and requirements. If rubygems had never become popular, this wouldn't even be an issue.

PS Rubygems isn't a web framework, it's a package management tool, so the straw man you're hacking away at is the wrong one.

Re: Drop millions of allocations by using a linked list

#29

I'm torn on this one. It's a great performance improvement, but on the other hand I would expect this way sooner than after almost 4 years of usage.

It's because the main bottleneck for most apps that uses lots of gems is elsewhere (the load path grows with each extra gem, meaning a simple 'require' gets more and more expensive the more gems your app uses).

Re: Drop millions of allocations by using a linked list

#30
post #5

Earlier quoted context omitted.

1. Make it work 2. Optimize

From working with Ruby guys its normally 1. Make it work 2. Spin up more AWS boxes

Nothing's wrong with spinning up more AWS boxes. If it costs 300$ annually to solve a problem that would cost 5k$ in development to fix, I believe it's a wise choice.

Yeah, down the line you will eventually have to do optimization, but you will prioritize.

Post reply on HN