Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Drop millions of allocations by using a linked list

github.com

Post reply on HN
Print view View original

51–60 of 158 posts

Re: Drop millions of allocations by using a linked list

#51

Post by rfrey »

I'm really surprised by the amount of smugness in the comments here. A bit of good-natured teasing, followed by a wheelbarrow full of "ruby-devs" this and "web-devs" that.

Take off your Hats of Superior Coding. Any one of us, regardless of honorific titles, could have made this mistake, and you know it. Being steeped in CS Fundamentals does not immunize you against bugs.

Congratulations to tenderlove for finding the bug. Remember the details - it'll be a great war story in a few years.

Re: Drop millions of allocations by using a linked list

#52

Post by richthegeek »

mbrock wrote: post #44

Semi-related: does anyone know why installing gems is so ridiculously slow? What is the thing doing? Downloading tarballs, yes, but then? It's a dynamic language; there is no compilation or verification! Why can I install Ruby packages using apt almost immediately, when gem/bundle install takes half a coffee break? I'm growing more impatient with the years. I have measured out my life with slow software. We talk abou…

Not that I install gems very often (Node.JS is my primary platform) but I find that the majority of the time is installing the different docs.

Check the difference between "gem install sass" and "gem install sass --no-rdoc --no-ri" and be amazed.

Re: Drop millions of allocations by using a linked list

#53

Post by pothibo »

rosspanda wrote: post #33
pothibo wrote: post #30

Earlier quoted context omitted.

Nothing's wrong with spinning up more AWS boxes. If it costs 300$ annually to solve a problem that would cost 5k$ in development to fix, I believe it's a wise choice. Yeah, down the line you will eventually have to do optimization, but you will prioritize.

A agree somewhat, but I've seen 10 box systems that could run on raspberry pie with good code

I wasn't advocating for badly written code to run on a whole datacenter. I was just pointing the alternative with the assumption that the code was somewhat healthy and adding one new instance to cover the sub-optimization wasn't a big deal.

Of course, if you have 10k users and it runs on 3 machines, you got a problem which no amount of boxes can solve.

Re: Drop millions of allocations by using a linked list

#54

Post by kaeluka »

rakoo wrote: post #39
kaeluka wrote: post #34

Earlier quoted context omitted.

AFAICT, this performance bug is not at all related to the linked-list vs. vector issue.

Yes it does: vectors are good for random access, linked-lists are good for doing stuff in the front/back of the list. The performance bug we have here is solved by finding a way to insert stuff at the front/back (and also going through each item in the list); there is no need for random access.

If I remember Bjarne's talk correctly, vectors (in C++) are even fast at inserting because they have densely packed representation which rhymes well with modern computer architecture. Inserting in a linked list is slow, as walking the list to find the element at which to insert will already incur O(N) cache misses, whereas in vectors it's only O(1) cache misses. Moving the elements in the vector one to the right is fast due to computer architecture dealing well with predictable patterns.

The allocations here (ruby) are reduced because the implementation of appending is horribly slow in the first place, using defensive cloning (I'm taking jph's word here).

Re: Drop millions of allocations by using a linked list

#56

Post by ing33k »

mbrock wrote: post #44

Semi-related: does anyone know why installing gems is so ridiculously slow? What is the thing doing? Downloading tarballs, yes, but then? It's a dynamic language; there is no compilation or verification! Why can I install Ruby packages using apt almost immediately, when gem/bundle install takes half a coffee break? I'm growing more impatient with the years. I have measured out my life with slow software. We talk abou…

imo resolving dependencies is one of the factor

Re: Drop millions of allocations by using a linked list

#57

Post by knowtheory »

perdunov wrote: post #49
grey-area wrote: post #42

Earlier quoted context omitted.

You think the original authors (Chad Fowler, Rich Kilmer, Jim Weirich) were ignorant of linked lists? What hubris. The story is about rubygems, not web dev.

I do? I was saying that being ignorant in CS may lead to lame mistakes, and I have seen many web developers who appeared ignorant in CS, and there has been a case of some central web framework with an O(n^2) queue.

Yes there are many web developer who rely on rubygems, but it was written and continues to be maintained today by people who are (not perfect) but good developers who like Aaron, find shit like this and fix it.

If you actually cared, you'd be out there using your vaunted skills to improve the software other people use, instead of slagging off other developers on Hacker News.

Re: Drop millions of allocations by using a linked list

#58

Post by icebraining »

pointernil wrote: post #38
pothibo wrote: post #30

Earlier quoted context omitted.

Nothing's wrong with spinning up more AWS boxes. If it costs 300$ annually to solve a problem that would cost 5k$ in development to fix, I believe it's a wise choice. Yeah, down the line you will eventually have to do optimization, but you will prioritize.

In a certain, quite limited model of economics that could actually be named as "wise". Once a more holistic view is taken, wide spread total costs and benefits are taken into consideration, once costs are not only defined as money flowing out of my own pocket, once not only "Gesinnungsethik" but also and more importantly "Verantwortungsethik" gets applied, well, in such a world we would probably wish, that Amazon wou…

What social changes would you expect from the pricing policy changes of marginal EC2 instances?

Re: Drop millions of allocations by using a linked list

#59

Post by bcg1 »

I'm not a ruby dev, so I guess maybe my perspective is not that great on this particular issue... but hats off to the dev with the fix, indeed this is how free software collaboration is supposed to work in my opinion.

Even the dev with the fix wasn't rude about the original problem, he seemed pretty humble about it actually.

If you think the Ruby guys are such shitty programmers you should be able to dive into their codebases and find the plethora of problems to show them what's up... so either give them a pull request or STFU ;)

Re: Drop millions of allocations by using a linked list

#60

Post by lucozade »

Swizec wrote: post #19
aidos wrote: post #17

Earlier quoted context omitted.

Annnndddd.... what reaction do you expect? "Oh, please come and do web development so we can bask in the glow of your self-righteousness and infinite knowledge of computers." /snark (apologies for being offensive, but good lord, what a silly statement - unless I missed the joke) I stand before you as a counterpoint to your foolish generalisation, and guess what, I know plenty of other people that don't fit your stere…

I keep repeating myself on Hacker News, but once more, we've found the difference between Software Engineer and Computer Scientist. One makes things work, the other is a mathematician. Why do we keep conflating the two?

I'd argue that it's really a distinction between an engineer, who should understand this stuff, and a technician, who doesn't need to to do a job.

Certain segments of our industry are currently engineer heavy, such as embedded, and some appear to be technician heavy. I don't see that as a problem per se but it clearly causes friction occasionally as we tend to conflate them.

Post reply on HN