91–100 of 124 posts
This is a very inefficient implementation. Really, just poor quality work overall, as anyone with even a basic understanding of spatial indexing would know that an R-tree would be many times faster, as illustrated here: https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...
Earlier quoted context omitted.
> I mean there is a definite performance gain from Java to Go I don't think thats true.
The GC isn't very good yet, but brute-force sequential code can keep more state on the stack where it doesn't matter. (Just please codegen that instead of expecting everyone to read it.)
I think this write up is very fair for a solid engineering team. Is it groundbreaking and eye opening? Absolutely not, I’d say the most “hmm I didn’t know that” part of the entire thing was the part about R-Trees and S2. Is that bad? Absolutely not. These guys did the work, logged their performance and are sharing their story. However, and I believe this is where the animosity in the comments is coming from - given t…
> I guess Leetcoding and Production Engineering really are different things. This is key point. They delivered value to the business. That's the only thing that matters.
Although hardware is relatively cheap, 170 QPS on 40 servers for this type of query is astonishingly horrible even from a Business/Product Engineering standpoint.
Boasting this as "highest QPS* engineering achievement is just awkward. It may be better suited for an article on how throwing hardware at problems is cheaper than hiring engineers.
Earlier quoted context omitted.
I'm not sure what causes it—and I'm not even sure it's a bad thing, certainly it must not be hurting these folks careers too much, and maybe it's even helping them—but there's a certain path for developers that ends up leaving them helpless outside whatever narrow ecosystem they've grabbed onto. Usually it's Java or some Microsoft thing. As someone who very much did not develop (as a person/programmer) that way, it s…
Well, I think the difference between them and Silicon Valley programmers is that they don't constantly learn new technologies outside of work. It's just a 40-hour-a-week job. I try to learn new things, but I don't do much programming outside of work. I could learn Node.js, for instance... but I've got other things I want to do. Is that kind of what you mean?
Some folks, you say "do you think you can do [thing you're basically familiar with] in [language and platform you're not]?" and get a "yeah, probably lemme check it out... cool, compiler and language support's installed, see a couple tutorials here, I'll poke around and get something doing [subset of thing you need] then get back to you on timeline" and very likely it works out fine.
Others, you get "uh, I do (Java, .net), I don't... I don't understand what this is, is it a JVM language? Is there a jar for it?" and it's not really worth pushing any harder. And maybe some of them are deflecting because they just can't be bothered (mad respect), but most seem genuinely nervous and out-of-their-element at the mere suggestion of doing anything but their Java or .net thing (usually it's one of those) they're used to.
Though, again, the latter seem to get along just fine, career-wise. It's just a different sort of path, I guess. Seems really weird to me, but there it is.
This is a very inefficient implementation. Really, just poor quality work overall, as anyone with even a basic understanding of spatial indexing would know that an R-tree would be many times faster, as illustrated here: https://medium.com/@buckhx/unwinding-uber-s-most-efficient-s...
-----
Please omit swipes like "Really, just poor quality work" and "anyone with even a basic understanding" from your posts to HN. It's great to add relevant information, such as an applicable data structure and a link to a good article on the same topic. But it's not great to put others and their work down, and HN has at least two guidelines that ask you not to:
When disagreeing, please reply to the argument instead of calling names. "That is idiotic; 1 + 1 is 2, not 3" can be shortened to "1 + 1 is 2, not 3."
Please don't post shallow dismissals, especially of other people's work. A good critical comment teaches us something.
Actually, a third guideline is relevant too:
Please respond to the strongest plausible interpretation of what someone says, not a weaker one that's easier to criticize.
The strongest plausible interpretation is not that the engineers lacked a basic understanding of relevant CS—or, for that matter, how to use a search engine, since R-trees would pop up nearly any place you searched about this stuff. The strongest plausible interpretation is that they had some other reason for choosing the implementation they did. For example, perhaps it was efficient enough and they were smartly choosing not to build something more complicated than needed.
https://news.ycombinator.com/newsguidelines.html
Edit: and it turns out that the article explains why they didn't use R-trees.
Earlier quoted context omitted.
The GC isn't very good yet, but brute-force sequential code can keep more state on the stack where it doesn't matter. (Just please codegen that instead of expecting everyone to read it.)
Go's GC is better than most of the Java GCs except the new ZGC and Azul's proprietary one. JIT makes some Java code faster. Ultimately, it's more about programmer expertise in a language rather than the actual language.
That said, Go's focus may be the right one for many use cases. On a high throughput service where you would assume you want to optimize for throughput, 100 ms pause times can wreak havoc because they're unpredictable and can cause work queues to explode and such. This isn't easily mitigated by load balancing. Whereas "less efficient" GC is at least predictable and you can just add a server to balance that extra work.
Earlier quoted context omitted.
> I mean there is a definite performance gain from Java to Go I don't think thats true.
There are dramatic memory and startup time improvements. Much less so for CPU once everything is up and running.
Earlier quoted context omitted.
My sense is that most people moving to Go are coming from Node and that explains their excitement about the performance. Not many upsides when you’re already on Java.
Why is it that every Java application I've encountered in my 20 years of corporate IT, has been a performance pig? Is it just a case of bad developers, making shit code? Is it difficult to make Java perform well, but when it does, it shines? I can't have accidentally interacted with ONLY the crappiest Java apps in my tenure.
But enterprise programmers making internal software have little incentive to make their programs sleek and fast because they 1) have a captive user base who are forced to use their bad UIs, and 2) there aren't many users so you don't really need to optimize for minimal server use.
From the eyes of management, a good enterprise programmer can take a request from start to beginning and fulfill a business requirement in a reasonable amount of time. Architecture, UI, future maintainability, speed, security... that's not really on their radar at all.
There are plenty of managers who do not understand or respect feature creep and technical debt, so spending time refactoring is useless and opaque to them. Developers have no incentive to write good code, and bad developers who can talk a good game get hired on, so the codebase spirals into an unmaintainable sea of crap.
According to management, if you solve business problems, you're valuable to them. So if you care about good code, it's an uphill battle fighting for it against the other entrenched developers who really don't give a shit.
These programs usually happen to be written in Java because there are tons of Java developers out there, which is great if you don't live in a tech hub.
Earlier quoted context omitted.
clearly the most efficient solution is to pontificate about other people's implementations while never having done anything of the sort at that scale yourself.
For future internet-searchers that stumble upon this, it might be helpful to point out that better solutions exist..
Earlier quoted context omitted.
> I guess Leetcoding and Production Engineering really are different things. This is key point. They delivered value to the business. That's the only thing that matters.
> They delivered value to the business. That's the only thing that matters. Although hardware is relatively cheap, 170 QPS on 40 servers for this type of query is astonishingly horrible even from a Business/Product Engineering standpoint. Boasting this as "highest QPS* engineering achievement is just awkward. It may be better suited for an article on how throwing hardware at problems is cheaper than hiring engineers.