Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You could have invented the LMAX Disruptor, if only you were limited enough

hmijailblog.blogspot.com

Post reply on HN
Print view View original

21–30 of 30 posts

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#21

Post by hinkley »

lmilcin wrote: post #15
jandrese wrote: post #2

What this says to me is that Java's reputation for sluggishness is a result of its idiomatic coding styles more than the language itself. In particular the casual way people allocate and discard objects for everything they do. If you program it like it was C then you can get good performance, which make sense given that the language was built for embedded devices with anemic processors. Of course you undoubtedly give…

Hi! I have worked on algorithmic trading framework for third largest Polish brokerage house and I also re-implemented LMAX Disruptor for this project. Actually, allocating and discarding objects is not the problem. Java allocation and deallocation is extremely efficient for short living objects. I would say memory layout, access patters resulting from object orientedness are much more important factor. It is possible…

Information architecture always matters.

But in Java it always seemed that you are very much punished for having data structures that are at cross purposes to your dominant work load. I cut my teeth on implementing the last two parts of “make it work, make it right, make it fast”, often on projects where the existing team had declared that everything that could be done already had been done. There are a lot of refactorings that accomplish both goals, and I often got a 2-3x out of these projects by removing slow tech debt, and more by exposing a real info architecture.

It always surprised me that a language that so punished (especially in the early days where it was interpreted and all object lookups were double indirect) the Big Ball of Mud antipattern exhibited so many examples of it, so frequently.

We can do better. So much better.

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#22

Post by lmilcin »

hinkley wrote: post #21
lmilcin wrote: post #15

Earlier quoted context omitted.

Hi! I have worked on algorithmic trading framework for third largest Polish brokerage house and I also re-implemented LMAX Disruptor for this project. Actually, allocating and discarding objects is not the problem. Java allocation and deallocation is extremely efficient for short living objects. I would say memory layout, access patters resulting from object orientedness are much more important factor. It is possible…

Information architecture always matters. But in Java it always seemed that you are very much punished for having data structures that are at cross purposes to your dominant work load. I cut my teeth on implementing the last two parts of “make it work, make it right, make it fast”, often on projects where the existing team had declared that everything that could be done already had been done. There are a lot of refact…

On one of the projects I had trouble convincing the company to stay on Java for their application when they were displeased with the performance.

The previous developers were just careless/clueless about performance and when it started becoming a problem they cited good patterns they were following and blamed Java for their problems. The tech lead wanted to rewrite it in C++ which would be suicide IMO.

What I did, I created a graph in a form of horizontal bar which showed color-coded parts of transaction processing. The color codes showed different parts: business logic, frameworks, infrastructure, communication, etc. I then marked a different graph showing which parts of this was unnecessary with some notes of how this can be optimized. You guess, the parts that were not easily optimized away were very hard to find.

In the end we stayed on Java achieving almost two orders of magnitude performance improvement.

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#23

Post by blt »

jandrese wrote: post #2

What this says to me is that Java's reputation for sluggishness is a result of its idiomatic coding styles more than the language itself. In particular the casual way people allocate and discard objects for everything they do. If you program it like it was C then you can get good performance, which make sense given that the language was built for embedded devices with anemic processors. Of course you undoubtedly give…

The lack of value semantics for composite types is a huge problem in the language itself. It is often hard to get a Java program to have the same memory layout you would get writing idiomatic C/C++.

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#24

Post by hmijail »

jandrese wrote: post #2

What this says to me is that Java's reputation for sluggishness is a result of its idiomatic coding styles more than the language itself. In particular the casual way people allocate and discard objects for everything they do. If you program it like it was C then you can get good performance, which make sense given that the language was built for embedded devices with anemic processors. Of course you undoubtedly give…

(Author here)

"If you program it like it was C then you can get good performance" -> Note that one of the points of the post is that just using "like C" won't magically fix your performance!

If anything, you could say "If you program it like a system with limited memory then you can get good performance". That, I can imagine being a universally applicable thing.

My point is that, for example for the particular case of memory management, there is an associated cost, whatever your language is; and you need to deal with it. Ignorance of the law is no excuse, etc.

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#25

Post by hmijail »

wmfiv wrote: post #4
jandrese wrote: post #2

What this says to me is that Java's reputation for sluggishness is a result of its idiomatic coding styles more than the language itself. In particular the casual way people allocate and discard objects for everything they do. If you program it like it was C then you can get good performance, which make sense given that the language was built for embedded devices with anemic processors. Of course you undoubtedly give…

Java's reputation for sluggishness was really formed based on people's experience with Applets and Swing applications and especially badly written Applets and Swing applications. Java in the real world is fast. That's why it's used as the backbone of so many large organizations and so many scale out solutions (Cassandra, Kafka, Hadoop, etc) are written in Java.

(Author here)

But that's not "Java in the real world". If anything I'd presume that that's "well programmed Java".

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#26

Post by hmijail »

PaulHoule wrote: post #14
jandrese wrote: post #2

What this says to me is that Java's reputation for sluggishness is a result of its idiomatic coding styles more than the language itself. In particular the casual way people allocate and discard objects for everything they do. If you program it like it was C then you can get good performance, which make sense given that the language was built for embedded devices with anemic processors. Of course you undoubtedly give…

C was built for a minicomputer which might or might not be the brains of a phone switch, a software development environment, a word processing cluster for a newspaper, industrial control for a steel mill, etc. The thing about Java is that it has promised and delivered a much better threading experience than other programming languages. When somebody proved the Java memory model was unsafe, Sun fixed it. C and other l…

(Author here)

Actually, I think that talking about Java's multithreading here is a red herring; note that LMAX had to abandon multi-threading because it was destroying performance.

And this is important, because they explicitly made a very interesting point of current architectures being at odds with current thinking about best practices for concurrent programming.

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#27

Post by hmijail »

scarejunba wrote: post #20
draw_down wrote: post #12

This is the kind of thing I really love reading. It’s ostensibly about a technical subject, but that’s really a jumping-off point for examining how we think and talk about things, how portions of our field become Balkanized, how what we encounter shapes our sense of what’s possible and ultimately hardens into “common knowledge”.

Reminded me of the reaction to this paper in Medicine https://doi.org/10.2337/diacare.17.2.152 It's just rudimentary integration approximation to a mathematics undergraduate but to those in Medicine unfamiliar with it, it was an achievement.

That sounds really interesting. Can you point to any of those reactions?

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#28

Post by mindslight »

wmfiv wrote: post #4
jandrese wrote: post #2

What this says to me is that Java's reputation for sluggishness is a result of its idiomatic coding styles more than the language itself. In particular the casual way people allocate and discard objects for everything they do. If you program it like it was C then you can get good performance, which make sense given that the language was built for embedded devices with anemic processors. Of course you undoubtedly give…

Java's reputation for sluggishness was really formed based on people's experience with Applets and Swing applications and especially badly written Applets and Swing applications. Java in the real world is fast. That's why it's used as the backbone of so many large organizations and so many scale out solutions (Cassandra, Kafka, Hadoop, etc) are written in Java.

> Java's reputation for sluggishness was really formed based on people's experience with Applets and Swing applications

... and contemporary interactive performance.

    python
There's of course a theoretical point about being shoehorned into the UNIX execution model, and if Java were able to run as a persistent OS (eg nailgun or whatever it is these days) then things get much better. But still, when you start off a project having to suffer and design workarounds for the language's and implementation's flaws...

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#29

Post by girvo »

scarejunba wrote: post #20
draw_down wrote: post #12

This is the kind of thing I really love reading. It’s ostensibly about a technical subject, but that’s really a jumping-off point for examining how we think and talk about things, how portions of our field become Balkanized, how what we encounter shapes our sense of what’s possible and ultimately hardens into “common knowledge”.

Reminded me of the reaction to this paper in Medicine https://doi.org/10.2337/diacare.17.2.152 It's just rudimentary integration approximation to a mathematics undergraduate but to those in Medicine unfamiliar with it, it was an achievement.

I learned that integration approximation method in high-school. I’d love to see the responses too!

Re: You could have invented the LMAX Disruptor, if only you were limited enough

#30

Post by scarejunba »

hmijail wrote: post #27
scarejunba wrote: post #20

Earlier quoted context omitted.

Reminded me of the reaction to this paper in Medicine https://doi.org/10.2337/diacare.17.2.152 It's just rudimentary integration approximation to a mathematics undergraduate but to those in Medicine unfamiliar with it, it was an achievement.

That sounds really interesting. Can you point to any of those reactions?

When it was discovered by the Mathematics community it was met with bemusement. Sorry, I'm recalling from memory here from years ago when I was a graduate student in Mathematics and don't have ready posts.

Meanwhile, it accumulated 300 citations in medicine and chem https://scholar.google.com/scholar?hl=en&as_sdt=0%2C4&q=A+Ma...

Post reply on HN