Live data from Hacker News

Every Byte Matters

fzakaria.com

121–130 of 159 posts

Re: Every Byte Matters

#121

Earlier quoted context omitted.

I think all operating systems have these. In this one case, allocating a MapView via storyboard, caused some kind of cascading strong reference stuff. Simply allocating it programmatically, fixed it. Took awhile to get there, though.

Can you elaborate? Did you turn SIP off or what did you do?

No. No need. Just spent a lot of time, logging my code, and eliminating every possible leak.

Tedious, but there really wasn’t anything else I could do. Finding out about the programmatic solution was really just a wild guess.

Re: Every Byte Matters

#122
post #57
post #51

Anyways find it odd that major languages don’t have a built in way of asking for an array of objects to be optimized as SoA or AoS

It doesn't quite make sense to keep object identity at the language level. Inherently the data in the arrays cannot be the same memory of the data in the objects fields. To get the speed up, you can't just abstract it as an access pattern because it's tied to the specific way the memory is laid out. If you were trying to make some kind of collection type that could be queried by both row and column, you would need to…

I was thinking about just asking the compiler to do the expensive reshuffle when I need it to, but you could go further and expect the compiler to figure out the likely access pattern and spend the transformation budget.

Heck memory is cheap (fine was cheap) give me a data structure that amortizes writes cleverly by maintaining both SoA and AoS at the same time

Re: Every Byte Matters

#123
post #57

Earlier quoted context omitted.

It doesn't quite make sense to keep object identity at the language level. Inherently the data in the arrays cannot be the same memory of the data in the objects fields. To get the speed up, you can't just abstract it as an access pattern because it's tied to the specific way the memory is laid out. If you were trying to make some kind of collection type that could be queried by both row and column, you would need to…

Database structure in a programming language! I think I've heard this idea before but never seen it implemented. Define a table with rows and columns, give it some implementation hints and query it like "SELECT id FROM monsters WHERE alive=false" and the compiler could translate it several completely different ways depending on your layout.

Yes! Exactly

Re: Every Byte Matters

#124
post #77
post #70

Earlier quoted context omitted.

Presumably they're referring to resizing the arrays.

Array resizing is avoidable with an embedded free list if ordering is of no concern.

If you take out ordering, then lookups on your SoA are now a search, and n-field lookup on an entity is now a JOIN operation.

The smarter you get about it, the closer you get to an OLAP db

Which leads to my theory… I feel like Bevy could be implemented on top of an in-memory DuckDB and get away with it

Re: Every Byte Matters

#125
post #39

Earlier quoted context omitted.

> Do you have concrete examples of large scale Java programs that are significantly more performant than comparable programs in native languages like C++? Yes. I was working in a place that made large sensor-fusion applications, air-traffic control applications, and logistical planning, each in the 2-8MLOC range. Over time, we ported all of them from C++ to Java because C++'s performance overheads were too annoying t…

”they ported a small-to-medium service from Java to Rust. The result was such a huge performance drop that it wouldn't meet their minimum requirements” That result would say less about performance of languages than it would about competency of developers with a language. I just don’t buy that a task could be assigned to two teams with comparable expertise and domain knowledge in Rust and Java, and have the Rust resul…

> That result would say less about performance of languages than it would about competency of developers with a language.

> B-b-but skill issue!

That's one of the dimensions of the language too. Not only raw performance matters.

Re: Every Byte Matters

#126
post #71
post #64

Earlier quoted context omitted.

We compiled one of our Java app to native binary using GraalVM (for encyption and secret managment needs). Side effect is the Java native binary performance is excellent, app startup time also significantly less compared to JVM version. I am not sure how it compares with C++, Rust and Zig, but we made a benchmark with a similar Go binary, Java native version performance (load tests) is similar to Go binary. Only RAM…

The RAM difference is primarily because both Native Image (what you call Graal VM) and Go use much simpler and less efficient memory management techniques. HotSpot uses much more RAM by design as there are inefficiencies caused by using too little of it. Memory management - and especially very sophisticated approaches that are only used by the best resourced teams - is an especially misunderstood aspect. I gave a tal…

> This is because taking up 100% of the CPU effectively captures 100% of RAM

Isn’t that only true though specifically at 100% CPU utilization?

If it were at 90% CPU, then you have no RAM capture, and then you can’t say anything about whether 80 or 800MB should be taken; it’s only a freebie if and only if literally no other program can do work on the machine.

I don’t see how you can map X% CPU utilization to Y% RAM capture.

Like a program could be network heavy, CPU light and mmaps a large file? Or streaming a file from disk with a constant memory allocation, but doing heavy nonstop CPU work.

The CPU / RAM capture ratio would be wildly different; the ideal for your program, while other competing programs of unknown behaviors exist, I don’t see any way for hotspot to approximate

Re: Every Byte Matters

#127
post #10

Earlier quoted context omitted.

Most of real world use of Java platform has next to 0 concerns like those. Some more niche use case may benefit, good, but overall success map isn't changing anytime soon. Reasons for its long term success lie elsewhere.

Android Java apps' memory consumption is definitely a relevant concern.

It uses ART, which is not a Java platform.

Re: Every Byte Matters

#128
post #100

Earlier quoted context omitted.

I’ve done performance-engineering for decades in Java, C++, and C for both data analytics and supercomputing/HPC. Java performs significantly worse than C++ in all cases without exception. This is the result you should expect from first principles; something has gone horribly wrong with your software optimization if Java is faster than C++ or even Rust. There are good reasons to use Java in environments that care abo…

> I’ve done performance-engineering for decades in Java, C++, and C for both data analytics and supercomputing/HPC. Java performs significantly worse than C++ in all cases without exception. I've done similar work (not supercomputing/HPC, but yes for soft and hard realtime software, including safety-critical software) and I couldn't disagree more. Of course, we didn't get to write every program in both Java and C++,…

I'm not sure I understand what exactly you're talking about. I personally moved away from Java to Rust, because of the obvious and immediate performance benefits and this is possible because Rust manages to stay safe despite the lack of a garbage collector.

Re: Every Byte Matters

#129
post #119

Earlier quoted context omitted.

If what you are saying is correct, the performance of Java has to be the best-kept secret in the industry. Because you are the only person I've ever heard making such claims seriously. But this looks more like an apples-to-oranges comparison. You might be talking more about performance in complex business logic, while others are talking about performance in computation. I can imagine that Java could be faster than C+…

> the performance of Java has to be the best-kept secret in the industry Is it, though? It's the first language of choice for a large number, if not most performance-critical applications. > Because you are the only person I've ever heard making such claims seriously. Your sources must be very limited, then, because in serious compiler and runtime design and memory management circles this is quite common. There is a…

>I wouldn't say HPC and video game engines are "traditional performance critical work". Not because they're not performance critical, but because the range of performance critical programs is far larger - think bank card transaction processing; think mobile phone routing, and there are many more examples (also, AAA video game engines are indeed very traditional in their design and tech choices, but their performance-sensitivity these days is not so much around CPU-related optimisations but about scheduling the GPU, and their tech choices are much more constrained by the consoles they need to support than by performance).

In "business oriented" contexts, the usual culprits are database access and serialization/communication overheads. If you use Rust with serdes, you get access to one of the fastest ways to turn JSON documents into struct accessible data on the entire planet. The same implementation effort could be spent on any industry specific data formats.

I am struggling to think of any scenarios where Rust is supposed to be uniquely unsuited and Java would have an obvious win to make the broad and sweeping statements you've made.

If everything you said is true, people would be building JVM backends for C++/Rust the same way LLVM has been used as a backend and there would be constant discussions about JVM vs clang vs gcc. It just doesn't add up.

Re: Every Byte Matters

#130
post #104
post #99

Earlier quoted context omitted.

This sounds similar to relational databases vs document oriented databases, at least when I briefly looked into database like MongoDB when such things were all the rage 15-20 years ago. For the internal web site that customer support people used a document oriented database would be great because that wants to load everything about one customer and pretty much doesn't need anything else until the user is done support…

The Array-of-Struct vs Struct-vs-Array organization is actually more similar to row-major ordering vs column-major ordering, i.e. the data structure that analysis databases use to optimize for aggregate calculations. Document databases are not really comparable because they don't impose structure on the data; with document databases you just have a tree of JSON elements, which is neither AoS nor SoA.

Or, another name for the same thing columnar: storage vs. row storage.
Post reply on HN