Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The compiler will optimize that away

blog.royalsloth.eu

Post reply on HN
Print view View original

261–270 of 329 posts

Re: The compiler will optimize that away

#261

Post by vbezhenar »

toss1 wrote: post #217
jupp0r wrote: post #125

Earlier quoted context omitted.

Do you use browsers? Browser makers spend billions of dollars in engineering time making them "fast", with great results.

>"Browsers... "fast", with great results." Are you joking? I have seen progressively worse performance from even the best browsers, and page load times that should be instant often literally take minutes or never load unless I completely kill the browser process and return. The slowdown is nearly inexorable, with occasional improvements in some versions before resuming the dismal trend. Simple word processing and spr…

I'm using very underpowered laptop (Dell 3410) and web is extremely fast for me. There's some serious problem in your OS or your network if websites take minutes to load. Dreadful gmail or youtube loads in 1-2 seconds for me and then works instantly. Simpler websites like HN loads in a fraction of second.

Re: The compiler will optimize that away

#262

Post by aspaceman »

jupp0r wrote: post #79

The implicit premise of the article seems to be that all software has to be heavily optimized. This is completely wrong. All the languages mentioned in the article are still around because it doesn’t matter how performant 99% of code is. For the 1%, we can think about cache misses, SIMD and data parallel approaches. In my experience this is totally possible and not too hard, but has the enormous downside of making th…

High performing code doesn’t have to be hard to reason about.

You wouldn’t say the plan for a safer building is harder to read.

People excusing their poorly written code as “highly-optimized” is the oldest cop-out we’ve got as programmers.

Re: The compiler will optimize that away

#263

Post by lumost »

jupp0r wrote: post #79

The implicit premise of the article seems to be that all software has to be heavily optimized. This is completely wrong. All the languages mentioned in the article are still around because it doesn’t matter how performant 99% of code is. For the 1%, we can think about cache misses, SIMD and data parallel approaches. In my experience this is totally possible and not too hard, but has the enormous downside of making th…

I generally only observe value in application code optimizing for correct data structures/batching/data parallelism. Once you're getting into fine tuning the batch size/SimD ops for a particular platform you are quickly into the realm of diminished ROI due to fragility against changing hardware, application usage patterns, and diminished returns where dropping an op from 1 microsecond to 10 nanoseconds just doesn't matter as much.

There are cases at the tail where this will make or break a project, but those are pretty rare.

Re: The compiler will optimize that away

#264

Post by 29athrowaway »

jayd16 wrote: post #189
Firehawke wrote: post #186

Earlier quoted context omitted.

It had Shoutcast.

Was Shoutcast all that fast? It didn't seem nearly as rock solid as the rest of winamp but I guess it could have been my internet at the time.

You could go to the shoutcast website, get the shoutcast links for the radios you were interested in, then make a playlist that could be dealt with just like any other playlist.

And it was almost instant.

Re: The compiler will optimize that away

#265

Post by f00zz »

PixelOfDeath wrote: post #195
f00zz wrote: post #138

Earlier quoted context omitted.

> There is some, lets call them experimental, C++ libraries for that. Could you provide some links?

https://github.com/Yamahari/struct_array It uses C++20 features and macros. The macros limit it to a POD count of 8.

Thanks!

Re: The compiler will optimize that away

#266

Post by rualca »

incrudible wrote: post #258
rualca wrote: post #252

Earlier quoted context omitted.

An array of structs is already a struct of arrays if you're willing to do some offset math.

That would make the whole thing moot. The crucial difference between AOS and SOA is that the "physical" memory layout is different: AOS: [xyzw][xyzw][xyzw][xyzw] vs SOA: [xxxx][yyyy][zzzz][wwww] If you have such a layout, a SIMD instruction touching 4 packed words has quite different behavior. Arguably, SIMD is pointless without SOA or AOSOA.

> SOA: [xxxx][yyyy][zzzz][wwww]

That difference only occurs if, for some random reason, you came about what's the address of element zero of that array, or if for any even rarer reason you have a hard requirement of when you cease to process x and start to process y, and you find a single memory jump prohibitively expensive.

Meanwhile, AOS is already by definition [xxxx] and [yyyy] and [zzzz] and [wwww], given the remaining members of the struct are treated as padding.

Re: The compiler will optimize that away

#267

Post by incrudible »

rualca wrote: post #266
incrudible wrote: post #258

Earlier quoted context omitted.

That would make the whole thing moot. The crucial difference between AOS and SOA is that the "physical" memory layout is different: AOS: [xyzw][xyzw][xyzw][xyzw] vs SOA: [xxxx][yyyy][zzzz][wwww] If you have such a layout, a SIMD instruction touching 4 packed words has quite different behavior. Arguably, SIMD is pointless without SOA or AOSOA.

> SOA: [xxxx][yyyy][zzzz][wwww] That difference only occurs if, for some random reason, you came about what's the address of element zero of that array, or if for any even rarer reason you have a hard requirement of when you cease to process x and start to process y, and you find a single memory jump prohibitively expensive. Meanwhile, AOS is already by definition [xxxx] and [yyyy] and [zzzz] and [wwww], given the re…

> Meanwhile, AOS is already by definition [xxxx] and [yyyy] and [zzzz] and [wwww]

There must be a misunderstanding here. AOS is by definition [xyzw][xyzw](...), the (C/C++) compiler has no leeway in rearranging this. I'm also assuming that there's no padding necessary, in either case.

Perhaps my way of notation is confusing, in that case I'm going to refer you to this page, which has code examples:

https://software.intel.com/content/www/us/en/develop/article...

Re: The compiler will optimize that away

#268

Post by dragontamer »

While this article starts to discuss the low level details that underlie our code, I feel like its incomplete.

* Latency hiding -- CPUs work extremely hard on latency hiding. A lot of those OOP-indirect calls will be branch predicted by the CPU in the simple cases (ex: if one object-type is more common than others, then those indirect branches will be branch predicted correctly in most situations).

There are other bits of latency hiding: Hyperthreads, Out-of-order execution, pipelines.

* Cache: L1, L2, and L3 caches can grossly mitigate a lot of the problems. While CPUs today are far faster than DDR4 RAM, L1 cache scales perfectly with the CPU. If you can keep your code+data under 64kB, you'll always be executing in L1 cache and basically get the full benefits.

If 64kB is too restrictive, then you have 256kB or 512kB (L2 cache), or 8MB (L3 cache), both of which scale with the CPU.

------------

SIMD and Multicore goes backwards IMO. It turns out that the 1970s and 1980s were filled with very intelligent programmers working on extremely sophisticated supercomputers. I enjoy going back to the 1980s and reading Connection Machine articles on C-Star (a parallel language that's arguably the precursor to CUDA), or Star-Lisp.

--------

If you write multicore software, then SMT / Hyperthreads are very effective at latency hiding. Running two threads on one core (or 4-threads or 8-threads, in the case of some IBM processors) allows your cores to "find something else to do" during those long latency periods where your CPU is traversing memory and waiting for RAM to respond.

Working extremely hard to optimize just a single thread seems somewhat counterproductive. Its often easier to write "less optimized" multicore software than highly optimized single-core software.

And SMT / hyperthreads will converge those onto a single core ANYWAY. So might as well take advantage of that feature.

Re: The compiler will optimize that away

#269

Post by toss1 »

pwdisswordfish8 wrote: post #230
toss1 wrote: post #217

Earlier quoted context omitted.

>"Browsers... "fast", with great results." Are you joking? I have seen progressively worse performance from even the best browsers, and page load times that should be instant often literally take minutes or never load unless I completely kill the browser process and return. The slowdown is nearly inexorable, with occasional improvements in some versions before resuming the dismal trend. Simple word processing and spr…

> Are you joking? I have seen progressively worse performance from even the best browsers Where is the evidence that you're seeing poor performance from the browser itself and that the source of the problem does not lie in the difference between what the server is sending down the tubes today compared to what it was sending 10 years ago?

I agree that what the server is 'sending down the tubes' is a huge part of the problem.

But this is the root cause we're discussing - programmers selecting tools for their convenience (and worse yet, cool factor), instead of FIRST considering the responsiveness of the system as they design and code.

Optimization as an afterthought is about as good as security as an afterthought - anything from a complete waste of time to a disaster.

There are indeed pages that load like lightning, so it can be done (e.g., HN takes about 1.5sec to create a new window and load, so not exactly lightning, but usable), but many are horrible, and clearly due to bad programming.

For starters, when I see a page that loads code from 25 different sites that need NoScript privs to even display, that alone is pretty questionable - license and manage your own damn code (for the sake of minimizing dependency alone!). Twitter is particularly egregious in the last year or so, a new page taking 10sec-?? to load, and the LAST thing that loads is the list of posts -- the same load times it would feel so much more responsive if that was the first to load, and the other navigation, news, etc. panels loaded later while I was reading. That is a many bad programming choices.

Re: The compiler will optimize that away

#270

Post by cma »

Const-me wrote: post #24

Disagree about garbage collected runtimes. A lot of widely used software is written in C or C++ and uses malloc/free extensively (C++ new/delete is mostly a wrapper around it). This results in memory layout worse than an equivalent managed heap would be. Happens because the memory allocated on C heap is immovable, while garbage collectors may move data around to defragment the heap.

Data-oriented designs like ECS implementations often use generational indices, which can support relocation and defragmentation.
Post reply on HN