Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Old box, dumb code, few thousand connections, no big deal

rachelbythebay.com

Post reply on HN
Print view View original

281–288 of 288 posts

Re: Old box, dumb code, few thousand connections, no big deal

#281

Post by kragen »

augustt wrote: post #119
highfrequency wrote: post #90

Earlier quoted context omitted.

> single-threaded nonvectorized C wastes on the order of 97% of your computer's computational power Can you elaborate on what this means exactly? For example, is there some reasonable C code that runs 33 times slower than some other ideal code? In what sense are we wasting 97% of our computer's computational power?

A good example of getting a ~3000x speedup from naive matrix multiplication in C here (slides 20 onward): https://ocw.mit.edu/courses/electrical-engineering-and-compu... Includes a 9-level nested for loop, which is always great to see.

Thank you very much for posting this!

Roughly that 3000× is 18× from multithreading, 3× from SIMD instructions, 15× from tuning access patterns for locality of reference, and 3× for turning on compiler optimization options. This is a really great slide deck!

I was assuming "single-threaded nonvectorized C" already had compiler optimization turned on and locality of reference taken into account. As the slide deck notes, you can get some vectorization out of your compiler — but usually it requires thinking like a FORTRAN programmer.

So I think in this case reasonable C code runs about 54× slower than Leiserson's final code. However, you could probably get a bigger speedup in this particular case with GPGPU. Other cases may be more difficult to get a GPU speedup, but get a bigger SIMD speedup. So I think my 97% is generally in the ballpark.

A big problem is that we can't apply this level of human effort to optimizing every subroutine. We need better languages.

Re: Old box, dumb code, few thousand connections, no big deal

#282

Post by chris_wot »

epicide wrote: post #279
chris_wot wrote: post #275

Earlier quoted context omitted.

What part of "I think some of us who have been doing this a while have been doing a terrible job of showing what's possible to those folks who are somewhat newer." didn't you understand?

Yeah, first sentence no less. No excuse for that one!

You just earned my respect.

Re: Old box, dumb code, few thousand connections, no big deal

#283

Post by mratsim »

kragen wrote: post #281
augustt wrote: post #119

Earlier quoted context omitted.

A good example of getting a ~3000x speedup from naive matrix multiplication in C here (slides 20 onward): https://ocw.mit.edu/courses/electrical-engineering-and-compu... Includes a 9-level nested for loop, which is always great to see.

Thank you very much for posting this! Roughly that 3000× is 18× from multithreading, 3× from SIMD instructions, 15× from tuning access patterns for locality of reference, and 3× for turning on compiler optimization options. This is a really great slide deck! I was assuming "single-threaded nonvectorized C" already had compiler optimization turned on and locality of reference taken into account. As the slide deck note…

That's why you have people working on Halide, Taichi, DaCe, Tiramisu.

- https://halide-lang.org/

- http://taichi.graphics/

- http://spcl.inf.ethz.ch/Research/DAPP/

- http://tiramisu-compiler.org/

This way you can have a researcher implementing the algorithm (say bilinear filtering) and a HPC expert who tunes it with parallelism, SIMD, tiling.

I wrote an overview of most DSL for high performance or image processing in this issue: https://github.com/mratsim/Arraymancer/issues/347#issuecomme...

Re: Old box, dumb code, few thousand connections, no big deal

#284

Post by kragen »

mratsim wrote: post #283
kragen wrote: post #281

Earlier quoted context omitted.

Thank you very much for posting this! Roughly that 3000× is 18× from multithreading, 3× from SIMD instructions, 15× from tuning access patterns for locality of reference, and 3× for turning on compiler optimization options. This is a really great slide deck! I was assuming "single-threaded nonvectorized C" already had compiler optimization turned on and locality of reference taken into account. As the slide deck note…

That's why you have people working on Halide, Taichi, DaCe, Tiramisu. - https://halide-lang.org/ - http://taichi.graphics/ - http://spcl.inf.ethz.ch/Research/DAPP/ - http://tiramisu-compiler.org/ This way you can have a researcher implementing the algorithm (say bilinear filtering) and a HPC expert who tunes it with parallelism, SIMD, tiling. I wrote an overview of most DSL for high performance or image processing in…

This is great! Which of these do you think could be extended to general-purpose programming without the HPC expert? Taichi and DAPP seem to be aimed at that goal, but you seem to be implying they don't reach it yet?

Re: Old box, dumb code, few thousand connections, no big deal

#285

Post by Doxin »

pjmlp wrote: post #228
vidarh wrote: post #181

Earlier quoted context omitted.

This is something a lot of people don't get with most higher level languages. My first commercial use of Ruby was in 2005. Not web facing, but messaging middleware. As in a pub-sub type passing of messages between various endpoints. We had a C version. It was about 7k lines to support the bare minimum we needed. As an experiment to teach myself Ruby I wrote a Ruby implementation. With the usual caveats (it's often ea…

And other people don't get it is possible to have high level languages and almost C like performance. You don't need to give up on JIT and AOT compilation to use high level languages, and this is where current tooling for Ruby and Python ends up losing.

Fair enough, I've been playing with D a lot lately, which is basically "what if python was a C dialect instead". It's an incredibly simple language to learn but it's no less easy to write in than python. For most things I still reach for python though, probably because I've grown comfortable with duck typing.

side-note when starting with D: make sure to install dub. it's the package manager and basically eliminates makefiles from the compilation process. Just "dub init" and "dub run" and you're off to the races.

Re: Old box, dumb code, few thousand connections, no big deal

#286

Post by Doxin »

fnord123 wrote: post #217
Doxin wrote: post #173

Earlier quoted context omitted.

Not to mention that python is plenty fast compared to the time it takes to write stuff to the network. Of course heavyweight frameworks like django don't help the equation, but writing fast network code in python isn't exactly hard either.

>python is plenty fast compared to the time it takes to write stuff to the network. Give us some numbers. Inside the data centre you have 10G, 40G, 100G ethernet connections. I know for a fact that you will struggle to soak a 10G connection using a single thread so I know you can't do this in Python without multiple processes using SO_REUSEPORT.

So use multiple processes with SO_REUSEPORT then. Or find yourself a wsgi server that does, because it's not exactly an unsolved problem.

That said by far most programs don't need to worry about saturating a 10G connection. I'm not writing a file server in python, I'll leave that to nginx or S3. I'm writing business logic in python which tends to be bottlenecked by a database in any case.

Python is great for plumbing together other functionality, which it turns out means most backends you'd be writing anyways. Python is less great at handling a large quantity of data, though most of the time you can get away with handing the data handling to some library (e.g. numpy or libuv or any one of thousands of libraries).

Worst case you can easily plumb in some C-calling-convention code into python. With FFI it's a matter of copying the header definition and you're off to the races. That way you can still write the bulk of the program in python, delegating the bulk data wrangling to C or D or rust or go or whatever you prefer.

Re: Old box, dumb code, few thousand connections, no big deal

#287

Post by mratsim »

kragen wrote: post #284
mratsim wrote: post #283

Earlier quoted context omitted.

That's why you have people working on Halide, Taichi, DaCe, Tiramisu. - https://halide-lang.org/ - http://taichi.graphics/ - http://spcl.inf.ethz.ch/Research/DAPP/ - http://tiramisu-compiler.org/ This way you can have a researcher implementing the algorithm (say bilinear filtering) and a HPC expert who tunes it with parallelism, SIMD, tiling. I wrote an overview of most DSL for high performance or image processing in…

This is great! Which of these do you think could be extended to general-purpose programming without the HPC expert? Taichi and DAPP seem to be aimed at that goal, but you seem to be implying they don't reach it yet?

You can use them without the HPC expert, Halide for example has a good autotuner and has been used by Google and Adobe to create image filters for mobile devices.

Re: Old box, dumb code, few thousand connections, no big deal

#288

Post by kragen »

mratsim wrote: post #287
kragen wrote: post #284

Earlier quoted context omitted.

This is great! Which of these do you think could be extended to general-purpose programming without the HPC expert? Taichi and DAPP seem to be aimed at that goal, but you seem to be implying they don't reach it yet?

You can use them without the HPC expert, Halide for example has a good autotuner and has been used by Google and Adobe to create image filters for mobile devices.

Thank you!
Post reply on HN