Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What scientists must know about hardware to write fast code (2020)

viralinstruction.com

Post reply on HN
Print view View original

21–30 of 76 posts

Re: What scientists must know about hardware to write fast code (2020)

#21

Post by newswasboring »

th0ma5 wrote: post #11
mjan22640 wrote: post #9

Earlier quoted context omitted.

Math was always a must for a scientist, todays computer science is also a must. The study programmes should reflect that.

I agree with this sentiment, like the majority of CS people are telling statisticians that a lot of Julia remains a kind of snake oil or otherwise mystical thinking, it is very unfortunate. Even in the first page of the documentation "No need to vectorize code for performance; devectorized code is fast" is some kind of category error redefinition of how programming languages work in my opinion.

> Even in the first page of the documentation "No need to vectorize code for performance; devectorized code is fast" is some kind of category error redefinition of how programming languages work in my opinion.

Can you elaborate a bit? I don't really get what you are trying to say.

Re: What scientists must know about hardware to write fast code (2020)

#23

Post by AtlasBarfed »

eesmith wrote: post #13
mjan22640 wrote: post #9

Earlier quoted context omitted.

Math was always a must for a scientist, todays computer science is also a must. The study programmes should reflect that.

"Math" is such a wide topic that you certainly must qualify your statement. The standard entomologist curriculum does not require calculus, while a physics curriculum does. Both produce scientists. (For example, https://cals.cornell.edu/education/degrees-programs/entomolo... under "Major Requirements" says "One semester of college statistics or biometry", and the listed physics requirement doesn't require calculus.)…

The article basically implies that some non-professional coder will be doing assembly and basically doing the work of an optimizing compiler. I think the point of the parent is that if you are at this point already, and you're in an academic setting, you might as well real a full computer architecture textbook front to back.

I would be curious to know of all the "scientific coders" what percentage of them understood the entire article. I'd be similarly curious how much your typical "bootcamp" developers would understand of it. I know everything presented, so it basically comes off as a "lecture notes" for someone that already knows it. Someone that doesn't understand SIMD, CPU fundamentals, assembly, and compilers, I'd imagine their eyes would glaze right when the assembly code appeared.

And while SSDs are MUCH FASTER than HDDs, the basics of interacting with storage is the same, just that rather than waiting a million years for data to arrive from the CPU's perspective, it comes in 10,000s of years.

Latency numbers all programmers should be aware of:

https://gist.github.com/jboner/2841832

Re: What scientists must know about hardware to write fast code (2020)

#24

Post by a1o »

I don't know what is used to render this post, but the table of content as a floating icon would work best being closer to bottom at left, on mobile, because there is a scrollbar floating on top of it at the right that makes hard to tap it, and also because the eyes on the screen look at top of the screen mostly.

Re: What scientists must know about hardware to write fast code (2020)

#25

Post by dartos »

pjmlp wrote: post #15
havercosine wrote: post #7

Solid post. It also shows how powerful Julia is: allowing to operate at different levels of abstractions (down to seeing the assembly) using the same set of tools.

This was the premise of Lisp Machines, unfortunely the industry took another path.

Was it? I thought even in the lisp machine days, lisp was kind of garbage collected, so that was always the bottleneck.

I remember a quote that was like “Lisp programmers know the value of everything and the cost of nothing” in reference to that.

Re: What scientists must know about hardware to write fast code (2020)

#26

Post by mathisfun123 »

havercosine wrote: post #7

Solid post. It also shows how powerful Julia is: allowing to operate at different levels of abstractions (down to seeing the assembly) using the same set of tools.

I don't see affordances for operating at multiple levels of abstraction. The single example of another level is ccall to an LLVM instrinsic - that's not any different from inline assembly in basically any other compiled language. Supporting multiple levels would mean you can do all (or most) the same things with LLVM IR that you can do with Julia itself.

Re: What scientists must know about hardware to write fast code (2020)

#27

Post by lispm »

dartos wrote: post #25
pjmlp wrote: post #15

Earlier quoted context omitted.

This was the premise of Lisp Machines, unfortunely the industry took another path.

Was it? I thought even in the lisp machine days, lisp was kind of garbage collected, so that was always the bottleneck. I remember a quote that was like “Lisp programmers know the value of everything and the cost of nothing” in reference to that.

> “Lisp programmers know the value of everything and the cost of nothing”

Obviously the developers of Lisp Machine operating systems could not ignore the cost of the operations. Especially since they developed ambitious software (an operating system and its application) on relatively slow machines (a Symbolics 3600 was as fast as a 1 MIPS DEC VAX 11/780).

Re: What scientists must know about hardware to write fast code (2020)

#28

Post by jschveibinz »

jschveibinz wrote: post #2

This subject is taught in undergrad computer architecture courses along with machine coding. As an EE, I learned it in grad school.

Well, apparently I offended a lot of people by merely providing a piece of information about computer architecture curriculum. I apologize for commenting.

Re: What scientists must know about hardware to write fast code (2020)

#29

Post by morcus »

In college (for a time) I was a double major in CS and Physics. I found a job as a programmer at a Physics lab, which fit my interests very well. The previous person roughly showed me the ropes for just a few days before she left to go to grad school.

The PI started asking me to run some analyses on a raw dataset. Since I was so new at it, I often messed up and had to rerun the whole thing after looking at the output; this was painful because the entire script took a few hours to run.

I started poking around to see whether it could be optimized at all. the raw data was divided up into hundreds files from different runs, sensors, etc..., that were each processed independently in sequence, and the results were all combined together into a big array for the final result. Seems reasonable enough.

Except this code was all written by scientists, and the combination was done in the "naive" way - after each of data files was processed, a new array was created and the previous results were copied into the new array, as were the results from the current data file. This meant that for the iterations at the end, we roughly needed to have Memory = 2 * Size of final data, which eventually exceeded the amount of physical memory on the machine (and because there were so many data files, it was doing this allocation and copying dozens of times after it used all the RAM).

I updated this to pre-allocate the required size at the beginning for a very very easy 3-4 fold improvement in the overall runtime and felt rather proud of myself.

Re: What scientists must know about hardware to write fast code (2020)

#30

Post by lebuffon »

pjmlp wrote: post #15
havercosine wrote: post #7

Solid post. It also shows how powerful Julia is: allowing to operate at different levels of abstractions (down to seeing the assembly) using the same set of tools.

This was the premise of Lisp Machines, unfortunely the industry took another path.

Ditto for Forth.
Post reply on HN