Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What scientists must know about hardware to write fast code (2020)

viralinstruction.com

Post reply on HN
Print view View original

11–20 of 76 posts

Re: What scientists must know about hardware to write fast code (2020)

#11

Post by th0ma5 »

mjan22640 wrote: post #9
eesmith wrote: post #5

Earlier quoted context omitted.

Congratulations. Studying one field means you know that field. This link is not meant for you. It is meant for a scientist, and most scientists do not also have an EE degree or CS degree. How much graduate level biology, oceanography, physics, geology, chemistry, meteorology, or other scientific field do you know? All of those have subfields where computational performance is important. My experience is scientists ar…

Math was always a must for a scientist, todays computer science is also a must. The study programmes should reflect that.

I agree with this sentiment, like the majority of CS people are telling statisticians that a lot of Julia remains a kind of snake oil or otherwise mystical thinking, it is very unfortunate. Even in the first page of the documentation "No need to vectorize code for performance; devectorized code is fast" is some kind of category error redefinition of how programming languages work in my opinion.

Re: What scientists must know about hardware to write fast code (2020)

#12

Post by jarvist »

th0ma5 wrote: post #10
havercosine wrote: post #7

Solid post. It also shows how powerful Julia is: allowing to operate at different levels of abstractions (down to seeing the assembly) using the same set of tools.

I feel the biggest misleading statements around Julia is that for true speed you can somehow ignore the lower abstractions, or that there is some kind of free lunch, but always what you gain in performance you'll spend in development time. Julia has some neat tricks, but they are not generally and universally applicable at least not like other languages. I dunno. These arguments against Julia are many, but I'm still…

My experience is a first implementation / novice programmer will write Julia code of a similar speed to python. But then an intermediate Julia programmer can adapt that same code to be of order C / Fortran performance, without stopping the novice programmer from being able to work on the code base. So it's this ability to iteratively improve and collaborate across very different skill sets that's really important.

This is quite a different situation to traditional scientific computing.

Re: What scientists must know about hardware to write fast code (2020)

#13

Post by eesmith »

mjan22640 wrote: post #9
eesmith wrote: post #5

Earlier quoted context omitted.

Congratulations. Studying one field means you know that field. This link is not meant for you. It is meant for a scientist, and most scientists do not also have an EE degree or CS degree. How much graduate level biology, oceanography, physics, geology, chemistry, meteorology, or other scientific field do you know? All of those have subfields where computational performance is important. My experience is scientists ar…

Math was always a must for a scientist, todays computer science is also a must. The study programmes should reflect that.

"Math" is such a wide topic that you certainly must qualify your statement.

The standard entomologist curriculum does not require calculus, while a physics curriculum does. Both produce scientists. (For example, https://cals.cornell.edu/education/degrees-programs/entomolo... under "Major Requirements" says "One semester of college statistics or biometry", and the listed physics requirement doesn't require calculus.)

On the other hand, an entomologist interested in population ecology may need to know differential equations.

Your use of "study program" suggests your experience is at the undergrad level, and not at the grad school level, which is how most scientists I know got their training.

At the undergrad level the study programs do reflect what's needed for a solid education. If a student is interested in computational biology, that program will emphasize taking more CS courses than the program for a student interested in marine biology.

But at the grad level, the "study program" is much less formalized. You might take graduate level classes the first couple of years, but then you are expected to pick up the missing bits on your own.

Once you have your PhD and are a working scientist, you rarely have the luxury of following any study program.

And if you've been a scientist for 20 years, any CS training you had likely did not cover SIMD, and emphasized practices which are no longer relevant. (For example, the link points out "That advice [about HDDs] is mostly outdated today [with SSDs]".)

Those latter categories are who the linked-to piece is for, not undergrads in a well-defined study program.

Re: What scientists must know about hardware to write fast code (2020)

#14

Post by reachtarunhere »

jarvist wrote: post #12
th0ma5 wrote: post #10

Earlier quoted context omitted.

I feel the biggest misleading statements around Julia is that for true speed you can somehow ignore the lower abstractions, or that there is some kind of free lunch, but always what you gain in performance you'll spend in development time. Julia has some neat tricks, but they are not generally and universally applicable at least not like other languages. I dunno. These arguments against Julia are many, but I'm still…

My experience is a first implementation / novice programmer will write Julia code of a similar speed to python. But then an intermediate Julia programmer can adapt that same code to be of order C / Fortran performance, without stopping the novice programmer from being able to work on the code base. So it's this ability to iteratively improve and collaborate across very different skill sets that's really important. Th…

> My experience is a first implementation / novice programmer will write Julia code of a similar speed to python.

No because of JIT compilation he would write code faster than Python by default. Now to truly rival optimized C++ code one has to do the tricks mentioned in this post like optimizing memory access, SIMD and maximizing instruction parallelism.

The key point is you are better off by default and can do some ugly stuff in the critical parts of the code while still using the same language.

Re: What scientists must know about hardware to write fast code (2020)

#15

Post by pjmlp »

havercosine wrote: post #7

Solid post. It also shows how powerful Julia is: allowing to operate at different levels of abstractions (down to seeing the assembly) using the same set of tools.

This was the premise of Lisp Machines, unfortunely the industry took another path.

Re: What scientists must know about hardware to write fast code (2020)

#16

Post by eigenspace »

reachtarunhere wrote: post #14
jarvist wrote: post #12

Earlier quoted context omitted.

My experience is a first implementation / novice programmer will write Julia code of a similar speed to python. But then an intermediate Julia programmer can adapt that same code to be of order C / Fortran performance, without stopping the novice programmer from being able to work on the code base. So it's this ability to iteratively improve and collaborate across very different skill sets that's really important. Th…

> My experience is a first implementation / novice programmer will write Julia code of a similar speed to python. No because of JIT compilation he would write code faster than Python by default. Now to truly rival optimized C++ code one has to do the tricks mentioned in this post like optimizing memory access, SIMD and maximizing instruction parallelism. The key point is you are better off by default and can do some…

It really depends on what you end up doing. A lot of "python" code is really just thin wrappers around fairly optimized C routines (although there's inefficiencies from the wrapper, and the optimization barriers), so if you're doing something where the bulk of the work is happening inside those routines, beginner-written julia code will end up being roughly comparable to expert-written numpy code or whatever.

But yeah if you're writing a loop or something else where the majority of work is actually being done by python itself, then it's going to typically be much much slower than the equivalent julia code.

Re: What scientists must know about hardware to write fast code (2020)

#17

Post by prhcbsc »

adding onto multithreading, other parallelization models such as OpenMP or OMPSs take sequential code and parallelise it. They delegate onto a runtime system the efficient execution of the code to achieve a balance between programmers productivity and code performance.

But for large problems the article falls short. Scientific applications may need to use several computers at a time, COMP Superscalar (COMPSs) is a task-based programming model which aims to ease the development of applications for distributed infrastructures. COMPSs programmers do not need to deal with the typical duties of parallelization and distribution, such as thread creation and synchronization, data distribution, messaging or fault tolerance. Instead, the model is based on sequential programming, which makes it appealing to users that either lack parallel programming expertise or are looking for better programmability. Other popular frameworks such as LEGION offer a lower-level interface.

Re: What scientists must know about hardware to write fast code (2020)

#18

Post by cpach »

kylepdm wrote: post #8
dang wrote: post #3

Related: What scientists must know about hardware to write fast code (2020) - https://news.ycombinator.com/item?id=29601342 - Dec 2021 (29 comments)

FYI the underlying link in that previous discussion post seems to be defunct and kind of suspicious.

Looks like biojulia.net got taken over by spammers :(

Re: What scientists must know about hardware to write fast code (2020)

#19

Post by jakobnissen »

cpach wrote: post #18
kylepdm wrote: post #8

Earlier quoted context omitted.

FYI the underlying link in that previous discussion post seems to be defunct and kind of suspicious.

Looks like biojulia.net got taken over by spammers :(

Yeah. The single person who ran the BioJulia website disappeared (as people sometimes do in open-source volunteer work), and no-one else had permissions to the website or the domain. Oops.

We moved the website to https://biojulia.dev/, with permissions given to more people, including a core dev of Julia. That should reduce the risk of this happening again.

Re: What scientists must know about hardware to write fast code (2020)

#20

Post by jakobnissen »

th0ma5 wrote: post #10
havercosine wrote: post #7

Solid post. It also shows how powerful Julia is: allowing to operate at different levels of abstractions (down to seeing the assembly) using the same set of tools.

I feel the biggest misleading statements around Julia is that for true speed you can somehow ignore the lower abstractions, or that there is some kind of free lunch, but always what you gain in performance you'll spend in development time. Julia has some neat tricks, but they are not generally and universally applicable at least not like other languages. I dunno. These arguments against Julia are many, but I'm still…

I don't think that's right. Well, you're right that some enthusiasts of Julia are too quick to say "fast as C, easy as Python" without appending an asterisk to that statement. You can't get really fast performance without paying any attention to the hardware.

Performance is on a spectrum, and usually a tradeoff against readability and conciseness. I think it IS true that Julia excels in that it gives, by far, the best expressibility/performance tradeoff.

Also, often, you really can get "free lunch" - there are many times where if you just do the obvious thing, Julia and its backend LLVM can optimise it to extremely efficient code. For a simple example, just summing an array with a for loop, for example.

Post reply on HN