Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Verbatim: Stats guru Jeff Sagarin

college-basketball.si.com

Post reply on HN
Print view View original

11–20 of 44 posts

Re: Verbatim: Stats guru Jeff Sagarin

#11

Post by noonespecial »

cing wrote: post #9
na85 wrote: post #2

The point of this submission being....? The fact that someone runs statistics calculations in Fortran (a language whose most recent standard is dated 2008) should be shocking only to the recent generation of coders who have become "code hipsters." The guys looking for the Next Big Thing™ in programming languages. These are the guys trying to cram Python into every possible goddamned use case, whether that be embedded…

Is Fortran actually the best choice for this application? I don't know how heavy the calculations are for sports analytics, but I'm guessing it's less advanced than scientific simulations. Who cares how numerically efficient your code is if you use 10x the lines than a modern language and spend half your time struggling to maintain it.

The language is only half the machine. The human is the other half. The question is not if Fortran is the best choice, it's is Fortran the best choice for him.

Re: Verbatim: Stats guru Jeff Sagarin

#12

Post by kyllo »

na85 wrote: post #2

The point of this submission being....? The fact that someone runs statistics calculations in Fortran (a language whose most recent standard is dated 2008) should be shocking only to the recent generation of coders who have become "code hipsters." The guys looking for the Next Big Thing™ in programming languages. These are the guys trying to cram Python into every possible goddamned use case, whether that be embedded…

These are the guys trying to cram Javascript into every possible goddamned use case

Fixed that for you.

Yes Fortran is probably the right tool for the job in this case, but I also think that creative experimentation with programming languages, invention of new languages, new implementations of and enhancements to existing languages, are good and important, and not just a "hipster" thing. All modern programming languages and their implementations have warts and deficiences, and building a new and improved one requires a level of skill and dedication that goes far beyond hipsterism.

Re: Verbatim: Stats guru Jeff Sagarin

#13

Post by na85 »

cing wrote: post #9
na85 wrote: post #2

The point of this submission being....? The fact that someone runs statistics calculations in Fortran (a language whose most recent standard is dated 2008) should be shocking only to the recent generation of coders who have become "code hipsters." The guys looking for the Next Big Thing™ in programming languages. These are the guys trying to cram Python into every possible goddamned use case, whether that be embedded…

Is Fortran actually the best choice for this application? I don't know how heavy the calculations are for sports analytics, but I'm guessing it's less advanced than scientific simulations. Who cares how numerically efficient your code is if you use 10x the lines than a modern language and spend half your time struggling to maintain it.

Not sure why code to crunch some statistics would have a huge maintenance cost. There goes one point out the window.

As for the first, a) I touched on the fact that Fortran is a modern language and b) unless you've got examples showing two side-by-side implementations of the same algorithm with Fortran being 10x longer, then it sounds like you're just another of the aforementioned hipsters who is experiencing aversion to something just because it is not new and shiny.

Re: Verbatim: Stats guru Jeff Sagarin

#14

Post by jofer »

kyllo wrote: post #8
Wilya wrote: post #5

Fortan isn't dead, by far. There is a lot of scientific and industrial code (in astronomy, aeronautics, ...) that is in Fortran. And I'm not talking about legacy stuff that is only in maintenance mode. These codes are still evolving and get new functionalities. They're not in Fortran only for legacy reasons (though that plays some part). They're in Fortran because Fortan is still the right tool for the job. Scientifi…

Yeah, anyone who has installed numpy, matplotlib, etc has probably noticed that you need a Fortran compiler to build the native extensions...

Actually, you don't. A fortran compiler is optional, and used mostly for f2py and I think(?) for the LAPACK bindings. (The fortran bindings are more efficient than the C bindings for reasons I don't remember at the moment.)

Numpy's (and matplotlib's, actually) core is written in C.

Scipy is a different story, however. A lot of scipy's internals are written in fortran.

That having been said, fortran is a _very_ nice language for a significant portion of scientific programming tasks. IO in fortran is rather painful, though, i.m.o. Tools like f2py (which still doesn't really support fully modern fortran, unfortunately) make it very easy to combine fortran and python, for whatever it's worth.

Re: Verbatim: Stats guru Jeff Sagarin

#15

Post by MichaelSalib »

na85 wrote: post #2

The point of this submission being....? The fact that someone runs statistics calculations in Fortran (a language whose most recent standard is dated 2008) should be shocking only to the recent generation of coders who have become "code hipsters." The guys looking for the Next Big Thing™ in programming languages. These are the guys trying to cram Python into every possible goddamned use case, whether that be embedded…

Um, people have been doing high performance numeric and scientific computing in python for over 15 years now. The truth is, for a lot of numeric computing, you're better off using python, which calls fortran or c code to do the heavy lifting.

Fortran can run very fast. If all you need to do is numerical calculations, that might be enough. But that's never all you need to do: you need to preprocess your input data, or deserialize it, normalize it, join it or transform it based on some rdbms data, then you do your calculations, and then you need to graph it, serialize it, etc. Most of those tasks are somewhere between excruciating and impossible in fortran, however modern a dialect you use.

Re: Verbatim: Stats guru Jeff Sagarin

#16

Post by na85 »

MichaelSalib wrote: post #15
na85 wrote: post #2

The point of this submission being....? The fact that someone runs statistics calculations in Fortran (a language whose most recent standard is dated 2008) should be shocking only to the recent generation of coders who have become "code hipsters." The guys looking for the Next Big Thing™ in programming languages. These are the guys trying to cram Python into every possible goddamned use case, whether that be embedded…

Um, people have been doing high performance numeric and scientific computing in python for over 15 years now. The truth is, for a lot of numeric computing, you're better off using python, which calls fortran or c code to do the heavy lifting. Fortran can run very fast. If all you need to do is numerical calculations, that might be enough. But that's never all you need to do: you need to preprocess your input data, or…

>Um, people have been doing high performance numeric and scientific computing for over 15 years now.

Have I disputed this fact somewhere?

>The truth is, for a lot of numeric computing, you're better off using python, which calls fortran or c code to do the heavy lifting.

Proof that Python can't get the job done in that particular area. Nevertheless there remain Python zealots who will tell you that it's the best for everything.

>Fortran can run very fast. If all you need to do is numerical calculations, that might be enough.

Actually it sounds like that is all he needs to do. Reading and spitting out a .csv file from Fortran is trivial, despite its clunky IO syntax.

Re: Verbatim: Stats guru Jeff Sagarin

#17

Post by vonmoltke »

cing wrote: post #9
na85 wrote: post #2

The point of this submission being....? The fact that someone runs statistics calculations in Fortran (a language whose most recent standard is dated 2008) should be shocking only to the recent generation of coders who have become "code hipsters." The guys looking for the Next Big Thing™ in programming languages. These are the guys trying to cram Python into every possible goddamned use case, whether that be embedded…

Is Fortran actually the best choice for this application? I don't know how heavy the calculations are for sports analytics, but I'm guessing it's less advanced than scientific simulations. Who cares how numerically efficient your code is if you use 10x the lines than a modern language and spend half your time struggling to maintain it.

On the sports analytics question I don't have any knowledge or experience, but I can see operations there turning into multi-matrix statistical correlations and such.

On the subject of being the best language for the job, Fortran does not use 10x more lines than more modern languages. In fact, in many cases it uses less. When I was working on real-time signal processing algorithms we used to prototype algorithms in Fortran[1], then implement them in C. I also used to use Python with SciPy and NumPy for analysis, as well as Matlab. Matlab's huge toolbox library notwithstanding, the Fortran implementations were usually shorter, simpler, and easier to understand than any of the other languages. It was only bias and preconceived notions that kept it from being used in the real system. I would have loved to see Fortran subroutines running the algorithms underneath the C distributed framework.

[1] Until someone in management and the customer team decided to have the scientists write their algorithms directly in the real-time C. That is a rant for another day.

Re: Verbatim: Stats guru Jeff Sagarin

#18

Post by mturmon »

jgrahamc wrote: post #4

I'm guessing the title was chosen because the submitter thought coding in Fortran was stupid, or old, or whatever. The reality is that Fortran has been updated frequently and for what he's doing seems like a reasonable choice (plus he probably has a store of routines that he likes to use for certain analyses). Old language != Bad language From TFA: "SI: Is it true you still code in Fortran? JS: Yeah, what’s wrong wit…

I agree with your observations, especially that the the expert has a built-up library of routines that he understands and that have proven themselves.

My question about using Fortran as the main infrastructure is that this does not sound like a big data problem. It sounds more like an exploratory data analysis and model development problem. In which case a REPL-based environment like Python, Matlab, or R would have a lot to offer: interactivity, integrated plotting, modeling toolboxes.

People in my group do scientific computing with Fortran, large datasets, and big iron, and it works great in that domain. As people in this thread have noted, there is significant support for automated parallelization. I used to snicker at these dinosaurs (their own self-deprecating term), but I have noticed their publication record seems to be piling up faster than mine. I have therefore gotten over it, and hope that the dinosaurs are not snickering back at me.

Re: Verbatim: Stats guru Jeff Sagarin

#20

Post by nhebb »

jgrahamc wrote: post #4

I'm guessing the title was chosen because the submitter thought coding in Fortran was stupid, or old, or whatever. The reality is that Fortran has been updated frequently and for what he's doing seems like a reasonable choice (plus he probably has a store of routines that he likes to use for certain analyses). Old language != Bad language From TFA: "SI: Is it true you still code in Fortran? JS: Yeah, what’s wrong wit…

The problem isn't with the programming language; it with his algorithms. Sagarin's rankings are often an outlier, frequently in ways that just don't make sense.
Post reply on HN