CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R • Hacker News Board

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#101

Post by eesmith » Sun, Oct 11, 2020, 7:29 PM UTC

leovailati wrote: post #75

bonoboTP wrote: post #14

Maybe I'm a cranky old man, but I think there's immense value in the fact that Python's (scientific and non-scientific) ecosystem has matured and has become standard over the last 1-2 decades. Starting again from the ground up in a shiny new language (even if the language is perhaps slightly better) is a waste of so much effort. Transitioning from Python 2 to 3 was already a nightmare. I know there's some interoperab…

Disruption is the price of progress. Like cars with electric motors vs combustion engines, sometimes you have to start from scratch. You can't just keep improving combustion engines forever, you eventually reach a technological limit. To your point though, I do see a lot of unnecessary disruption particularly in the web dev world. I think people like working on new stuff. It's exciting to take the first steps, see or…

As an aside, I think that's a poor analogy. Electric cars have been around since the late 1800s, only a decade or two after the first internal combustion engine, and for a while set land speed records. The last 30 years of electric cars did not start from scratch.

Electric motors reached a technological plateau. As https://en.wikipedia.org/wiki/Electric_car points out, it required MOSFET power converters and lithium-ion batteries. Before then, it could appear that it reached a technological limit.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#102

Post by mlthoughts2018 » Sun, Oct 11, 2020, 7:30 PM UTC

StefanKarpinski wrote: post #99

dmw_ng wrote: post #51

There is a ton of conflation in this post, yesterdays post, and the comments. * Comparing parallel implementations (Julia) to single thread (Python csv module). This is still quite relevant because in places like Python, it is impossible to parallelize the csv module without essentially updating its native implementation. That's not true in other places, where some simple wrapper code may be all required to get that…

You may be missing the point of the comparison, which is this: to compare the performance of the de facto standard CSV parsers of each system when used the way that a typical user would use them, i.e. to parse a CSV file into a data frame. Regarding your specific points: 1. Yes, it's possible to rig up some way of parsing in parallel in Python, like splitting the CSV file into multiple files and then parsing them in…

A typical user will rarely ever use the built in CSV library in Python, and would choose among the many third party options for much greater performance, ease of use, parity with rectangular data structure tools like pandas, and so forth.

Comparing anything against the built-in CSV reader in Python is pretty much an exercise in nothing.

In 20 years of programming Python professionally, I have used the built in CSV parser about 5 total times, and all were use cases where 10-20x improvement of CSV parser performance did not matter whatsoever to the application.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#103

Post by mlthoughts2018 » Sun, Oct 11, 2020, 7:32 PM UTC

nextos wrote: post #96

dmw_ng wrote: post #51

There is a ton of conflation in this post, yesterdays post, and the comments. * Comparing parallel implementations (Julia) to single thread (Python csv module). This is still quite relevant because in places like Python, it is impossible to parallelize the csv module without essentially updating its native implementation. That's not true in other places, where some simple wrapper code may be all required to get that…

I would also say, for those not familiar with Julia, that the main advantage of Julia vs Python or R is that you can write performant code in Julia that will be fast enough for most scenarios. For example, there are efforts to write a pure BLAS in Julia that is still performant [1]. If you are into numerical computing, you will quickly understand this is crazy cool. A consequence of that is composability. Most librar…

This is also very easy to achieve in Python by using Cython to selectively optimize code. That way for the 99% of code where such optimizations are meaningless and their requirement for static typing is a liability, you can ignore them, and only focus on the 1% of use cases where it matters.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#104

Post by montalbano » Sun, Oct 11, 2020, 7:32 PM UTC

krastanov wrote: post #34

bonoboTP wrote: post #25

Earlier quoted context omitted.

> By that logic people should have kept using Fortran instead of Python There were very real pain points with Fortran, C++, MATLAB etc. compared to Python. Now this is hard for me to argue in a few sentences as it's a distillation of having worked with many languages, but I feel that Python strikes a great balance for productivity. It feels like pouring your thoughts directly onto the screen, with no need to fight th…

> The only issue with Python is speed, but if you use Numpy properly, even that isn't a big issue. Numpy leaves a couple of orders of magnitude of performance on the table, especially if you use small arrays and a lot of intermediary computation. It is terrible in terms of memory allocation overhead. Cython or @tensorflow.function fix much of that, but then you are not really using python anymore. > They often don't…

Agree, but to add, there are many transcendental functions (e.g. I needed to use the Mittag-Leffler function many times in my work) which are near-impossible to implement in Numpy in a way that gives anywhere near usable performance. It's not a common enough function to be pre-implemented in Scipy special functions.

In my case, I wrote the algorithm in Numpy/python myself which was almost unusably slow. I then outsourced it to a precompiled fortran program. Finally I just switched to Julia. There was already a library, MittagLeffler.jl which did everything I needed and was written in pure Julia. In the end everything was much faster than the Python/Fortran frankenstein code I had.

Edit: For more info on Julia and difficult (computationally) mathematical functions see:

https://youtu.be/mSgXWpvQEHE?t=1262

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#105

Post by StefanKarpinski » Sun, Oct 11, 2020, 7:33 PM UTC

mlthoughts2018 wrote: post #102

StefanKarpinski wrote: post #99

Earlier quoted context omitted.

You may be missing the point of the comparison, which is this: to compare the performance of the de facto standard CSV parsers of each system when used the way that a typical user would use them, i.e. to parse a CSV file into a data frame. Regarding your specific points: 1. Yes, it's possible to rig up some way of parsing in parallel in Python, like splitting the CSV file into multiple files and then parsing them in…

A typical user will rarely ever use the built in CSV library in Python, and would choose among the many third party options for much greater performance, ease of use, parity with rectangular data structure tools like pandas, and so forth. Comparing anything against the built-in CSV reader in Python is pretty much an exercise in nothing. In 20 years of programming Python professionally, I have used the built in CSV pa…

The fact that the standard CSV reader is not good enough to actually use except in toy applications is itself an issue, frankly.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#106

Post by st1x7 » Sun, Oct 11, 2020, 7:36 PM UTC

elcomet wrote: post #69

genomez4fun wrote: post #62

Earlier quoted context omitted.

julia is interpreted -- it's type system enables the speedups often presented in benchmarks.

It's a bit more complex. Both python and julia are compiled to bytecode. Then python bytecode gets interpreted, but I think that julia bytecode is actually compiled with a JIT compiler. So julia is not really interpreted.

Can you explain the difference between the two at a beginner level? (the two sound kind of the same)

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#107

Post by krastanov » Sun, Oct 11, 2020, 7:38 PM UTC

montalbano wrote: post #104

krastanov wrote: post #34

Earlier quoted context omitted.

> The only issue with Python is speed, but if you use Numpy properly, even that isn't a big issue. Numpy leaves a couple of orders of magnitude of performance on the table, especially if you use small arrays and a lot of intermediary computation. It is terrible in terms of memory allocation overhead. Cython or @tensorflow.function fix much of that, but then you are not really using python anymore. > They often don't…

Agree, but to add, there are many transcendental functions (e.g. I needed to use the Mittag-Leffler function many times in my work) which are near-impossible to implement in Numpy in a way that gives anywhere near usable performance. It's not a common enough function to be pre-implemented in Scipy special functions. In my case, I wrote the algorithm in Numpy/python myself which was almost unusably slow. I then outsou…

And, unlike in many other languages, that MittagLeffler julia function can probably already work efficiently on special arrays (distributed arrays, GPU arrays, etc), is fussable into inner loop kernels, and supports automatic differentiation, without extra work from the author of the package.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#108

Post by dmw_ng » Sun, Oct 11, 2020, 7:39 PM UTC

StefanKarpinski wrote: post #99

dmw_ng wrote: post #51

There is a ton of conflation in this post, yesterdays post, and the comments. * Comparing parallel implementations (Julia) to single thread (Python csv module). This is still quite relevant because in places like Python, it is impossible to parallelize the csv module without essentially updating its native implementation. That's not true in other places, where some simple wrapper code may be all required to get that…

You may be missing the point of the comparison, which is this: to compare the performance of the de facto standard CSV parsers of each system when used the way that a typical user would use them, i.e. to parse a CSV file into a data frame. Regarding your specific points: 1. Yes, it's possible to rig up some way of parsing in parallel in Python, like splitting the CSV file into multiple files and then parsing them in…

Regarding 2), code like "for row in pandas.DataFrame(...).iterrows():" where the DataFrame was populated by some array-based reader is creating an indirection. Most good pandas code of course doesn't look like this, but it's regularly unavoidable, hence "There are use cases where either representation is preferred". Without specifying our use case, it is meaningless to talk about "faster" or "slower".

For 3) there inherently is no optimal solution to "a problem with large performance-correctness tradeoffs", additionally users do not care only about speed, for a huge number of applications, absolute reproduction of precision may be the difference between a good simulation and data science quackery. We saw elsewhere in the thread that Julia's float parsing code at least presently handles certain syntaxes incorrectly. Ignoring performance, this is criteria for deciding whether a solution is fit for purpose at all.

The overall point otherwise makes sense: most DS folk don't and maybe shouldn't care about this stuff, however this is HN, and elsewhere in the comments there are plenty of examples of people who do.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#109

Post by langitbiru » Sun, Oct 11, 2020, 7:40 PM UTC

dmos62 wrote: post #5

qaq wrote: post #2

Julia is a hidden gem Once ecosystem for web dev matures Julia will be the killer lang for building web apps

> Julia will be the killer lang for building web apps That would be fun, but Julia's community aren't web devs. Julia spawned around very specific needs of scientific computing, which is characterised by a short-running daemon or a script-type interpreter. A web server is a long-running process. Not knowledgeable enough about Julia to tell how it lends itself to server uses, but heard hearsay that it's problematic.

Seconded this. If you looked at the Julia website (https://julialang.org), in the Ecosystem section, they listed these things:

"Visualization, General Purpose, Data Science, Machine Learning, Scientific Domains, Parallel Computing."

They already positioned themselves in scientific computing battleground. Web development (or desktop programming) is not their focus (for better or worse).

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#110

Post by mlthoughts2018 » Sun, Oct 11, 2020, 7:45 PM UTC

StefanKarpinski wrote: post #105

mlthoughts2018 wrote: post #102

Earlier quoted context omitted.

A typical user will rarely ever use the built in CSV library in Python, and would choose among the many third party options for much greater performance, ease of use, parity with rectangular data structure tools like pandas, and so forth. Comparing anything against the built-in CSV reader in Python is pretty much an exercise in nothing. In 20 years of programming Python professionally, I have used the built in CSV pa…

The fact that the standard CSV reader is not good enough to actually use except in toy applications is itself an issue, frankly.

I disagree completely. It’s perfectly good enough for non-specialized use cases, which is exactly right for a stdlib offering. For anything else, factor it out as separate third party options (even if maintained by PSF itself) so users only install what they need for their special application.

One person may install something that’s blazingly fast. Someone else may install something because it can convert the data to an in memory parquet format, someone else may use something that can virtualize SQL queries over top of disk-backed csvs.

They should all be different for the nuance of each application.