Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

juliacomputing.com

Post reply on HN
Print view View original

131–140 of 236 posts

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#131

Post by anakaine »

dmw_ng wrote: post #51

There is a ton of conflation in this post, yesterdays post, and the comments. * Comparing parallel implementations (Julia) to single thread (Python csv module). This is still quite relevant because in places like Python, it is impossible to parallelize the csv module without essentially updating its native implementation. That's not true in other places, where some simple wrapper code may be all required to get that…

I'm surprised to have not seen a mention of the python library Dask in this thread. Whilst I agree that reading CSVs in Python could be a lot quicker, most people do tend to use third party libraries to do so. Dask is able to multi-core multi-thread almost all operations, including reading in a CSV. You can go from a Dask data frame to a pandas dataframe.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#132

Post by oizin »

nnm wrote: post #113

> The tools used for benchmarking were BenchmarkTools.jl for Julia, microbenchmark for R, and timeit for Python. Is this a meaningful comparison? These benchmarks repeatedly call the function (with same input file) many times and then take the average of the time spent. However, in real world, we only read a specific file ONCE -- if we call the csv reader several times, it is almost always for different files.

I think this is an important point, if you look at the warmup run on [1] the Julia CSV readers are slower than many of the R and python packages . It makes the claims of 10-20x faster quite disengenuous in my opinion. A lot of data science work will only read the file on once per session and what these benchmarks actually suggest for that sort of work is not reflected in the title.

[1] https://www.queryverse.org/benchmarks/

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#133

Post by lifthrasiir »

Sukera wrote: post #123
burntsushi wrote: post #77

I posted this comment on lobste.rs when this came up:[1] It kind of looks like Julia’s CSV parser is cheating: https://github.com/JuliaData/CSV.jl/blob/9f6ef108d195f85daa5... It’s doing parallel parsing, but I’m pretty sure their technique won’t work for all inputs. Namely, they try to hop around the CSV data and chunk it up, and then parse each chunk in a separate thread AIUI. But you can’t do this in general becaus…

I wouldn't call that cheating, how else would you multithread this? You basically have to chunk it up and try to find a boundary and from what I can tell, they try to find out if they're in a quoted block. If they are, they find the end of the quoted block (or EOF, which shouldn't happen => broken file) or ask the user to report the bug. This is more like failsaving, not cheating.

> [...] how else would you multithread this?

Use a fast "frontier" thread to determine quotes and split works at the safe boundary.

I'm personally okay with the speculative execution as long as its result is correct and I wouldn't consider it cheating. But it is concerning that the benchmark doesn't list all the caveats (as all proper benchmarks should do, like [1], which is incidentally written by who you are replying to). I can't even determine whether the machine is hitting I/O bound or not from the OP!

[1] https://blog.burntsushi.net/ripgrep/

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#134

Post by coliveira »

drej wrote: post #117
StefanKarpinski wrote: post #114

Earlier quoted context omitted.

Interestingly, CSV.jl can do all of those. This seems like a classic case of lack of composability in Python: you need a different CSV parser for each and every application even though the core logic of parsing the CSV format is the same. This is because anything fast has to be implemented in very concrete, specialized C code. In Julia, you can write something generic and let the compiler specialize it for you, so yo…

Lack of composability? What? Of course you can use one parser for everything, what’s stopping you from doing that. Separating IO from business logic and composing the two is a pretty standard practice. Not sure I follow the argument here.

The argument is simple: Python is by design made so that if you want things to go fast then you need to implement it in C. This creates a constant need to use specialized libraries (such as pandas), where in other languages they can simply do a better job in optimizing the standard library.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#135

Post by bonoboTP »

ddragon wrote: post #52
bonoboTP wrote: post #25

Earlier quoted context omitted.

> By that logic people should have kept using Fortran instead of Python There were very real pain points with Fortran, C++, MATLAB etc. compared to Python. Now this is hard for me to argue in a few sentences as it's a distillation of having worked with many languages, but I feel that Python strikes a great balance for productivity. It feels like pouring your thoughts directly onto the screen, with no need to fight th…

I feel like you considering Python as having few real pain points in data science as either lack of knowledge of other languages or imagination/ambition. I do work on Python for data science/engineering in a production environment and I do find many of those all the time. Python does not feel like pouring my thoughts because I cannot write Python directly without taking hours to handle a few million points of data, s…

> so I have to juggle with multiple dialects (pandas, numpy, pytorch, tensorflow)

NumPy should be enough for general computation. If you need autodiff or GPU then add in PyTorch. Pandas is more about various metadata than the actual numerical computing. If you want those types of features, the complexity doesn't disappear if you go to a different language.

There's an effect where a new generation of developers see complexity built by the earlier generation, say it's too complicated and mess up, we don't need all that, so start over clean and it all looks so easy. But it's deceptive, because it will get complicated again once you put in all the features but it will look familiar now to this generation of developers as they grow side-by-side with the new language/framework. After a few years the cycle repeats and a new generation says "what's all this mess, why do I need to juggle all this, I just need XY."

> Also Python for a dynamic language has pretty mediocre interactive story (using ptpython since the default repl is unusable), even simple things like copy pasting to a REPL can end up being a pain because of indenting, and the repl is far away from Lisp, Clojure and even Julia and Elixir.

Use Jupyter Notebooks (or IPython if you don't want to leave the shell).

> disappointed with Python's multithreading story

Thread pools (executors, futures etc.) and process pools (multiprocessing module) work quite nicely.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#136

Post by liuliu »

Sukera wrote: post #123
burntsushi wrote: post #77

I posted this comment on lobste.rs when this came up:[1] It kind of looks like Julia’s CSV parser is cheating: https://github.com/JuliaData/CSV.jl/blob/9f6ef108d195f85daa5... It’s doing parallel parsing, but I’m pretty sure their technique won’t work for all inputs. Namely, they try to hop around the CSV data and chunk it up, and then parse each chunk in a separate thread AIUI. But you can’t do this in general becaus…

I wouldn't call that cheating, how else would you multithread this? You basically have to chunk it up and try to find a boundary and from what I can tell, they try to find out if they're in a quoted block. If they are, they find the end of the quoted block (or EOF, which shouldn't happen => broken file) or ask the user to report the bug. This is more like failsaving, not cheating.

> how else would you multithread this?

Doing two pass parsing: https://liuliu.me/eyes/loading-csv-file-at-the-speed-limit-o...

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#137

Post by johnmyleswhite »

elcomet wrote: post #69
genomez4fun wrote: post #62

Earlier quoted context omitted.

julia is interpreted -- it's type system enables the speedups often presented in benchmarks.

It's a bit more complex. Both python and julia are compiled to bytecode. Then python bytecode gets interpreted, but I think that julia bytecode is actually compiled with a JIT compiler. So julia is not really interpreted.

What do you mean when you use the word "bytecode" in reference to Julia?

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#138

Post by Rotareti »

mlthoughts2018 wrote: post #103
nextos wrote: post #96

Earlier quoted context omitted.

I would also say, for those not familiar with Julia, that the main advantage of Julia vs Python or R is that you can write performant code in Julia that will be fast enough for most scenarios. For example, there are efforts to write a pure BLAS in Julia that is still performant [1]. If you are into numerical computing, you will quickly understand this is crazy cool. A consequence of that is composability. Most librar…

This is also very easy to achieve in Python by using Cython to selectively optimize code. That way for the 99% of code where such optimizations are meaningless and their requirement for static typing is a liability, you can ignore them, and only focus on the 1% of use cases where it matters.

> This is also very easy to achieve in Python by using Cython to selectively optimize code

There is also mypyc [0] coming up on the horizon.

https://github.com/python/mypy/tree/master/mypyc

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#139

Post by formerly_proven »

StefanKarpinski wrote: post #99
dmw_ng wrote: post #51

There is a ton of conflation in this post, yesterdays post, and the comments. * Comparing parallel implementations (Julia) to single thread (Python csv module). This is still quite relevant because in places like Python, it is impossible to parallelize the csv module without essentially updating its native implementation. That's not true in other places, where some simple wrapper code may be all required to get that…

You may be missing the point of the comparison, which is this: to compare the performance of the de facto standard CSV parsers of each system when used the way that a typical user would use them, i.e. to parse a CSV file into a data frame. Regarding your specific points: 1. Yes, it's possible to rig up some way of parsing in parallel in Python, like splitting the CSV file into multiple files and then parsing them in…

> If float parsing is slow, make it faster.

Use a binary data format or hexfloats instead. Decimal floats are a pain and especially pointless if they are being generated from and converted back to IEEE 754.

Re: CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

#140

Post by orbifold »

StefanKarpinski wrote: post #99
dmw_ng wrote: post #51

There is a ton of conflation in this post, yesterdays post, and the comments. * Comparing parallel implementations (Julia) to single thread (Python csv module). This is still quite relevant because in places like Python, it is impossible to parallelize the csv module without essentially updating its native implementation. That's not true in other places, where some simple wrapper code may be all required to get that…

You may be missing the point of the comparison, which is this: to compare the performance of the de facto standard CSV parsers of each system when used the way that a typical user would use them, i.e. to parse a CSV file into a data frame. Regarding your specific points: 1. Yes, it's possible to rig up some way of parsing in parallel in Python, like splitting the CSV file into multiple files and then parsing them in…

I installed CSV.jl and tried it out on the insurance dataset from here "https://support.spatialkey.com/spatialkey-sample-csv-data/".

First import took >10s just to precompile and then CSV.File took 5.21s to read the file. A fresh pandas install takes ~10s to import and then reads this file in 81.2 ms ± 1.25 ms. CSV.jl takes 38 ms on a second run so it is twice as fast, however especially to someone that just starts using Julia, these >100x slowdowns constantly happen. Especially when using libraries that heavily rely on multiple dispatch specialisation for performance.

Post reply on HN