CSV Reader Benchmarks: Julia Reads CSVs 10-20x Faster than Python and R

- Julia lures newcomers by the way of attractive marketing (At one point in my career, I was such a huge fan of Julia that I dismissed many people and their concerns. It was like I was following a cult). This is pre v0.4 days.

- Too much focus on optimization. As a seasoned engineer, speed has been less and less of a concern where primary packages that need to be fast in Python are already quite fast. Majority of the tasks most engineers (not academicians) do is plumbing and connecting various pieces together. Julia just isn't good at this and perhaps it will get better.

- Julia markets itself boldly as a general purpose language but doesn't have a good systems library, no real competitive webframework and many things that are necessary for a true general purpose language - for e.g. GUI. At best, it is a scientific computing language such as MATLAB but free (that is a huge thing for Academicians but not for engineers working in Top 500 companies). At worse, it is lacking features and documentation that MATLAB has.

- Julia isn't humble in its approach towards marketing. It over-extends and over-reaches. This leads to people like myself lured into it and then completely burned by the experience. For e.g., it flaunts about being able to see the inline assembly instructions - in my entire 15 year career as a software engineer, I've never had the need to do this. It is almost a meaningless feature for vast majority of the users, but marketing wise it is pretty attractive.

- Julia recruits inexperienced software engineers and non-software "script junkies". It has lost its vision and original premise since 0.4v I feel like. Syntax has gotten worse. Bad decisions have been made. I think the founders of Julia are very very smart folks, it's just the community that is full of people who haven't built anything remotely large and of medium complexity. Are there exceptions? Yes, Julia has been adopted in a few large scale projects but I dispute whether that was a good decision.

Note: My experience is based on engineering mindset, not academics and research. Julia would probably be pretty good on a professor's macintosh and can gain wide adoption in those areas.

[1] https://julialang.org/benchmarks/

In other words, there is an even crankier and older community for which Julia may be the first actual change in a very long time.

This is particularly the case in HPC, where everyone still uses Fortran, C, or C++, and has never moved nor ever will move to Python because Python is fundamentally unsuited to these workloads. But in some cases, Julia is [1].

The best differential equation solvers (e.g., SUNDIALS [2] for a modern example) have been written in FORTRAN for the last 50 years. If Julia can challenge Fortran as the go-to language for this type of work (e.g. DifferentialEquations.jl [3] and SciML), that would hardly count as excessive churn.

[1] https://github.com/jeff-regier/Celeste.jl

[2] https://computing.llnl.gov/projects/sundials

[3] https://github.com/SciML/DifferentialEquations.jl

What does "hackers" here even mean? Is it just a throw-away disparaging term?

Hacker has at least three distinct meanings: deeply knowledgeable software developers, shallowly knowledgeable programmers, and people who break into computer security systems.

I assume you are not using the first of these, else you would likely say the Julia developers are hackers too. And you are definitely not using the third.

van Rossum's training, for example, came through the language design and implementation process for ABC.

Tim Peters, another early designer, worked on compilers for Kendall Square Research and Cray Research supercomputers. (Eg, https://bugs.python.org/msg303574 describes developing libm for KSR, and http://stackless.com/coroutines.tim.peters.html shows his knowledge of features of Icon and Simula 67.)

So already there it was a multi-disciplinary effort.

What don't you say Python was designed for an era of single-core computing and for tasks where raw performance took second place to usability instead of going hand-in-hand? Seems rather more correct and less needlessly antagonistic.

But you seem rather fond of needless antagonisms, as people certainly do write "real" programs all in Python.

Regarding your specific points:

1. Yes, it's possible to rig up some way of parsing in parallel in Python, like splitting the CSV file into multiple files and then parsing them in separate processes and then joining that. But no matter how you do it, you have to jump through extra hoops to do it. And that is not something most users do.

2. I don't entirely follow this point. Perhaps using PyArrow's parser would be faster than what is timed here, but is that what the typical Python data science user would do? Most Python users use Pandas and the CSV parsing that it provides. If that changes, then it would be good to do a new benchmark.

3. Yes, floating-point decoding is hard, but Julia's is fully accurate, so if it's also faster than Python's how is it an unfair comparison? If Julia was cheating by doing imprecise float parsing, sure, that would be a valid complaint, but it's not.

These "conflations" sound suspiciously like excuses. If float parsing is slow, make it faster. If concrete object decoding is slow, stop doing that. If threading would be faster, bite the bullet and use threading. If the language doesn't provide good support for these things, then maybe the language is at fault.