Live data from Hacker News

Why I Use Nim instead of Python for Data Processing

benjamindlee.com

121–130 of 183 posts

Re: Why I Use Nim instead of Python for Data Processing

#121
post #114

Earlier quoted context omitted.

and with style insensitivity they become a non-issue?

The compiler might not be sensitive to style, but *I* am.

I guess the best thing would be having a formatter that can switch the style for you locally. I don’t think Nim has this currently, would be neat if they did.

Re: Why I Use Nim instead of Python for Data Processing

#122
post #14

Earlier quoted context omitted.

> It's primarily a testament to how simply mind bogglingly slow Python is outside of its optimised numerical science ecosystem. From my experience in using Python at my last job, I'll also add that Python is decent at tasks that aren't CPU-bound. I wrote a lot of scripts that polled large amounts of network devices for information and then did something with it (typically upsert the data into a database, either via d…

I agree with your entire post but , and I‘m saying this as a fulltime python dev, there‘s often a point where it starts being bothersome, and that usually comes only later in the lifecycle of an application after it had some organic growth. Some day e.g. a sales manager comes down to your lair and asks you if you couldn‘t just also parse this little 200MB Excel spreadsheet after it came over the network such that you…

> there‘s often a point where it starts being bothersome

My team and I have been using Python for web, scripting, and ETL development since 2007. I don't recall the last time Python wasn't "fast enough" for anything I needed to do. I'm sure it's legitimately too slow for plenty of use cases and classes of programming domains. But for a general purpose language that makes our developers incredibly productive (which is what we optimize for organizationally), I've not "often" found the point where it becomes bothersome. On the contrary, I'd say I've rarely found it. And in those cases, the workaround to make it fast enough is there.

That's not to dispute your experience. I just want to provide a counter example.

Re: Why I Use Nim instead of Python for Data Processing

#123
post #104

Earlier quoted context omitted.

> It’s intent is to allow a given codebase to maintain a consistent style (eg camel vs snake) even when making use of upstream libraries that use different styles. This doesn't make sense. For an entirely new language you can just have the entire ecosystem use the same style, e.g. like Rust does. Or even Python!

It actually makes a lot of sense. I personally don't want to touch .Net langs because of their PascalCase and camelCase coding style which I find very unfortunate arbitrary decision. And well, didn't really want to get into Nim because of camelCase but since I've learned about this feature I'm reconsidering again.

Do you really dislike camelCase that much to avoid an _entire_ ecosystem because of it?

I prefer snake_case too, but avoiding a language because of a minor convention like that seems weird.

Re: Why I Use Nim instead of Python for Data Processing

#124
post #104

Earlier quoted context omitted.

It actually makes a lot of sense. I personally don't want to touch .Net langs because of their PascalCase and camelCase coding style which I find very unfortunate arbitrary decision. And well, didn't really want to get into Nim because of camelCase but since I've learned about this feature I'm reconsidering again.

Do you really dislike camelCase that much to avoid an _entire_ ecosystem because of it? I prefer snake_case too, but avoiding a language because of a minor convention like that seems weird.

When there are so many excellent languages to choose among, a trivial distinction can almost be a relief.

Re: Why I Use Nim instead of Python for Data Processing

#125

I don't doubt Nim, looks like a great language. But that is just an awful Python implementation. I'd do it in this way: lines = (line for line in lines("orthocoronavirinae.fasta") if not line.startswith(">")) gc_lines = (1 if ('G' in line or 'C' in line) else 0 for line in lines) gc = sum(gc_lines) total = len(list(gc_lines)) # Alternatively, a more "memory efficient" total would be: total = sum(1 for _ in lines) Edi…

Given that there is a fixed set of Alphabets, I would rather use Counters. It avoids for loops (the biggest source of non-efficiency in python) and is imho is much more idiomatic.

Re: Why I Use Nim instead of Python for Data Processing

#126
post #8

It's primarily a testament to how simply mind bogglingly slow Python is outside of its optimised numerical science ecosystem. Which also why I don't use it that much, because while numerical analysis is a big part of what I do, so is what I would call "symbolic manipulation" and unless you go to quite some effort to transform every problem into a numerical one, Python is just awful at that. But Nim is only one of a w…

> It's primarily a testament to how simply mind bogglingly slow Python is outside of its optimised numerical science ecosystem.

CPython's slowness doesn't boggle my mind at all. It's a bytecode interpreter for an incredibly dynamic language that states simplicity of implementation as a goal. I would say performance is actually pretty impressive considering all that. What _does_ boggle my mind is the performance of cutting-edge optimizing compilers like LLVM and V8!

At least there is a benefit to a simple implementation: Someone like me can dive into CPython's source and find out how things work.

Re: Why I Use Nim instead of Python for Data Processing

#127
post #82

Earlier quoted context omitted.

You have to be reaaaaaally slow to be beaten by a network. Also, touched by the OP, if it takes 3 hours to write some code that in Python takes only 1, or if the compile times are huge, Python can beat other languages in speed (that edge fades when the same program is used over and over again). But as demonstrated, Nim is fast to write and fast to compile, so Python has little edge. Just it's huge ecosystem.

> Just it's huge ecosystem. self-contradiction at its best. kindly be reminded the same advantage was the only thing that kept Java alive for so long until it finally started to enter the XXI century a couple years ago. you just can't discount an ecosystem, especially if its huge.

Exactly. Also I’m not impressed with a lot of the ecosystem. After getting more acquainted with many very important libraries I can see the poor quality and it’s scary. Grateful for the free tools but IMO there other Languages like go where more essential libraries and tooling are written by better programmers. Like actual professionals instead of hobby tier

Re: Why I Use Nim instead of Python for Data Processing

#128
I notice that python has `rstrip` while `nim` doesn't. Python's `rstrip()` is allocating a whole new line there. Does the nim iterator skip over whitespace or something? Do those bits of code output the same thing? The presence or absence of the whitespace will affect the `total` count.

Re: Why I Use Nim instead of Python for Data Processing

#129

One pitfall that I ran into when trying out Nim for scientific computing is that Nim follows more a computer science convention than mathematics convention for exponentiation and negation operators. That is in Nim -2^3=(-2)^3 unlike more scientific computing oriented languages where -2^3=-(2^3). To someone like myself who mainly does scientific work this was quite unexpected and it causes a surprising amount of menta…

Although Nim using weird order of operations is unfortunate, your example is not well chosen. Replace the exponent 3 with an even integer and your point will be clear.

Maybe not that weird, although not the most common syntax. In Nim 0 - 2^2 evaluates to -4, while -2^2 evaluates to +4. So in -2, the - is treated as a unary minus and binds tightly to the 2. It would be bad if Nim were doing + or - before ^, or before *. I have a feeling some other languages treat the unary minus the same way, but don’t know of any examples off hand.

Re: Why I Use Nim instead of Python for Data Processing

#130

One pitfall that I ran into when trying out Nim for scientific computing is that Nim follows more a computer science convention than mathematics convention for exponentiation and negation operators. That is in Nim -2^3=(-2)^3 unlike more scientific computing oriented languages where -2^3=-(2^3). To someone like myself who mainly does scientific work this was quite unexpected and it causes a surprising amount of menta…

I don't think that is a mathematics convention vs computer science convention thing. Most languages, whether aimed at mathematics or computer science, have exponentiation at higher precedence than unary minus. (JavaScript does not, but it also does not allow unary minus directly in front of the base of an exponentiation so you have to add parenthesis no matter which interpretation you want).

The main places you find it the other way are spreadsheets and shells.

Is there an explanation from the Nim authors as to why they made such an odd choice?

Post reply on HN