Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Why I Use Nim instead of Python for Data Processing

benjamindlee.com

Post reply on HN
Print view View original

111–120 of 183 posts

Re: Why I Use Nim instead of Python for Data Processing

#111

Post by aEverr »

stavros wrote: post #96
IshKebab wrote: post #88

Earlier quoted context omitted.

> It’s intent is to allow a given codebase to maintain a consistent style (eg camel vs snake) even when making use of upstream libraries that use different styles. This doesn't make sense. For an entirely new language you can just have the entire ecosystem use the same style, e.g. like Rust does. Or even Python!

I'd agree, if you want a consistent style, ship a formatter with your package manager, like Rust does with `cargo fmt`. If it's there by default, it'll quickly be a standard.

there are many packages and maintainers of such who just dont give a damn about the formatter or the style, some people just *want what they want*

Re: Why I Use Nim instead of Python for Data Processing

#112

Post by stavros »

aEverr wrote: post #111
stavros wrote: post #96

Earlier quoted context omitted.

I'd agree, if you want a consistent style, ship a formatter with your package manager, like Rust does with `cargo fmt`. If it's there by default, it'll quickly be a standard.

there are many packages and maintainers of such who just dont give a damn about the formatter or the style, some people just * want what they want*

Sure, but there's a standard formatter, they become a small minority.

Re: Why I Use Nim instead of Python for Data Processing

#113

Post by leephillips »

cycomanic wrote: post #34

One pitfall that I ran into when trying out Nim for scientific computing is that Nim follows more a computer science convention than mathematics convention for exponentiation and negation operators. That is in Nim -2^3=(-2)^3 unlike more scientific computing oriented languages where -2^3=-(2^3). To someone like myself who mainly does scientific work this was quite unexpected and it causes a surprising amount of menta…

Although Nim using weird order of operations is unfortunate, your example is not well chosen. Replace the exponent 3 with an even integer and your point will be clear.

Re: Why I Use Nim instead of Python for Data Processing

#114

Post by aEverr »

stavros wrote: post #112
aEverr wrote: post #111

Earlier quoted context omitted.

there are many packages and maintainers of such who just dont give a damn about the formatter or the style, some people just * want what they want*

Sure, but there's a standard formatter, they become a small minority.

and with style insensitivity they become a non-issue?

Re: Why I Use Nim instead of Python for Data Processing

#115

Post by jhgb »

_dain_ wrote: post #84
santiagobasulto wrote: post #79

Earlier quoted context omitted.

True, you'd need to re create the generator expression. Still, the other implementation seemed too naive.

There's probably some itertools trick to do it with only one iteration. EDIT: you can do it with functools.reduce and a generator of tuples: from functools import reduce with open("orthocoronavirinae.fasta") as f: lines = (line.strip() for line in f if not line.startswith(">")) sums = ((len(line), sum(1 for ch in line if ch in "CG")) for line in lines) total, gc = reduce(lambda x, y: (x[0] + y[0], x[1] + y[1]), sums)…

> There's probably some itertools trick to do it with only one iteration.

Or an arithmetic trick. Since there's the `1 if ('G' in line or 'C' in line) else 0 for line in lines` construct in the code, if you simply replace 1 with (len(line)<<K)+1 and 0 with len(line)<<K (for a suitable value of K), you can extract the two results using div/mod.

Re: Why I Use Nim instead of Python for Data Processing

#116

Post by stavros »

aEverr wrote: post #114
stavros wrote: post #112

Earlier quoted context omitted.

Sure, but there's a standard formatter, they become a small minority.

and with style insensitivity they become a non-issue?

The compiler might not be sensitive to style, but *I* am.

Re: Why I Use Nim instead of Python for Data Processing

#117

Post by Jensson »

CraigJPerry wrote: post #81
zmmmmm wrote: post #8

It's primarily a testament to how simply mind bogglingly slow Python is outside of its optimised numerical science ecosystem. Which also why I don't use it that much, because while numerical analysis is a big part of what I do, so is what I would call "symbolic manipulation" and unless you go to quite some effort to transform every problem into a numerical one, Python is just awful at that. But Nim is only one of a w…

The thing with Python is it's usually pretty easy to optimise quite impressively. E.g. random example: Sprinkle some cdef's in your python and suddenly you're faster than c++ https://github.com/luizsol/PrimesResult https://github.com/PlummersSoftwareLLC/Primes/blob/drag-race... 25.8 seconds down to 1.5

I've tried cython and it isn't competitive with C in real world cases. Cherry picking a case where it is competitive doesn't help your case. Its performance tend to be around the level of Java, another language people say is competitive or faster than C++ but in practice C++ is still twice faster than Java in most cases.

Still, getting Java level performance out of python is a huge improvement and should be enough for most cases.

Re: Why I Use Nim instead of Python for Data Processing

#118

Post by kgeist »

_dain_ wrote: post #100
sound1 wrote: post #93

Earlier quoted context omitted.

> However, indeed, if you are to choose between, for example, Nim and Go for a new project, then I am not sure why would anyone prefer Nim. I'm really interested to know. Same here, curious to know what HN crowd recommends between Nim vs Go for new projects.

Nim has hygienic AST macros, so it has really good metaprogramming capabilities. An example of the sort of thing you can do with it: use pattern matching to implement a functional programming DSL: https://nim-lang.org/blog/2021/03/10/fusion-and-pattern-matc... This makes it really easy to reduce boilerplate and create low- or 0-cost abstractions for your problem domain. Example of this done in a microcontroller proje…

>use pattern matching to implement a functional programming DSL

DSLs have some use cases but generally I'd avoid being too clever and creating basically entire sublanguages for the sake of it - unless I really need to. Most projects don't need it, it's harder to pick up for novices (especially if there's a zoo of DSLs, different between projects). In Go the last resort is usually code generation, there are built-in tools for it.

>Async/await is also implemented by metaprogramming, rather than as a "core" part of the language

I guess most programmers don't really care if a feature is implemented in the language or in the library, provided it's easy to use. In Go, it's not extendable however; but so far, I've never had a need to extend the default goroutine scheduler.

>Unrelated to the above, Nim also compiles to Javascript. So you can use the same language for both the backend and frontend.

Go can be compiled to WASM, but practically I've never seen code that could be reused between backend/frontend because they solve different problems using different principles, so the idea of using one language for frontend/backend was never compelling to me. Maybe it's just me.

Re: Why I Use Nim instead of Python for Data Processing

#119

Post by DeathArrow »

sound1 wrote: post #93
kgeist wrote: post #57

Earlier quoted context omitted.

>if we want to start a new project how could we find programmers well-versed in Nim? If a programmer can't pick up a language like Nim in a few weekends (from what I gather, it's similar to Python and not much different from most common languages, i.e. not something relatively exotic like Haskell) then I don't know. Our mainly PHP shop transitioned to Go quite effortlessly. Today we hire PHP juniors without any Go ex…

> However, indeed, if you are to choose between, for example, Nim and Go for a new project, then I am not sure why would anyone prefer Nim. I'm really interested to know. Same here, curious to know what HN crowd recommends between Nim vs Go for new projects.

Nim is also faster than Go if we look at most benchmarks. Nim's memory management can be tuned quite a bit. Garbage collector can be even turned off. You can better tune to the underlying hardware if you need to.

https://nim-lang.org/docs/gc.html

Re: Why I Use Nim instead of Python for Data Processing

#120

Post by [deleted] »

ad404b8a372f2b9 wrote: post #108
santiagobasulto wrote: post #37

I don't doubt Nim, looks like a great language. But that is just an awful Python implementation. I'd do it in this way: lines = (line for line in lines("orthocoronavirinae.fasta") if not line.startswith(">")) gc_lines = (1 if ('G' in line or 'C' in line) else 0 for line in lines) gc = sum(gc_lines) total = len(list(gc_lines)) # Alternatively, a more "memory efficient" total would be: total = sum(1 for _ in lines) Edi…

That's not good python, it's unreadable and less optimized than the original code. Your "1 if ... else 0" condition serves no purpose and you iterate twice over the same loop for no reason. Use normal loops and save everyone who comes after a lot of reading time and bugs.

[deleted]
Post reply on HN