Live data from Hacker News

Show HN: Nimic – Pure Python as a systems language with AOT compilation

github.com

21–30 of 31 posts

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#21
post #11

Earlier quoted context omitted.

True, nimic is a statically typed Python subset without much of its dynamism with the aim to be foremost an efficient systems language. Though in nimic, without using isinstance, the instance-based dispatch is realised via variant types. Yes, emulating the Nim constructions in Python was the hardest part, while making the transpiler was straightforward. Indeed, the AOT compilation leads to great speed-ups for heavy c…

the variant-types-for-dispatch thing is a nice tradeoff, makes sense. honestly the strongest pitch in there is the "write the hot module in nimic instead of dropping to c/rust/cython" angle, which is basically the cython niche. so the question is what nimic adds over cython for that. my guess is dual-mode: a .pyx isn't valid python and won't run under plain cpython, but nimic stays importable and debuggable in cpytho…

> how do you handle that

It looks like (please correct me, OP, if I'm wrong) it works the other way around: you use sized int types in Nimic code, and their semantics are emulated on the Python side. See here: https://github.com/dima-quant/nimic/blob/main/src/nimic/ntyp...

So I'd say in Nimic Python you get Nim-style integer emulation when run from Python, keeping both paths consistent with each other - but breaking consistency with the rest of Python. Which is OK, I think, given it's explicitly a subset(s) of the language(s). It would be possible to make `int` transpile to some BigInt Nim implementation, but you'd need an external dependency for this, as they are not in Nim's stdlib. However, in the speed-focused context, I'm not sure if defaulting to BigInt every time the compiler sees an `: int` annotation would work well. It's a hard decision to make. Curious what's the OP opinion here?

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#22
post #7

The line I'd push on is "valid Python that runs unmodified in CPython." True at the syntax level, but the speedup isn't really coming from Nim as a backend, it comes from how much of Python's dynamism you can pin down at compile time. Refcounting semantics, __getattr__, values that change type at runtime, isinstance-based dispatch, monkeypatching in tests: the moment you statically commit to any of those, you've defi…

Curious why your comment below’s got flagged. I personally don’t see any issue with it.

It happens; sometimes it's a misclick, sometimes some automated filter. You can 'vouch' for such comments if you go into that comment's details: https://news.ycombinator.com/item?id=48671561 - in this case, a combination of a new account and maybe posting too quickly could have caused it. I vouched for it, and it should be visible now.

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#23
post #7

The line I'd push on is "valid Python that runs unmodified in CPython." True at the syntax level, but the speedup isn't really coming from Nim as a backend, it comes from how much of Python's dynamism you can pin down at compile time. Refcounting semantics, __getattr__, values that change type at runtime, isinstance-based dispatch, monkeypatching in tests: the moment you statically commit to any of those, you've defi…

Cython is the gold reference standard. There is no wall. You can selectively add type annotations and get optimized hot paths, or not and still get PyObject code that bypasses the interpreter for a free speedup.

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#24

nimic is a lightweight pure Python package that emulates Nim types and constructions, making it straightforward to transpile to Nim and compile AOT. Key principle: nimic code is valid Python that runs unmodified in CPython and also transpiles to equivalent Nim code. Because nimic code is just standard Python with type hints and ctypes shims, it is a fully valid CPython script, so you can use the Python REPL during de…

If it's possible and you intended for it to be a long-term project, I would suggest looking at designing an MIR for the Python to lower to and then lowering it to Nim, and that is probably the most valuable feature you can add to Nimic. That's what Rust had to learn the hard way, and that's how LLVM works, and having an intermediate representation means your compiler is going run into a lot less edge cases during development.

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#25
post #9

Earlier quoted context omitted.

so it is what https://github.com/mypyc/mypyc was supposed to be ... i can just use write my python code add some type hints and it should run faster ... like cython but without learning a new dsl?

Yeah, indeed, I actually read about mypyc a few years ago before starting nimic. Though the goal is very similar, mypyc does not support compilation to a native executable and the performance gains were somehow limited to 2x-4x, as reported in https://sichard.ca/blog/2022/05/compiling-black-with-mypyc-p...

[deleted]

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#26
post #11

Earlier quoted context omitted.

the variant-types-for-dispatch thing is a nice tradeoff, makes sense. honestly the strongest pitch in there is the "write the hot module in nimic instead of dropping to c/rust/cython" angle, which is basically the cython niche. so the question is what nimic adds over cython for that. my guess is dual-mode: a .pyx isn't valid python and won't run under plain cpython, but nimic stays importable and debuggable in cpytho…

> how do you handle that It looks like (please correct me, OP, if I'm wrong) it works the other way around: you use sized int types in Nimic code, and their semantics are emulated on the Python side. See here: https://github.com/dima-quant/nimic/blob/main/src/nimic/ntyp... So I'd say in Nimic Python you get Nim-style integer emulation when run from Python, keeping both paths consistent with each other - but breaking…

Your explanation is completely correct. There are Nim implementations of BigInt, but the foremost focus of Nimic is enabling as many standard Nim features as possible. In general, there is another parallel stream, which aims at implementing many Python features in Nim (both approaches can be combined, extending the Python subset covered by Nim implementation): https://github.com/nimpylib/nimpylib/wiki

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#27

nimic is a lightweight pure Python package that emulates Nim types and constructions, making it straightforward to transpile to Nim and compile AOT. Key principle: nimic code is valid Python that runs unmodified in CPython and also transpiles to equivalent Nim code. Because nimic code is just standard Python with type hints and ctypes shims, it is a fully valid CPython script, so you can use the Python REPL during de…

If it's possible and you intended for it to be a long-term project, I would suggest looking at designing an MIR for the Python to lower to and then lowering it to Nim, and that is probably the most valuable feature you can add to Nimic. That's what Rust had to learn the hard way, and that's how LLVM works, and having an intermediate representation means your compiler is going run into a lot less edge cases during dev…

Currently, Nim fits quite well with Python but there is a project to directly emit LLVM IR from Nim and MIR might be relevant there: https://github.com/arnetheduck/nlvm

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#28
post #7

The line I'd push on is "valid Python that runs unmodified in CPython." True at the syntax level, but the speedup isn't really coming from Nim as a backend, it comes from how much of Python's dynamism you can pin down at compile time. Refcounting semantics, __getattr__, values that change type at runtime, isinstance-based dispatch, monkeypatching in tests: the moment you statically commit to any of those, you've defi…

Cython is the gold reference standard. There is no wall. You can selectively add type annotations and get optimized hot paths, or not and still get PyObject code that bypasses the interpreter for a free speedup.

The comparison with Cython is indeed very relevant as it is currently the standard for making Python modules and it has greatly improved support for the standard Python syntax over the last few years, e.g. the type hints. However, it still can not produce an executable independent from the Python runtime. Besides the manual memory management available in Cython, Nimic also provides deterministic memory management (ARC/ORC) rather than the Python's reference counting for the compiled code. Nim intermediate code is easier to debug than Cython produced C and the code can also compile to LLVM IR. I'm not an expert on Cython, does it have the modern systems language functionality at level it is present in Nim?

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#29

Earlier quoted context omitted.

If it's possible and you intended for it to be a long-term project, I would suggest looking at designing an MIR for the Python to lower to and then lowering it to Nim, and that is probably the most valuable feature you can add to Nimic. That's what Rust had to learn the hard way, and that's how LLVM works, and having an intermediate representation means your compiler is going run into a lot less edge cases during dev…

Currently, Nim fits quite well with Python but there is a project to directly emit LLVM IR from Nim and MIR might be relevant there: https://github.com/arnetheduck/nlvm

Oh, I wasn't referring to emitting LLVM IR directly from Nim, but that you should consider adding an MIR layer between your Nimic subset of Python and Nim instead of emitting Nim from Python directly. Otherwise, compilation would get difficult and adding new supported Python will get painful.

Re: Show HN: Nimic – Pure Python as a systems language with AOT compilation

#30
post #11

Earlier quoted context omitted.

True, nimic is a statically typed Python subset without much of its dynamism with the aim to be foremost an efficient systems language. Though in nimic, without using isinstance, the instance-based dispatch is realised via variant types. Yes, emulating the Nim constructions in Python was the hardest part, while making the transpiler was straightforward. Indeed, the AOT compilation leads to great speed-ups for heavy c…

the variant-types-for-dispatch thing is a nice tradeoff, makes sense. honestly the strongest pitch in there is the "write the hot module in nimic instead of dropping to c/rust/cython" angle, which is basically the cython niche. so the question is what nimic adds over cython for that. my guess is dual-mode: a .pyx isn't valid python and won't run under plain cpython, but nimic stays importable and debuggable in cpytho…

indeed, as far as I know, at this moment nimic is the only package that provides systems language functionality running in CPython, while the corresponding code also compiles AOT to an efficient native binary.
Post reply on HN