Show HN: Lucen a Python compiler that parallelizes for-loops via comment pragmas • Hacker News Board

Re: Show HN: Lucen a Python compiler that parallelizes for-loops via comment pragmas

#21

Post by [deleted] » Tue, Aug 04, 2026, 6:07 AM UTC

zbentley wrote: post #19

soumik15630m wrote: post #14

Earlier quoted context omitted.

Sure man thanks for the critique, genuinely.... To be precise..the core concurrency shapes, disjoint-chunk commit ordering and DAG-level wavefront scheduling, are model-checked (SequentialEquivalence; DependencySafety + Termination), continuously re-verified in CI via a second independent executable encoding. The TLA+ file itself is bounded to a small instance and a 2-chunk model... the paired Python checker checks a…

I do appreciate the rigor that was included, and please don't take my comment above as saying "this is a bad project"; rather, it's not a project I personally am comfortable using at this time due to the combination of its purpose, maturity, and development methodology. Formal verification is great, and I wish more authors included TLA+. However, changes to another platform's semantics are difficult to formally verif…

[deleted]

Re: Show HN: Lucen a Python compiler that parallelizes for-loops via comment pragmas

#22

Post by soumik15630m » Tue, Aug 04, 2026, 6:13 AM UTC

zbentley wrote: post #19

soumik15630m wrote: post #14

Earlier quoted context omitted.

Sure man thanks for the critique, genuinely.... To be precise..the core concurrency shapes, disjoint-chunk commit ordering and DAG-level wavefront scheduling, are model-checked (SequentialEquivalence; DependencySafety + Termination), continuously re-verified in CI via a second independent executable encoding. The TLA+ file itself is bounded to a small instance and a 2-chunk model... the paired Python checker checks a…

I do appreciate the rigor that was included, and please don't take my comment above as saying "this is a bad project"; rather, it's not a project I personally am comfortable using at this time due to the combination of its purpose, maturity, and development methodology. Formal verification is great, and I wish more authors included TLA+. However, changes to another platform's semantics are difficult to formally verif…

You were right on both of the issues you seemed unsure..

The issues you flagged are closed by version 1.1.1 - also mentioned in issue threads.

On the codec suggestion ... it seems to go against my design motivation ...A file with "# -- coding: lucen --" doesn't degrade to native python when Lucen isn't installed, it fails to compile: "SyntaxError: encoding problem: lucen", which fails the property that a file with lucen absent will run as native python. But the import order problem is real so i'm investigating about a .pth based hook installation at interpreter startup ...

Threads only i'm not taking as on a GIL build thread cannot spedup cpu bound python.

Thanks for all the suggestions tho..

Re: Show HN: Lucen a Python compiler that parallelizes for-loops via comment pragmas

#23

Post by zbentley » Wed, Aug 05, 2026, 1:31 PM UTC

soumik15630m wrote: post #22

zbentley wrote: post #19

Earlier quoted context omitted.

I do appreciate the rigor that was included, and please don't take my comment above as saying "this is a bad project"; rather, it's not a project I personally am comfortable using at this time due to the combination of its purpose, maturity, and development methodology. Formal verification is great, and I wish more authors included TLA+. However, changes to another platform's semantics are difficult to formally verif…

You were right on both of the issues you seemed unsure.. The issues you flagged are closed by version 1.1.1 - also mentioned in issue threads. On the codec suggestion ... it seems to go against my design motivation ...A file with "# -- coding: lucen --" doesn't degrade to native python when Lucen isn't installed, it fails to compile: "SyntaxError: encoding problem: lucen", which fails the property that a file with lu…

Thanks, I saw the pings on github! Definitely let me know if any of the additional things I mentioned are worth my doing some testing or filing issues/feature requests, always happy to help out.

Re: the codec option, that's fair; I hadn't thought of that angle. I think your options are 1) stick with what you have, 2) use a codec and require anyone using the codec have Lucen installed, but have it disabled by default, or 3) add a runtime component (like a "with require_lucen_to_be_imported_before_this_code_was_compiled()" context manager or something) that emits an error or warning when someone's import of Lucen happened too late.

Re: threads, that's a fair point about the GIL, but I really think that there are a lot of unappreciated drawbacks to the process model. Just off the top of my head: depending on your OS, the multiprocessing backend used inside ProcessPoolExecutor may reinitialize/reimport all your Python code in the worker, throwing off your profitability analysis due to an extremely expensive startup and causing the memory consumption of Lucen'd code to go crazy; ProcessPoolExecutor suddenly requires everything to be pickleable, which lots of things aren't; PPE can leave orphaned or hung/stuck processes across the stack; extra diligence is required when surfacing exceptions back up from the PPE due to pickleability of the exception stack and everything on it; many many more.

Re: Show HN: Lucen a Python compiler that parallelizes for-loops via comment pragmas

#24

Post by soumik15630m » Tue, Aug 11, 2026, 6:33 AM UTC

zbentley wrote: post #23

soumik15630m wrote: post #22

Earlier quoted context omitted.

You were right on both of the issues you seemed unsure.. The issues you flagged are closed by version 1.1.1 - also mentioned in issue threads. On the codec suggestion ... it seems to go against my design motivation ...A file with "# -- coding: lucen --" doesn't degrade to native python when Lucen isn't installed, it fails to compile: "SyntaxError: encoding problem: lucen", which fails the property that a file with lu…

Thanks, I saw the pings on github! Definitely let me know if any of the additional things I mentioned are worth my doing some testing or filing issues/feature requests, always happy to help out. Re: the codec option, that's fair; I hadn't thought of that angle. I think your options are 1) stick with what you have, 2) use a codec and require anyone using the codec have Lucen installed, but have it disabled by default,…

thanks again! will consider these suggestions ... and also feel free to open issues and pr in the repo if you're interested