X rewrote Y in Rust is the new “Blazing fast Y written in Rust”.
no, its "rewrote x in rust and now it's blazing fast". I still dont know fast "blazing" is. 10x? 100x? any increase at all?
Fable Converted Pylint to Rust
101–110 of 122 posts
Re: Fable Converted Pylint to Rust
#102Earlier quoted context omitted.
Making something 50-2000x faster is pointless? Besides that, Rust code is actually much easier to maintain , thanks to type system guarantees.
You can't get 2000x faster without actually eliminating some large percentage of the work, you just cannot. You have to find some way to get rid of 99.95% of the work you were doing. Usually those inflated numbers come from single-thread to multi-thread comparisons, where you can fudge as much as you want by adding more cores. They claim this is a single-core to single-core comparison, so basically that means they ha…
The 2000x number is based on a pathological directory in black's repo https://github.com/psf/black/tree/main/profiling which make duplicate-code really slow and a fix was done in pylint 4.1.0 according to pylint's maintainer.
Re: Fable Converted Pylint to Rust
#103IMHO there is little point of these conversion projects. It screams of "look at me, see what I made" and when the attention goes down a little nothing was ever pushed to the repo ever again. Perhaps I am out of touch, but a project with author/s that have passion for every line, function and purpose, feels more real and worth my trust to spend time using it.
Currently, there are things pylint does that ruff doesn't. To use these, I was running pylint on pypy to get it running at a reasonable speed.
Having pylint reimplemented in Rust seems like a very useful thing to have from my perspective. I get another several x speed up, and I can stop having to worry about a separate python interpreter used for one tool that is a few Python versions behind my codebase.
Re: Fable Converted Pylint to Rust
#104Of course I'm not sure I'd trust to install or use in my main machine something that has been 100% written by an LLM, that doesn't have enough reviews and scrutiny, but the numbers look astounding: codebase pylint prylint speedup black 26.7 hr 41s 2328×
TBH this is true only on the root of black. If you do `uvx pylint src/` it takes 5s on my machine. It's still impressive but it looks like a pathological case in a test directory.
Re: Fable Converted Pylint to Rust
#105Earlier quoted context omitted.
You can't get 2000x faster without actually eliminating some large percentage of the work, you just cannot. You have to find some way to get rid of 99.95% of the work you were doing. Usually those inflated numbers come from single-thread to multi-thread comparisons, where you can fudge as much as you want by adding more cores. They claim this is a single-core to single-core comparison, so basically that means they ha…
>You can't get 2000x faster without actually eliminating some large percentage of the work Yeah, exactly. That's how optimization works! Recognizing that in many cases, the work is that was being done is not necessary to produce the result you actually need, that it could have been in a different way. This can sometimes involve adding complexity (by means of a better data structure or algorithm, e.g. a quadtree), and…
I like the other commenter's information here https://news.ycombinator.com/item?id=48597915. It confirms what i suspected: the source of the 2000x slowdown is very much fixable in Python.
It's not a huge error, it just speaks to the people doing the work not having a solid footing in the concepts
Re: Fable Converted Pylint to Rust
#106> A Rust reimplementation of pylint that produces byte-for-byte identical output — 15–2300× faster (median ~85×). > prylint is not "inspired by" pylint. [...] Where pylint has bugs, prylint reproduces them. Where pylint crashes, prylint reports the same crash message. This looks very strange to me. There's no paper or explanation as to why the output should be identical to the real Pylint. Looking at GitHub, all the…
You can see the validation approach they used here:
https://github.com/adamraudonis/prylint/blob/main/harness/ch...
Re: Fable Converted Pylint to Rust
#107Earlier quoted context omitted.
TBH this is true only on the root of black. If you do `uvx pylint src/` it takes 5s on my machine. It's still impressive but it looks like a pathological case in a test directory.
https://github.com/psf/black/tree/main/profiling
Yes it's that, in particular `duplicate-code`.
`pylint . --disable=R0801` will work, `pylint profiling/ --enable=duplicate-code` doesn't seem to exit in a reasonable time. So that's likely hitting some pathological case, possibly accidentallyquadratic.tumblr.com material.
Re: Fable Converted Pylint to Rust
#108Interesting to read the comments and see the reaction here.
I didn't use Fable (just Opus and Gemini) but I recently ported the `djlint` Python library to Rust, also relying heavily on LLMs (but not trying to one-shot it). `djlint` is a library to lint and reformat HTML and Django template files (and some other formats but I haven't tackled them yet as I don't really have any need). It's currently unmaintained (for a couple years at least) but was part of our CI/CD and the reformat in particular is very slow on a large codebase with thousands of templates. For our code, it took about 4 minutes to run. I generated a Rust port in a similar way, aiming for byte-for-byte output compatability. The Rust version runs on the same code in a fraction of a second; fast enough to put in a pre-commit hook. Some of that was the raw Python->Rust conversion, but a lot was some optimization work that I did afterwards. The Python version was heavily regexp-based, basically running a big slow regexp for every rule that it implements, sometimes running the regexp in a loop starting with each tag it encounters as it goes through each file. For the Rust version, I switched it to properly tokenize the files and then match rules on the AST in a single (or relatively few in some cases) pass. Honestly, there's still a lot of low hanging fruit to make it faster but it's already such a big improvement that I'm pretty happy with it. I'm sure those optimizations could've been done in Python but if I'm the one maintaining the code now, I'd much rather deal with Rust than Python with or without LLM assistance.
Re: Fable Converted Pylint to Rust
#109Earlier quoted context omitted.
Essentially pay-to-win for coding.
With moves like this, I'm not so sure victory is assured... but yes. Pay to play. I, like most, beat linting delay by running it in the background; asynchronous. Having been negative, so far, I want to say something positive. I appreciate that LLMs weren't used for license washing in this case.
It has many nice applications, but those rewrites in general is something I don’t like.
1. Language semantics don’t translate, way people usually write python is not how you write rust.
2. There’s no such thing as line-for-line rewrite with llm, even if you’re really great at spec planning etc, if you don’t review each line, it’s going to be lazy.
3. Brings me to last point, it’s usually such a huge amount of code that llm “ports” and fact it is going to be lazy somewhere, means you get ticking bomb.
On the good side - abandoned software that needs to be migrated to compile on modern systems is for the most part solved and I love it.
Re: Fable Converted Pylint to Rust
#110IMHO there is little point of these conversion projects. It screams of "look at me, see what I made" and when the attention goes down a little nothing was ever pushed to the repo ever again. Perhaps I am out of touch, but a project with author/s that have passion for every line, function and purpose, feels more real and worth my trust to spend time using it.
Ah, that reminded me to check up on these "AI wonder projects". Cursor's Web Browser? Last update 5 months ago: https://github.com/wilsonzlin/fastrender The Claude C compiler? 4 months since any changes: https://github.com/anthropics/claudes-c-compiler Zero moderation too, nice. https://github.com/anthropics/claudes-c-compiler/issues/264#... CloudFlare's slop NextJS project is still going though. https://github.com/c…
CloudFlare's thing at least has some business value compared to the rest.