It wonders me that a coupe of days ago I did the same with Unique and a single skill.md, repo: https://github.com/daedalus/uniq-reconstruction, on succes I tried with rar but failed. Kudos
Rars: a Rust RAR implementation, mostly written by LLMs
41–50 of 92 posts
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#42I have never attempted something so ambitious with AI, but this feels spot on in terms of experience. As you cede more control to the model, you will find yourself losing control on things like code quality and performance.
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#43How do we know it's actually correct?
By using it.
Thus all software that can be used is correct?
You know what I meant: How can we have confidence that this implementation of RAR is functionally identical to what it's based on? What would give me the confidence to use it in a critical piece of infrastructure?
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#44Earlier quoted context omitted.
It works == it's correct?
Yes? What do you think fuzzing, unit testing, integration testing is for? It's an empirical evaluation of correctness. Literally just try and see. For actual correctness verification in the strong sense, you'd need to start from a specification written in a formal language so that it's machine checkable, which if I had to guess not even win.rar GmbH has.
You're being needlessly dismissive.
From a philosophical perspective, there's no way to know that any piece of software is truly correct without formal verification.
But in the present, non-philosophical context, it's obvious that what we mean is, colloquially, "how well-tested is this against a variety of edge-case files which the official winrar handles correctly? Is there a test suite, and how robust is it? Plenty of software that claims to be compatible with the rar format, doesn't actually successfully read all rar files."
It's also equally obvious, in the present context, that we would prefer these steps to have been taken by the author of the software before we install it and run it on our own computers and data. The parent commenter wasn't just asking about the software's correctness for the sake of academic curiosity.
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#45Would it really take 5 years to develop rare compress and decompression that seems an extreme overestimate in time. I don't know of the compressor decompression but that seems really high
Yeah, sounds closer to a 5 week thing, if you know what you're doing.
5 week is a decompressor for 1 version. If this supports multiple versions of RAR, then writing decompressors alone for all of them is probably a year effort of work.
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#46Earlier quoted context omitted.
By using it.
Thus all software that can be used is correct? You know what I meant: How can we have confidence that this implementation of RAR is functionally identical to what it's based on? What would give me the confidence to use it in a critical piece of infrastructure?
Validating compression systems is usually really straightforward. There are 3 layers - decode known values from compressed files (or encode, same), round trip without any alterations, and fuzzing with arbitrary binaries
Because it's a defined format there can be binary exact comparisons between the input and output files - we already have an oracle in the form of proper RAR format software, so if they are identical, you don't need to look further for that specific case.
You can see a version of this that I did quite similarly, for postgresql wire format, here: https://github.com/pgdogdev/pgdog/tree/main/integration/sql
It validates that sql with the same setup, teardown, and test results in perfectly exact compatibility between raw postgresql as the control and various configurations of PgDog, with both the text format and binary format, so ultimately a 6-way multivariate test that should always result in binary-exact results.
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#47[flagged]
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#48Earlier quoted context omitted.
Really unsure why this is getting downvoted, to my understanding this is a massive, unsettled concern. It wasn't even a disasm/pseudocode to formal spec flow, and then a separate human implementation. The same human has been in the loop throughout, and large parts of it were generated directly. It's basically guaranteed tainted. Edit: I should have skimmed a bit more patiently, there was in fact no "disasm/pseudocode…
The human wasn't looking at the copyrighted code and was giving high level steering instructions. If you look at the spec generated it doesn't look like a derivative work of the copyrighted material. The program was generated from the spec. It seems mostly fine from my perspective.
If I use a decompiler on existing binaries, then some machine translation utility to turn that into a different language, that still feels like a derivative work, even if no human were reviewing the specifics.
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#49> It’s sloppy, it’s slow, it’s almost two megabytes in size and somewhat worse than WinRAR on compression. As mathematicians say, optimization is left as an exercise to the reader. You did the hard part.
I mean, not really...? A vibecoded mess that runs badly, that's not really the hard part for something like compression/decompression tools.
Re: Rars: a Rust RAR implementation, mostly written by LLMs
#50How do we know it's actually correct?
By using it.
I could be correct but way too slow in edge cases (unlikely with Rust but you never know), leaking temporary files, having security holes, etc.
There's much more about correctness of a piece of software than: "produces the same output as the original on x test cases".
I'm not saying it's a bad implementation and, if anything, LLMs are much better at translating/porting existing code (and finding bugs) than at writing things unheard of.
You're basically saying, if I may make a pun: "rust me bro, it's correct".