Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Rars: a Rust RAR implementation, mostly written by LLMs

bitplane.net

Post reply on HN
Print view View original

71–80 of 92 posts

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#71

Post by wolttam »

rebolek wrote: post #29

> "For the last 15 months or so my hobby has been shouting at Claude" How can you shout at Claude when it’s 1) foobaring, bamblabooing and fghrtawing all the time without telling you what’s going on 2) when it finally interacts, it’s asking for a permission you told it 30 seconds ago "yes and do not ever ask me again until heat death of the Universe" 3) and after all of that, it just spits out: "you’re out of tokens,…

> foobaring, bamblabooing and fghrtawing all the time without telling you what’s going on

Oh man, now I have to plug my tool[0]... it doesn't hide anything, but by default tries to provide a pleasant interface (ctrl+o to toggle details similar to CC, but less janky?)

Disclaimer: It's way simpler than Claude Code or even pi (on purpose)

[0]: https://codeberg.org/mlow/lmcli

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#72

Post by 542458 »

no-name-here wrote: post #69
esafak wrote: post #54

Earlier quoted context omitted.

What's your easy way of reverse engineering every previous version of this file format, if you don't think that was the hard part?

Wouldn’t previous human-coded implementations already be in LLMs’ training sets? OP article even mentions a number of previous implementations. And even recent LLM “clean room” designs seem to gloss over how the final LLM implementer still has access to previous implementations’ code. Still, OP claims to have done the best job to date at creating (via AI) specs, and the non-optimal Rust implementation, so a net gain?

I don’t believe any of the extant open source rar implementations cover the range of features and versions OP’s does. I think that’s the point - OP’s isn’t the cleanest or fastest implementation, but it is the most broad open source version available.

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#73

Post by ameliaquining »

Georgelemental wrote: post #67
ameliaquining wrote: post #22

Earlier quoted context omitted.

I read the post you're replying to as saying "this is copyright-encumbered and nonfree because it's a derivative work of everything in Claude's and GPT-5.5's training corpus", which is an argument I find fairly tiresome. (Realistically, if courts actually rule that this is the case, this tiny little project will be the least of anyone's concerns.) "This is copyright-encumbered and nonfree because it's a derivative wo…

I also am skeptical of the "LLM output is derivative of everything in the training corpus" argument in general, but in this specific case I think it may have more merit. If the model was trained on unrar source code, and obtained specific information about the RAR format from that code which it then used in the code generation step, then the output is arguably tainted because of that.

Does the source-available UnRAR do anything that the existing FOSS implementations can't do? IIUC the interesting part of this particular project is that it supports really old versions of the file format that were never publicly documented anywhere.

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#74

Post by gibspaulding »

perching_aix wrote: post #12
themafia wrote: post #3

> But, it works, and the world now has a free software RAR implementation. Does it? How are you legally intending to use copyright to license this machine output? How would you know it's not encumbered in any way?

Really unsure why this is getting downvoted, to my understanding this is a massive, unsettled concern. It wasn't even a disasm/pseudocode to formal spec flow, and then a separate human implementation. The same human has been in the loop throughout, and large parts of it were generated directly. It's basically guaranteed tainted. Edit: I should have skimmed a bit more patiently, there was in fact no "disasm/pseudocode…

> Really unsure why this is getting downvoted

Because it’s a boring argument that we’re not going to make progress on until it is actually tested in court.

Also, if/when this is is tested, the court’s options seem to be (a) say yeah this is fine, or (b) cause unending havoc that if followed through on would destroy the economy (a precedent that any org who’s proprietary code made it into ai training data could sue any org that was using code generated by that model? Do the math on how many suits that is.)

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#75

Post by davidsong »

q3k wrote: post #17
xphos wrote: post #10

Would it really take 5 years to develop rare compress and decompression that seems an extreme overestimate in time. I don't know of the compressor decompression but that seems really high

Yeah, sounds closer to a 5 week thing, if you know what you're doing.

Well, it is every version of RAR. Documenting the quirks of rar 1.4, 1.5, 2.0, 2.9, 3.0, 4.0, 5.0 and 7.0, multiple compression strategies, PPMd, RARVM, compression levels, encryption, multi volume support, a huge test corpus, round trips for compatibility... The spec docs are linked.

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#76

Post by davidsong »

themafia wrote: post #3

> But, it works, and the world now has a free software RAR implementation. Does it? How are you legally intending to use copyright to license this machine output? How would you know it's not encumbered in any way?

I generally don't anyway. Since the WTFPL came out I've been licensing under that with a warranty clause (don't blame me).

My main goal here was an experiment to see how far I could push the technique, and learn things along the way. Regardless of whether people dare to use it commercially or not, we have interoperability for the foreseeable future. As an archivist/computing historian I think that's important.

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#77

Post by davidsong »

slopinthebag wrote: post #8

How do we know it's actually correct?

I compressed thousands of files, went through libarchive's and Sembiance's test data at least for the decompressor side. I recompressed the files, and round-tripped them against 7zip, unrar, every later version of winrar. It failed a lot at the start, and codex burned a lot of tokens instrumenting the binaries and dividing and conquering until things settled down and round-trips worked properly.

I can't really say it works in every case as I honestly didn't spend that much time on it. But it works in the majority of cases. There's likely some nasty bugs hiding in there.

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#78

Post by davidsong »

Imustaskforhelp wrote: post #7

Kudos, this is a really cool project (even if it might be AI generated), I have starred the repo, (3rd starrer here) One thing I have been curious at is are there any ways to stop a rar compression mid way and then continue it later? Like suppose I have a compression happening for a large file, then would there be a possibility with this project to shut down the computer mid compression and continue it after starting…

Thanks!

I guess you could save the state to a file on SIGINT, flush what's been written and pick it back up again if the state file exists when you restart, and use the CRCs of the files to abort if things have changed. I don't fancy doing that for so many versions of RAR, but it would be a cool feature to add it to an `xz` fork. I like the idea.

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#79

Post by davidsong »

TacticalCoder wrote: post #50
perching_aix wrote: post #9

Earlier quoted context omitted.

By using it.

I could be correct but way too slow in edge cases (unlikely with Rust but you never know), leaking temporary files, having security holes, etc. There's much more about correctness of a piece of software than: "produces the same output as the original on x test cases" . I'm not saying it's a bad implementation and, if anything, LLMs are much better at translating/porting existing code (and finding bugs) than at writin…

Yeah the main things are DoS attacks and path traversal issues. I intentionally guarded against these with resource limits and checks, but I can't guarantee that it's safe. I mean, basically anyone who carefully reads it knows more about it than me - you play the AI slot machine at this scale and who knows what prizes you'll win!

Re: Rars: a Rust RAR implementation, mostly written by LLMs

#80

Post by davidsong »

themafia wrote: post #65
ameliaquining wrote: post #55

Earlier quoted context omitted.

What do you mean by "checking the output"? Is there some kind of check the author says he didn't do that you think he should have? Or is your claim that using an LLM for coding is always copyright infringement? If so, I think the risk that I'll personally be the test case that resolves whatever ambiguities exist in the law is basically zero, and I don't think derailing the thread to be about that topic enlightens any…

> What do you mean by "checking the output"? At the very least you could see if it's already been open sourced under a different license. If you take GPL code and just slap MIT on it do you not consider that a violation? > Or is your claim that using an LLM for coding is always copyright infringement? I'm claiming you cannot really know. > I'll personally be It may be someone who uses or redistributes your code in an…

I didn't actually read any code. I generated spec documents using Claude, then later on used Codex to generate from the spec docs. Are the specs tainted? If someone else independently develops from my spec, is that also tainted? What if they hear it second hand? It's an interesting legal situation for sure.
Post reply on HN