Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

621–630 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#621

Post by throw10920 »

bluegatty wrote: post #583
dijksterhuis wrote: post #572

Earlier quoted context omitted.

> it's hilarious to see how all these HN intellectuals, who on any other day are rabidly against most DRM / IP issues all of a sudden become private property absolutists. for the record, i've always been rabidly pro-copyright since i worked at a performing royalty organization (prs for music) circa 15 years ago, way before i joined hn. i don't use llms for that reason. > It's also sadly hypocritical to see all this r…

I'm not attacking you, you don't have to defend yourself. I'm just nothing that HN rhetoric is contradictory. But this: "when i see a spade, i call it a spade." -> this is anti intellectual absolutism. If it were some true injustice, then fine, but that is clearly not the case. There is ample room to contemplate that even copyrighted works could be considers fair use as training material. "the corpos can get fucked a…

> I'm just nothing that HN rhetoric is contradictory.

Periodic reminder that HN is not a collective or a singular entity and is actually a bunch of different people with different opinions. Often the people with the loudest opinions get upvoted to the top - and often the "side" represented at the top is different from thread to thread.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#622

Post by dang »

stego-tech wrote: post #236

[flagged]

Can you please not fulminate or post flamebait on HN? This is in the site guidelines: https://news.ycombinator.com/newsguidelines.html.

You may not owe AmericanAIBros better but you owe this community better if you're participating in it.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#623

Post by skeledrew »

tristanj wrote: post #547
skeledrew wrote: post #77

Super interesting. So Fable was really made available... a couple weeks ago? And K3 a few days ago? That's a really impressive feat to distill enough data AND train AND review to get a release that works really well in that time period. Mad props to the Moonshot team :flame:.

Fable was launched on June 9 (for 72 hours), then K3 launched on July 16. Timeline wise, Moonshot had over a month to post-train K3 on Fable distilled data, which is more than enough time.

So in 3 days they were able to distill enough data to make a significant difference in the K3's performance? Without triggering any limiters when there's already suspicion of distillation? That's still a heck of a feat. Like the bank robbers who were able to keep coming back to take more even though the bank was aware they had gotten robbed recently and had the resources to put sophisticated security systems in place.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#624

Post by janderson215 »

aunty_helen wrote: post #365
jaggederest wrote: post #142

Earlier quoted context omitted.

Perhaps even more importantly, the current frontier LLM models are self-admittedly the product of enormous quantities of copyright infringement and even less savory inputs, so calling them out for distilling the fruit of that tainted tree reads as highly hypocritical at best.

No, they settled that yesterday, so all is forgotten. Press releases were queued for today so just in the nick of time.

I think that was specifically on the piracy aspect.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#626

Post by cwmoore »

jgilias wrote: post #63
cwmoore wrote: post #20

Earlier quoted context omitted.

I still believe taxing the bots, and implementing actual UBI, would address both problems.

Will the UBI apply to everyone worldwide? As that’s where the original dataset came from.

Will it?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#628

Post by antiamerican634 »

customguy wrote: post #596
4bpp wrote: post #581

Earlier quoted context omitted.

And some people might want to know about [insert your favourite beyond-the-pale-in-the-US topic here, we're on a US forum after all]. I think it would be great if those people could turn to Chinese models, while anyone wanting to know about Uyghur camps can ask the US ones.

> beyond-the-pale-in-the-US topic Can you name one? It's an honest question, first of I'm not American, but also the stuff I do come up with (asking an LLM how to blow up a school or whatever) would also be handled similarly in China, so those would be a wash, and I can't think of any that aren't.

From an outsiders point of view:

- The Trail of Tears

- The Tuskegee syphilis study

- Use of Agent Orange in the Vietnam War

- Open Air biological warfare testts in civilians eg. in 1950 San Francisco

- The only use of nuclear weapons against civilians?

- Coca Cola and "american culture"

- Neoliberalist economy

- Spreading blame for their sins to other "white" nations

plus one: The text input method to HN comments :(

Re: “We have information that Moonshot distilled Fable for the development of K3”

#629

Post by bluegatty »

HarHarVeryFunny wrote: post #593
bluegatty wrote: post #580

Earlier quoted context omitted.

I see your 'fine point' but I don't think it holds - 'distillation' is a perfectly reasonable term to describe the process of creating outputs from one model to that expose key training element, to use in another model. I think where the definition may be be invalid, is in the creation of 'unrelated data sets for training' models, for unrelated issues. Creating training sets that mach a models core training, is defin…

No - you can't distill if what you are given doesn't have the thing in it that you want to distill out of it. I don't know why it's so important to you to use the word "distillation", but it's the wrong word to use. BTW OpenAI on twitter also said that Kimi 3 "cannot be explained away by distillation or anything like that". The timeline of how long it takes to train a model and when Fable was released don't even line…

Distillation is absolutely - and uncontroversially - a valid term for what is happening here.

This isn't really a debate, I'm not making a fine point - just check with all of the various defintions of the term.

Moreover - the 'reasoning traces' are not required for distillation at all.

Finally - it's entirely possible for them to have used Fable for later stage fine tuning.

It's fair to be skeptical of Anthropic (and everyone else) - but this is 'distilling'.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#630

Post by bluegatty »

queenkjuul wrote: post #592
bluegatty wrote: post #515

Earlier quoted context omitted.

It's hilarious to see how all these HN intellectuals, who on any other day are rabidly against most DRM / IP issues all of a sudden become private property absolutists. It's also sadly hypocritical to see all this rhetoric here on this thread decrying SOTAs for using a variety of content for their inputs as somehow sTeaLINg sTufF! ... ... but Chinese SOTA foundries directly using distillation as fair game. I don't th…

Like most things, i support rights for people, and not companies. Copyright was created for authors, artists, and inventors. Rules for corporations can and should be different.

This makes little sense, either from either a moral or pragmatic perspective.

You do realize the 'investors' are the one's who 'own' companies and therefore the IP?

Post reply on HN