Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

“We have information that Moonshot distilled Fable for the development of K3”

twitter.com

Post reply on HN
Print view View original

561–570 of 742 posts

Re: “We have information that Moonshot distilled Fable for the development of K3”

#561

Post by Grimblewald »

qwertox wrote: post #67
throwa356262 wrote: post #12

Kimi K3 was released July 16, Fable ban was lifted on July 1 but access was still limited. How did Moonshot "distil" a huge model in such short time and still had time to run the benchmarks and do the usual release thingies? I think Anthropic is desperate to stop foreign competition and the administration is happy to help because they too are heavily invested in these companies

It looks like these frontier-model companies don't really monitor their systems. Like OpenAI not realizing that it is their own AI which is attacking HuggingFace.

alternativly the HF is a gpt2/strawberry/mythos style marketing stunt.

Does no one remember the extreme fearmongering around gpt2 which barely produced coherent text?

Re: “We have information that Moonshot distilled Fable for the development of K3”

#562

Post by usef- »

Bratmon wrote: post #549
usef- wrote: post #534

Earlier quoted context omitted.

Fair, but isn't "illegal" access what they're talking about in OP? It does seem to be becoming the norm for AI companies to licence premium content in America, judging by the deals they're making. It doesn't seem to be done by the international distillers. It's a cost that American open models will seem to have to pay but not international.

> It does seem to be becoming the norm for AI companies to licence premium content in America, judging by the deals they're making. This is a very surprising claim to me (and I imagine many small website owners who keep getting scraped by Anthropic and OpenAI). Do you have a source?

There's been many news stories of it over the past year(s) as they signed each one. Here's the first result I could see with a rundown of many of them (am on mobile).

https://digiday.com/media/a-timeline-of-the-major-deals-betw...

Re: “We have information that Moonshot distilled Fable for the development of K3”

#563

Post by [deleted] »

breppp wrote: post #492
bilbo0s wrote: post #469

Earlier quoted context omitted.

Why? Serious question. I'm from the US, and I think it's hilarious.

One example is that a totalitarian government known for erasing historical record of its crimes will control an arbitrator of truth.

[deleted]

Re: “We have information that Moonshot distilled Fable for the development of K3”

#564

Post by dudeinhawaii »

sailingparrot wrote: post #523
dudeinhawaii wrote: post #511

I think most of the comments in this thread are missing the point. It's not about whether it's legal/ethical/etc. It's about the narrative that "Chinese models are at Fable level". The truth (if correct) is the China continues to copy, and the proprietary US Models continue to lead the state of the art. There is no K4 without Fable 6, GPT-6. That, matters.

> There is no K4 without Fable 6, GPT-6. That, matters. That's simply not true though. Chinese labs very clearly have the entire stack developed and working. Using traces from claude allows them to shorten their training time by some amount, that's it. Remove Fable 6 and you still have K4 eventually, just 2 months later at best.

[flagged]

Re: “We have information that Moonshot distilled Fable for the development of K3”

#565

Post by Bratmon »

usef- wrote: post #562
Bratmon wrote: post #549

Earlier quoted context omitted.

> It does seem to be becoming the norm for AI companies to licence premium content in America, judging by the deals they're making. This is a very surprising claim to me (and I imagine many small website owners who keep getting scraped by Anthropic and OpenAI). Do you have a source?

There's been many news stories of it over the past year(s) as they signed each one. Here's the first result I could see with a rundown of many of them (am on mobile). https://digiday.com/media/a-timeline-of-the-major-deals-betw...

Those are licenses for API access to data too new to be in the training data (for use by agents), not for the training itself.

I don't really understand why you think they're relevant, given that this conversation is about the training itself.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#566

Post by ilovecake1984 »

foxglacier wrote: post #455
ilovecake1984 wrote: post #325

Earlier quoted context omitted.

The didn’t pay for the books. It’s massive copyright infringement. The human buys the books.

It's worth keeping in mind the purpose of copyright. It's a pragmatic tool to encourage investment in creative work for the benefit of everybody/consumers. We may be entering a time where there's less need to incentivize people to write books. At least not non-fiction books which are simply a collection of existing knowledge presented in an a way that's suitable for human readers. A lot of the value those authors pro…

I don’t think it’s been common to write non fiction for money for decades. What they are doing is killing off the real motivation to do it, which is recognition and attribution.

We will all be sorry when professionally written and edited works disappear. An author has a reputation and the incentive to protect that reputation keeps standards high.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#567

Post by cortesoft »

iamniels wrote: post #168
throwa356262 wrote: post #89

Earlier quoted context omitted.

Dean Ball, "head of strategic futures" at openai. https://xcancel.com/deanwball/status/2078133895766114412#m

> One probable outcome of an open-weight-model-dominant world is full AI communism ... This future strikes me as a dystopian hellscape. Wow, just wow. He is not even subtle about it.

I would be very curious to hear him expand on this argument. I can’t imagine it is quite as self serving as it sounds at first, and I would like to hear what he is actually trying to say. I doubt I will agree, but I am very interested.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#568

Post by usef- »

trhway wrote: post #560
usef- wrote: post #534

Earlier quoted context omitted.

Fair, but isn't "illegal" access what they're talking about in OP? It does seem to be becoming the norm for AI companies to licence premium content in America, judging by the deals they're making. It doesn't seem to be done by the international distillers. It's a cost that American open models will seem to have to pay but not international.

>It does seem to be becoming the norm for AI companies to licence premium content in America, judging by the deals they're making. It doesn't seem to be done by the international distillers. International distillers doesn't use that premium content, so they don't pay for it. They do pay for their access to the models they are distilling. Thus providing the revenue stream to those models. Thus those models make profit…

> International distillers doesn't use that premium content, so they don't pay for it.

This doesn't seem to be true. They are training on their own scraped data overwhelmingly (we can extract copyright data from, eg, deepseek). They couldn't get nearly enough tokens through the American APIs to train a model on alone.

> American companies and their business are protected by American laws and have access to the market protected by those laws

Absolutely. Currently international providers are selling inference on the American market though, I don't know how that will sit legally the way things are currently going.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#569

Post by usef- »

Bratmon wrote: post #565
usef- wrote: post #562

Earlier quoted context omitted.

There's been many news stories of it over the past year(s) as they signed each one. Here's the first result I could see with a rundown of many of them (am on mobile). https://digiday.com/media/a-timeline-of-the-major-deals-betw...

Those are licenses for API access to data too new to be in the training data (for use by agents), not for the training itself. I don't really understand why you think they're relevant, given that this conversation is about the training itself.

It isn't just that, it includes publishing houses, Wiley etc, and non-live media.

Even the news orgs say explicitly in the press releases that it's about training on their archive

eg. http://ap.org/media-center/press-releases/2023/ap-open-ai-ag...

---

edit, examples:

Wiley https://newsroom.wiley.com/press-releases/press-release-deta...

Shutterstock https://investor.shutterstock.com/news-releases/news-release...

Axel Springer https://openai.com/index/axel-springer-partnership

Stack Overflow: https://stackoverflow.co/partnerships

Disney (for characters in video. Video is especially where licensing is a big difference internationally right now) https://openai.com/index/disney-sora-agreement

etc.

The news corp one had a leaked price ($250mill), so they don't seem to be insignificant. These would have to be included in API prices I presume.

Re: “We have information that Moonshot distilled Fable for the development of K3”

#570

Post by lovich »

bluegatty wrote: post #515
dijksterhuis wrote: post #471

Earlier quoted context omitted.

> it's only now the issues are coming into the commons in a way that industry / society needs the regulatory clarity. you mean like the regulatory clarity surrounding stealing shit to make the LLMs in the first place? > [There is] extensive litigation on the limits of Fair Use to AI development. Currently, we only have 3 first instance decisions out of the 53 cases being tried. It will likely take a decade before we…

It's hilarious to see how all these HN intellectuals, who on any other day are rabidly against most DRM / IP issues all of a sudden become private property absolutists. It's also sadly hypocritical to see all this rhetoric here on this thread decrying SOTAs for using a variety of content for their inputs as somehow sTeaLINg sTufF! ... ... but Chinese SOTA foundries directly using distillation as fair game. I don't th…

> It's also sadly hypocritical to see all this rhetoric here on this thread decrying SOTAs for using a variety of content for their inputs as somehow sTeaLINg sTufF! ...

> ... but Chinese SOTA foundries directly using distillation as fair game.

As someone who says it’s fair game, it’s less that I’m being hypocritical and more that I don’t care that one thief had their shit stolen by a second thief. I also wouldn’t care if someone distills the Chinese models. It’s just thieves all around and if they want legal protection or moral outrage from the common man then my view is that they should stop stealing first.

Post reply on HN