Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Hy4 preview

tencent.com

Post reply on HN
Print view View original

251–260 of 273 posts

Re: Hy4 preview

#251

Post by dotancohen »

rickydroll wrote: post #248
dotancohen wrote: post #244

Earlier quoted context omitted.

The Pleiades cluster is called the seven sisters in Greek. That's curious, because the human eye under the best conditions can discern only six stars in there. Even more curious, the aboriginal Australians also called this cluster the seven sisters. Ancient Greeks' and aboriginal Australians' last common ancestors split about 60,000 years ago. And astronomers tell us that 60,000 years ago, there were seven discernabl…

Interestingly, back in my ill-spent youth, a few of my fellow astronomers and I were out in a very, very dark-sky location in the late 70s/early 80s, and we were able to consistently count and draw between 9 and 11 stars. Although we would tease those who could see 11 stars as using averted imagination. :-) Today, if I can see six stars, it's an okay night in an okay sky. fwiw, if you can get out to dark skies where…

I'm actually out there looking up quite often! And I'm happy to mention that all three of my children come with me regularly as well.

Clear skies!

Re: Hy4 preview

#252

Post by embedding-shape »

bredren wrote: post #246
haaz wrote: post #198

Earlier quoted context omitted.

The idea that distillation is a significant contributor to the capabilities of the Chinese models is not true. Kimi K3 came out 2 weeks after Fable and uses a number of novel NN architecture innovations.

That may be true, though Anthropic reported 3.4 million exchanges with Moonshot months before Fable. Why bother creating hundreds of (presumably paid) accounts and the tooling to create and consume the data if not of tangible value to their core mission? https://www.anthropic.com/news/detecting-and-preventing-dist...

> Why bother

Because the founding pillars for most of these labs are basically "more data can't hurt" and "no one died from too much data" and "you can never have enough data".

Re: Hy4 preview

#253

Post by simonw »

stymaar wrote: post #233
ziofill wrote: post #195

Earlier quoted context omitted.

That’s a fair question, but it seems that it’s not yet necessary. See here https://dylancastillo.co/posts/pelicanmaxxing.html https://simonwillison.net/2026/Jul/22/

I don't think this argument is a good one though, as it would be quite natural for a lab rhat want to macimize the performance of their model on the pelican bench to train it for “text-to-svg simple image generation” rather than just “pelicans on bicycle”.

Gemini have done exactly that.

(I doubt it's because of my stupid benchmark, though!)

Re: Hy4 preview

#254

Post by [deleted] »

alexfortin wrote: post #137
jorl17 wrote: post #15

I experimented with Hy3 for a project and was surprised with how good it was. I don't know if it's good for coding, but as a general purpose agentic model, it was only beaten by deepseek4-flash in our tests. It was so close to deepseek behaviour I kept thinking it must have been forked from it.

For the last few days I've been experimenting with the _free_ version of Hy3 offered by Opencode Go and I was also surprised to see how (relatively) good it is on coding tasks too. The free quota from Opencode Go is also surprisingly generous, I perhaps hit limits one or two times and I've been using it _a lot_ for implementation tasks (using e.g. GLM-5.3-flash for working on specs and planning next steps).

[deleted]

Re: Hy4 preview

#256

Post by frabcus »

mirekrusin wrote: post #178
frabcus wrote: post #160

Earlier quoted context omitted.

Well, you can't add or alter data in pre-training from just the weights. Which, as I understand it, means you can't fundamentally increase core knowledge or cognitive ability, only what the model likes to do with those. You can only post-train, and you're subject as a result to catastrophic forgetting. To explain simply as far as I can tell (would love to be corrected) the large number of pre-training tokens only wor…

As I live next to EPFL, I'll give you example from them: their Meditron-70B model is adapted to the medical domain from Llama-2-70B through continued pretraining. They took weights of Llama-2-70B and continued training on PubMed, medical guidelines and general data. Weights aren't just executable artifact that's consumed by users. Third parties actually use released parameter state as the editable starting point for…

Nice example - although it seems it ended up specialised in medical texts, so did indeed ("catastrophically") forget other knowledge?

Re: Hy4 preview

#257

Post by mirekrusin »

frabcus wrote: post #256
mirekrusin wrote: post #178

Earlier quoted context omitted.

As I live next to EPFL, I'll give you example from them: their Meditron-70B model is adapted to the medical domain from Llama-2-70B through continued pretraining. They took weights of Llama-2-70B and continued training on PubMed, medical guidelines and general data. Weights aren't just executable artifact that's consumed by users. Third parties actually use released parameter state as the editable starting point for…

Nice example - although it seems it ended up specialised in medical texts, so did indeed ("catastrophically") forget other knowledge?

No, it didn't.

It lost 69.2% -> 67.8% on MMLU while improving medical performance. If you're trying to argue that loss of ~1.4 points is "catastrophic forgetting" (it's not) then look at later work, ie. Me-LLaMA that clearly demonstrates continued pretraining that improved both general MMLU and medical performance.

Not sure why you're fixating on catastrophic forgetting. How do you think model training works? Model training is just a sequence of checkpoints: pretraining produces it, training resumes from last, continued pretraining starts from last, supervised fine tuning starts from last, RL/post-training starts from last - it's just a sequence of checkpoints.

There isn't some fundamental distinction where original author continuing training from checkpoint X is training but a third party downloading checkpoint X and continuing training from it suddenly isn't.

ie. checkpoint doesn't somehow become a different kind of artifact when it's published.

Re: Hy4 preview

#258

Post by vasco »

zhahhshsj wrote: post #232
vasco wrote: post #175

Earlier quoted context omitted.

My only point is in a many agent system with different goals it "doesn't matter" that some agents have bad goals as long as there's enough variability of goals and resources that they can't put their vision in place.

I know but that is just .. not how it works. Humans don’t align on just about anything but they will do tremendous mindboggling amounts of harm if not checked by mountains of checks and balances. Sheer variety alone is not a guarantee of anything. Many types of Hitler does not make for a peaceful world all of sudden through sheer competition. It sounds nice but it will lead to certain hell.

Checks and balances are the result of agents having different goals. If they all had the same goals they wouldn't need to agree on rules of operations, they'd just paperclip their way to the goal. AI is going to be controlled by humans so the goal variability will be there anyway.

Re: Hy4 preview

#259

Post by dTal »

dotancohen wrote: post #177
beefsack wrote: post #91

Earlier quoted context omitted.

I can't help but imagine agents using caveman speak sometimes start behaving in a stereotypically caveman manner, even if it's subtle. Is there a chance the agent does less reasoning because of it?

Caveman invented fire, the wheel, domesticated wild plants and animals, organised society, survived the Toba catastrophe, cooked food, and was having sex ages before you and me. Don't write him off as stupid.

https://www.smbc-comics.com/comic/me

Re: Hy4 preview

#260

Post by Vachyas »

try-working wrote: post #90
codethief wrote: post #51

> Notably, Hy4 preview also contributed to its own development process, participating for the first time in the automated optimization of training methods, data strategies, evaluation frameworks, and low-level operators. The model proposed approaches, ran experiments, and iterated based on the results, with the resulting code, logs, and feedback feeding into subsequent rounds of exploration. This established an early…

Just like how Windows 95 contributed to its own development process.

Lmaoo great comment
Post reply on HN