Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Snowflake Arctic Instruct (128x3B MoE), largest open source model

replicate.com

Post reply on HN
Print view View original

151–160 of 224 posts

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#151

Post by temuze »

cs702 wrote: post #2

Wow, 128 experts in a single model. That's a lot more than everyone else. The Snowflake team has a blog post explaining why they did that: https://www.snowflake.com/blog/arctic-open-efficient-foundat... But the most interesting aspect about this, for me, is that every tech company seems to be coming out with a free open model claiming to be better than the others at this thing or that thing. The number of choices is…

In the short-term, these kinds of investments can hype up a stock and create a small bump.

However, in the long-term, as the hype dies down, so will the stock prices.

At the end of the day, I think it will be a transfer of wealth from shareholders to Nvidia and power companies.

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#152

Post by ctoth »

margalabargala wrote: post #126
renewiltord wrote: post #59

Earlier quoted context omitted.

500 tons is like a few flights between SF and NYC dude. And those 600k models are mostly fine-tunes. If running your 4090 at home is too much then we're going to have to get rid of the gamers. This CO2 objection is such an innumerate objection. Just making 100 cars already is more than making one of these LLMs from scratch. A finetune is so cheap in comparison. In fact, I bet if you asked most LLM companies they'd gl…

There are two groups here. One sees the high impact of the large model, and the growth of model training, and is concerned with how much that could increase in coming years. The other group assumes the first group is complaining about right now, and thinks they're being ridiculous. This whole thing reminds me of ten years ago when people were pointing out energy waste as a downside of bitcoin. "It's so little! Electr…

> The other group assumes the first group is complaining about right now, and thinks they're being ridiculous.

Except this is obviously not the case, as "the other group" is aware that many of these large training companies, such as Microsoft, have committed to being net negative on carbon by 2030, and are actively making progress with this whereas the other group seems to be motivated by flailing for anything they can use to point at AI and call it bad.

How many carbon-equivalent tons does training an AI in a net negative datacenter produce? Once the datacenters run on sunlight what is the new objection which will be found?

The rest of the world does not remain static with only the AI investments increasing.

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#153

Post by peteradio »

temuze wrote: post #151
cs702 wrote: post #2

Wow, 128 experts in a single model. That's a lot more than everyone else. The Snowflake team has a blog post explaining why they did that: https://www.snowflake.com/blog/arctic-open-efficient-foundat... But the most interesting aspect about this, for me, is that every tech company seems to be coming out with a free open model claiming to be better than the others at this thing or that thing. The number of choices is…

In the short-term, these kinds of investments can hype up a stock and create a small bump. However, in the long-term, as the hype dies down, so will the stock prices. At the end of the day, I think it will be a transfer of wealth from shareholders to Nvidia and power companies.

> as the hype dies down, so will the stock prices.

*Depending on govt interventions

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#154

Post by fsiefken »

ru552 wrote: post #121
CharlesW wrote: post #103

Earlier quoted context omitted.

To be fair, that's comparing their 17B model with the 70B Llama 3 model.

To stay fair, their "17B" model sits at 964GB on your disk and the 70B Llama 3 model sits at 141GB. unquantized GB numbers for both

to stay fairer, the required extra disk space for snowflake-arctic is cheaper then the required extra ram memory for llama3

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#155

Post by blackeyeblitzar »

cs702 wrote: post #2

Wow, 128 experts in a single model. That's a lot more than everyone else. The Snowflake team has a blog post explaining why they did that: https://www.snowflake.com/blog/arctic-open-efficient-foundat... But the most interesting aspect about this, for me, is that every tech company seems to be coming out with a free open model claiming to be better than the others at this thing or that thing. The number of choices is…

> What's the rationale for releasing all these models to the public? Do all these tech companies know something we don't? Why are they doing this?

It’s mostly marketing for the company to appear to be modern. If you aren’t differentiated and if LLMs aren’t core to your business model, then there’s no loss from releasing weights. In other cases it is commoditizing something that would otherwise be valuable for competitors. But most of those 600K models aren’t high performers and don’t have large training budgets, and aren’t part of the “race”.

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#156

Post by moffkalast »

1f60c wrote: post #33

It appears to have limited guardrails. I got it to generate some risqué story and it also told me how to trade onion futures, which is illegal in the US.

No! Not the onion futures! FBI, arrest this man, he is too dangerous to be allowed to remain at large.

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#157

Post by ko27 »

Zambyte wrote: post #150
ko27 wrote: post #145

Earlier quoted context omitted.

> Is the IDE a primary input for building the program? Is the IDE a build dependency? No, the same way training is not a build dependency for the weights source code. You can literally compile and run them without any training data.

Training data is a build dependency for the weights. You cannot realistically get the same weights without the same training data.

Developer's mindset, knowledge and tooling is also a build dependency for any open source code. You can not realistically get the same code without it.

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#158

Post by cryptonector »

ponector wrote: post #146
cryptonector wrote: post #138

Earlier quoted context omitted.

Eh, the knowledge of how to construct a nuclear weapon has long been widely and publicly available. And it's not useful, not without weapons-grade fissile material. And if you could make weapons-grade fissile material you could figure out how to make a weapon if it wasn't already well-known.

And yet, chatgpt is censored to not hallucinate about it.

A bit silly, no?

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#159

Post by leblancfg »

bfirsh wrote: post #19

If you want to have a conversation with it, here's a full chat app: https://arctic.streamlit.app/ Official blog post: https://www.snowflake.com/blog/arctic-open-efficient-foundat... Weights: https://huggingface.co/Snowflake/snowflake-arctic-instruct

Wow that is *so fast*, and from a little testing writes both rather decent prose and Python.

Re: Snowflake Arctic Instruct (128x3B MoE), largest open source model

#160

Post by coder543 »

ukuina wrote: post #95

So many models fail at basic reasoning. > What weighs more: a pound of feathers or a great british pound? > A pound of feathers and a Great British Pound weigh the same, which is one pound. It works if you add "Think step by step," though.

Llama-3 8B got it on the first try: https://imgur.com/a/xQy1828
Post reply on HN