221–230 of 279 posts
You can chat with these new models at ultra-low latency at groq.com. 8B and 70B API access is available at console.groq.com. 405B API access for select customers only – GA and 3rd party speed benchmarks soon. If you want to learn more, there is a writeup at https://wow.groq.com/now-available-on-groq-the-largest-and-m... . (disclaimer, I am a Groq employee)
Earlier quoted context omitted.
I have found it to be better than GPT-4o at math too, despite the latter being better at several math benchmarks.
I personally disagree. But i haven't used sonnet that much
Earlier quoted context omitted.
As far as I know it's not just the weights. it's everything but the dataset. So the code used to generate the weights is also open source.
Is there any other case where "open source" is used for something that can't be reproduced? Seems like a new term is required, in the concept of "open source, non-reproducible artifacts". I suppose language changes. I just prefer it changes towards being more precise, not less.
What kind of machine do I need to run 405B local?
You can't. Sorry. Unless... You have a couple hundred $k sitting around collecting dust... then all you need is a DGX or HGX level of vRAM, the power to run it, the power to keep it cool, and place for it to sit.
Earlier quoted context omitted.
Most things do dèbute in the EU, unless the product or company behind it doesn't value your privacy. Meta does not value your privacy.
Privacy was the first thing that the EU did that started this trend of companies slowing their EU releases because of GDPR. Now there's the Digital Markets Act and the AI Act that both have caused companies to slow their releases to the EU. Each new large regulation adds another category of company to the list of those who choose not to participate. Sure, you can always label them as companies who don't value princip…
The word fault somehow implies that something’s wrong - from the eu regulator’s perspective, what’s happening is perfectly normal, and what they want : at some point, the advances in insert new tech are not worth the (social) cost to individuals, so they make things more complicated/ ask companies to behave differently.
Now I’m not saying the regulations are good, required, etc : just that depending on your goal, there are multiple points of view, with different landing zones.
I also suspect that what’s happening now ( meta, apple slowing down) is a power play : they’re just putting pressure on the eu, but I’m harboring doubts that this can work at all.
Earlier quoted context omitted.
I really wish people would use "open weights" rather than "open source". It's precise and obvious, and leaves an accurate descriptor for actual "open source" models, where the source and methods that that generate the artifact , that is the weights, is open.
As far as I know it's not just the weights. it's everything but the dataset. So the code used to generate the weights is also open source.
https://aider.chat/docs/leaderboards/
77.4% claude-3.5-sonnet
75.2% DeepSeek Coder V2 (whole)
72.9% gpt-4o
69.9% DeepSeek Chat V2 0628
68.4% claude-3-opus-20240229
67.7% gpt-4-0613
66.2% llama-3.1-405b-instruct (whole)Earlier quoted context omitted.
I really wish people would use "open weights" rather than "open source". It's precise and obvious, and leaves an accurate descriptor for actual "open source" models, where the source and methods that that generate the artifact , that is the weights, is open.
It's not precise. People who want to use "open weights" instead of "open source" are focusing on the wrong thing. The weights are, for all practical purposes, source code in their own right. The GPL defines "source code" as "the preferred form of the work for making modifications to it". Almost no one would be capable of reproducing them even if given the source + data. At the same time, the weights are exactly what…
Fine-tuning and LoRAs and toying with the runtime are all directly equivalent to DLL injection[0], trainers[1], and various other techniques used to tweak a compiled binary before or at runtime, including plain taking at the executable with a hex editor. Just because that's all anyone except the model vendor is able to do, doesn't merit calling the models "open source", much like no one would call binary-only software "open source" just because reverse engineering is a thing.
No, the weights are just artifacts. The source is the dataset and the training code (and possibly the training parameters). This isn't fundamentally different from running an advanced solver for a year, to find a way to make your program 100 byes smaller so it can fit on a Tamagochi. The resulting binary is magic, can't be reproduced without spending $$$$ on compute for th solver, but it is not open source. The source code is the bit that (produced the original binary that) went into the optimizer.
Calling these models "open source" is a runaway misuse of the term, and in some cases, a sleigh of hand.
--
[0] - https://en.wikipedia.org/wiki/DLL_injection
[1] - https://en.wikipedia.org/wiki/Trainer_(games) - a type of programs popular some 20 years ago, used to cheat at, or mod, single-player games, by keeping track of and directly modifying the memory of the game process. Could be as simple as continuously resetting the ammo counter, or as complex as injecting assembly to add new UI elements.
Wow! The benchmarks are truly impressive, showing significant improvements across almost all categories. It's fascinating to see how rapidly this field is evolving. If someone had told me last year that Meta would be leading the charge in open-source models, I probably wouldn't have believed them. Yet here we are, witnessing Meta's substantial contributions to AI research and democratization. On a related note, for t…