251–260 of 279 posts
I've just finished running my NYT Connections benchmark on all three Llama 3.1 models. The 8B and 70B models improve on Llama 3 (12.3 -> 14.0, 24.0 -> 26.4), and the 405B model is near GPT-4o, GPT-4 turbo, Claude 3.5 Sonnet, and Claude 3 Opus at the top of the leaderboard. GPT-4o 30.7 GPT-4 turbo (2024-04-09) 29.7 Llama 3.1 405B Instruct 29.5 Claude 3.5 Sonnet 27.9 Claude 3 Opus 27.3 Llama 3.1 70B Instruct 26.4 Gemin…
Earlier quoted context omitted.
Just checked it out. Is pay-as-you-go API access available at all? It says 'Coming Soon' https://console.groq.com/settings/billing
I think you answered it yourself? It’s coming soon, so it is not available now, but soon.
I'm really impressed by what (&how) they're doing and would like to pay for a higher rate limit, or failing that at least know if "soon" means "weeks" or "months" or "eventually".
I remember TravisCI did something similar back in the day, and then Circle and GitHub ate their lunch.
Earlier quoted context omitted.
Just checked it out. Is pay-as-you-go API access available at all? It says 'Coming Soon' https://console.groq.com/settings/billing
I've found Bedrock to be nice with pay-as-you-go, but they take a long time to adopt new models.
Earlier quoted context omitted.
It's not precise. People who want to use "open weights" instead of "open source" are focusing on the wrong thing. The weights are, for all practical purposes, source code in their own right. The GPL defines "source code" as "the preferred form of the work for making modifications to it". Almost no one would be capable of reproducing them even if given the source + data. At the same time, the weights are exactly what…
Do the costs really matter here? "Weights" are "the preferred form of the work for making modifications to it" in the same sense compiled binary code would be, if for some reason no one could afford to recompile a program from sources. Fine-tuning and LoRAs and toying with the runtime are all directly equivalent to DLL injection[0], trainers[1], and various other techniques used to tweak a compiled binary before or a…
Earlier quoted context omitted.
It's not precise. People who want to use "open weights" instead of "open source" are focusing on the wrong thing. The weights are, for all practical purposes, source code in their own right. The GPL defines "source code" as "the preferred form of the work for making modifications to it". Almost no one would be capable of reproducing them even if given the source + data. At the same time, the weights are exactly what…
Do the costs really matter here? "Weights" are "the preferred form of the work for making modifications to it" in the same sense compiled binary code would be, if for some reason no one could afford to recompile a program from sources. Fine-tuning and LoRAs and toying with the runtime are all directly equivalent to DLL injection[0], trainers[1], and various other techniques used to tweak a compiled binary before or a…
Earlier quoted context omitted.
As far as I know it's not just the weights. it's everything but the dataset. So the code used to generate the weights is also open source.
In other words, it's everything except the one thing that actually matters .
Llama 3 Training System 19.2 exaFLOPS _____ / \ Cluster 1 Cluster 2 / \ 9.6 exaFLOPS 9.6 exaFLOPS / \ _______ _______ / ___ \ / \ / \ ,----' / \`. `-' 24000 `--' 24000 `----. ( _/ __) GPUs GPUs ) `---'( / ) 400+ TFLOPS 400+ TFLOPS ,' \ ( / per GPU per GPU ,' \ \/ ,' \ \ TOTAL SYSTEM ,' \ \ 19,200,000 TFLOPS ,' \ \ 19.2 exaFLOPS ,' \___\ ,' `----------------'
how much would it cost?
Earlier quoted context omitted.
It's not precise. People who want to use "open weights" instead of "open source" are focusing on the wrong thing. The weights are, for all practical purposes, source code in their own right. The GPL defines "source code" as "the preferred form of the work for making modifications to it". Almost no one would be capable of reproducing them even if given the source + data. At the same time, the weights are exactly what…
Its not open source. Your definition would make most video games open source - we modify them all the time. The small runtime framework IS open source but that's not much benefit as you cant really modify it hugely because the weights fix it to an implementation.
No, because most video games aren't licensed in a way that makes that explicitly authorized, nor is modding the preferred form of the work for making modifications. The video game has source code that would be more useful, the model does not have source code that would be more useful than the weights.
Earlier quoted context omitted.
It's not precise. People who want to use "open weights" instead of "open source" are focusing on the wrong thing. The weights are, for all practical purposes, source code in their own right. The GPL defines "source code" as "the preferred form of the work for making modifications to it". Almost no one would be capable of reproducing them even if given the source + data. At the same time, the weights are exactly what…
Do the costs really matter here? "Weights" are "the preferred form of the work for making modifications to it" in the same sense compiled binary code would be, if for some reason no one could afford to recompile a program from sources. Fine-tuning and LoRAs and toying with the runtime are all directly equivalent to DLL injection[0], trainers[1], and various other techniques used to tweak a compiled binary before or a…
No, because fine tuning is basically just a continuation of the same process that the original creators used to produce the weights in the first place, in the same way that modifying source code directly is in traditional open source. You pick up where they left off with new data and train it a little bit (or a lot!) more to adapt it to your use case.
The weights themselves are the computer program. There exists no corresponding source code. The code you're asking for corresponds not to the source code of a traditional program but to the programmers themselves and the processes used to write the code. Demanding the source code and data that produced the weights is equivalent to demanding a detailed engineering log documenting the process of building the library before you'll accept it as open source.
Just because you can't read it doesn't make it not source code. Once you have the weights, you are perfectly capable of modifying them following essentially the same processes the original authors did, which are well known and well documented in plenty of places with or without the actual source code that implements that process.
> Calling these models "open source" is a runaway misuse of the term, and in some cases, a sleigh of hand.
I agree wholeheartedly, but not because of "source". The sleight of hand is getting people to focus on that instead of the really problematic word.