Related ongoing thread:
Open source AI is the path forward - https://news.ycombinator.com/item?id=41046773 - July 2024 (278 comments)
Llama 3.1
121–130 of 279 posts
Re: Llama 3.1
#122Does anyone know why they haven't released any 30B-ish param models? I was expecting that to happen with this release and have been disappointed once more. They also skipped doing a 30B-ish param model for llama2 despite claiming to have trained one.
I suspect 30B models are in a weird spot, too big for widespread home use, too small for cutting edge performance. For home users 7B models (which can fit on an 8GB GPU) and 13B models (which can fit on a 16GB GPU) are in far more demand. If you're a researcher, you want a 70B model to get the best performance, and so your benchmarks are comparable to everyone else.
I thought home use is whatever fits in 24GB (a single 3090 GPU, which is pretty affordable), not 8 or 16. 30B models fit.
Re: Llama 3.1
#123Wow! The benchmarks are truly impressive, showing significant improvements across almost all categories. It's fascinating to see how rapidly this field is evolving. If someone had told me last year that Meta would be leading the charge in open-source models, I probably wouldn't have believed them. Yet here we are, witnessing Meta's substantial contributions to AI research and democratization. On a related note, for t…
Tried using msty today and it refused to open and demanded an upgrade from 0.9 - remotely breaking a local app that had been working is unacceptable. Good luck retaining users.
Re: Llama 3.1
#124You can already run these models locally with Ollama (ollama run llama3.1:latest) along with at places like huggingface, groq etc.
If you want a playground to test this model locally or want to quickly build some applications with it, you can try LLMStack (https://github.com/trypromptly/LLMStack). I wrote last week about how to configure and use Ollama with LLMStack at https://docs.trypromptly.com/guides/using-llama3-with-ollama.
Disclaimer: I'm the maintainer of LLMStack
Re: Llama 3.1
#125"Meta AI isn't available yet in your country" Hi from europe :/
Why are (some) Europeans surprised when they are not included in tech product débuts? My lay understanding could best be described as; EU law is incredibly business unfriendly and takes a heroic effort in time and money to implement the myriad of requirements therein. Am I wrong?
> Why are (some) Europeans surprised when they are not included in tech product débuts?
Why do you think he is surprised? I think very few are surprised.
Re: Llama 3.1
#126I have found Claude 3.5 Sonnet really good for coding tasks along with the artifacts feature and seems like it's still the king on the coding benchmarks
I have found it to be better than GPT-4o at math too, despite the latter being better at several math benchmarks.
Re: Llama 3.1
#127The 405b model is actually competitive against closed source frontier models. Quick comparison with GPT-4o: +----------------+-------+-------+ | Metric | GPT-4o| Llama | | | | 3.1 | | | | 405B | +----------------+-------+-------+ | MMLU | 88.7 | 88.6 | | GPQA | 53.6 | 51.1 | | MATH | 76.6 | 73.8 | | HumanEval | 90.2 | 89.0 | | MGSM | 90.5 | 91.6 | +----------------+-------+-------+
This nodel is not “open source”, free to use maybe.
Re: Llama 3.1
#128Does anyone know why they haven't released any 30B-ish param models? I was expecting that to happen with this release and have been disappointed once more. They also skipped doing a 30B-ish param model for llama2 despite claiming to have trained one.
Why should they?
Unless I'm misremembering, they announced it at one point. It's just giving people more options.
Re: Llama 3.1
#129The biggest win here has to be the context length increase to 128k from 8k tokens. Till now my understanding is there hasn't been any open models anywhere close to that.
It is notable, but it's not alone. Mistral NeMo just released last week with a 128k context window: https://news.ycombinator.com/item?id=40996058
Thanks! Not sure how I missed that :)
Re: Llama 3.1
#130Earlier quoted context omitted.
It is notable, but it's not alone. Mistral NeMo just released last week with a 128k context window: https://news.ycombinator.com/item?id=40996058
Thanks! Not sure how I missed that :)
It's easy to miss things. Trying to keep up with the latest in AI news is like drinking from the firehose -- it's never-ending.