Does anyone know if this works with llama.cpp?
GGML quantizations are already being uploaded to huggingface, suggesting it works out of the box.
GPTQ files are being uploaded too, meaning exLLaMA also might work.
Llama 2
231–240 of 860 posts
Re: Llama 2
#232Earlier quoted context omitted.
How do you find Whisper numbers, it’s open source yea?
It's not open source
He's making a pun referring to OpenAI's open-sourced Whisper voice recognition model:
Re: Llama 2
#233Earlier quoted context omitted.
I very much doubt that.
If they want to own the whole stack, I don't think they have much to work with. Their highest-end server chip is a duplex laptop SOC, with maxed-out memory that doesn't even match the lowest-end Grace CPU you can buy (nevermind a fully-networked GH200). Their consumer offerings are competitive, but I don't think Apple Silicon or CoreML is ready to seriously compete with Grace and CUDA.
While Apple silicone may not be there for training, I think it's probably there for inference. I expect next years device models to launch with exclusive support for Apples own LLM based Siri.
Re: Llama 2
#234Hey HN, we've released tools that make it easy to test LLaMa 2 and add it to your own app! Model playground here: https://llama2.ai Hosted chat API here: https://replicate.com/a16z-infra/llama13b-v2-chat If you want to just play with the model, llama2.ai is a very easy way to do it. So far, we’ve found the performance is similar to GPT-3.5 with far fewer parameters, especially for creative tasks and interactions. Dev…
I'm wondering how do people compare different models? I've been trying chatGPT 3.5, bing chat (chatgpt 4 I believe?), and bard, and now this one, and I'm not sure if there's a noticeable difference in terms of "this is better"
It depends -- do you mean as a general end-user of a chat platform or do you mean to include a model as part of an app or service?
As an end user, what I've found works in practice is to use one of the models until it gives me an answer I'm unhappy with. At that point I'll try another model and see whether the response is better. Do this for long enough and you'll get a sense of the various models' strengths and weaknesses (although the tl;dr is that if you're willing to pay GPT-4 is better than anything else across most use cases right now).
For evaluating models for app integrations, I can plug an open source combined playground + eval harness I'm currently developing: https://github.com/openpipe/openpipe
We're working on integrating Llama 2 so users can test it against other models for their own workloads head to head. (We're also working on a hosted SaaS version so people don't have to download/install Postgres and Node!)
Re: Llama 2
#235Well that is lamer than expected. The RLHF censorship was expected, but no 30B model, and single digit benchmark improvements with 40% more data? Wat. Some of the community fine tunes managed better than that. The 4k context length is nice, but RoPE makes it irrelevant anyway. Edit: Ah wait, it seems like there is a 34B model as per the paper: "We are releasing variants of Llama 2 with 7B, 13B, and 70B parameters. We…
The context extending methods still hurt perplexity/quality some. The longer the base model is, the more effective the context extending finetunes/post training tricks will be.
Sure it does, it's not magic. But the alternative is to start dropping out text out of context entirely, which is arguably far worse.
As someone else mentioned, this is probably more due to Llama 2 being already in training when this was figured out and it's not fully accepted yet, but I wouldn't be surprised if there was LLama 3 with out of the box dynamically scaled context at some point.
Re: Llama 2
#236Earlier quoted context omitted.
Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. > I wouldn't be surprised if Amazon does as well. I would - they are not a very major player in this space. TikTok also meets this definition and probably doesn't have LLM.
> Google's model is not as capable as llama-derived models, so I think they would actually benefit from this. Google's publically available model isn't as capable. But they certainly have models that are far better already in house.
Comments like this remind me of the old-timers from IBM saying "but wait, we invented the PC! and the cloud! and..."
Gotta put products in the market, or it didn't happen...
Re: Llama 2
#237Earlier quoted context omitted.
Google has far better models than llama based models. They just simply don't put them facing the public. It is pretty ridiculous that they essentially just set a marketing team with no programming experience to write Bard, but that shouldn't fool anyone into believing they don't have capable models in Google. If Deepmind were to actually provide what they have in some usable form, it would likely be quite good. Despi…
Hard disagree. Google has made it plainly clear that they don't have anything useable in this space. Bard scores below all other commercial model. Google is getting the asses handed to them, badly. I figured that the code red would whip them into shape but the rot runs deep.
> Google has made it plainly clear that they don't have anything useable in this space.
Google hasn't made their best models public because they're too expensive to run for free.
> Google is getting the asses handed to them, badly.
Bard has 30M active users and isn't even available in large parts of the world. They're in 2nd place - when they were pretty late to the game - that's an odd way to say someone is getting their ass handed to them.
Re: Llama 2
#238Very cool! One question, is this model gimped with safety "features"?
The LLaMA chat model is, the base model is not.
Re: Llama 2
#239Why doesn't FB create an API around their model and launch OpenAPI competitor? It is not like they don't have resources, and the learnings (I am referring to actual learning from users' prompts) will improve their models over time.
Because Ads are more profitable than APIs
Re: Llama 2
#240Earlier quoted context omitted.
Seeing a16z w/early access, enough to build multiple tools in advance, is a very unpleasant reminder of insularity and self-dealing of SV elites. My greatest hope for AI is no one falls for this kind of stuff the way we did for mobile.
Any entity working on something in beta has early access anyone could hypothetically email out in grunt work and get involved in a project and get early access therefore this logic when tested against a third party example such as any engineer this argument isn't valid as any engineer entitled or not could have early access
Anyone could worm their way into the project so it's not closed? It's a closed beta with invites to a select few. Some could cross that barrier but the truth is very few will be invited to the inner circle. This started as a private project and painted itself with an open source brush for advertising.