when will the llm race peak? have we peaked already?
When it's superhuman.
XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
31–40 of 96 posts
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#32From all the experimentation I've done, 7B parameter models just don't seem to be able to produce useful output reliably enough for my use cases. What use cases do people have for these smaller LLM's?
The use case is people who got some fear of missing out on AI and also they are not capable of seeing the difference between the model outputs themselves. Maybe you will say I'm elitist but it's disturbing to me how many people literally can't tell the difference between gpt 3 and gpt 4 outputs, it's like the "It's the same picture" meme. Maybe that makes me a hipster gpt connoisseur elitist that I can recognize and…
It doesn't make you an elitist hipster. It puts you in a category of most people on earth. An elist hipster would swear by a custom smaller parameter model finetuned by a wizard for specific tasks and have a spell book of models.
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#33From all the experimentation I've done, 7B parameter models just don't seem to be able to produce useful output reliably enough for my use cases. What use cases do people have for these smaller LLM's?
The main use case is that it's probably the only size consumers can run on their personal devices. If you don't want your data going into an external platform like OpenAI it's the only solution even if it's not very usuable.
You can run big models on the cloud yourself, or with a 3090/4090 quantized.
You don't have to go to openai.
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#34I have no idea what any of these words mean, but I'd like to. Can someone point me in the direction of an "AI for Dipshits"?
It's an open source language model with seven billion parameters (a measure of its complexity), and a longer than typical sequence length of 8K, which allows the provision of more context when querying the model. For example, it allows you to better generate text in someone else's voice by providing a longer example of their work. https://en.wikipedia.org/wiki/Foundation_models
dang so this means that just like my 30yo 4GB hard drive is to my 4GB RAM phone… we ain’t seen nothing yet if we’re still counting these metrics?
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#35From all the experimentation I've done, 7B parameter models just don't seem to be able to produce useful output reliably enough for my use cases. What use cases do people have for these smaller LLM's?
7B LLaMA is a terrible general purpose model, but the finetunes are pretty good at very specific roles, like dialogue/roleplay, a dungeon master bot or even code completion. The metrics are good though, perhaps placing this closer to 13B. And 8K context is huge . When you can stuff that much example text in, it gives the model more to "latch onto," and its also the point where you would start worrying about RAM/VRAM…
would appreciate resources on fine tuning
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#36From all the experimentation I've done, 7B parameter models just don't seem to be able to produce useful output reliably enough for my use cases. What use cases do people have for these smaller LLM's?
> What use cases do people have for these smaller LLM's? None. Training a functionally useless model and releasing it is a great way to demonstrate that your company is hip and current. That way when prospective clients ask about AI you can vaguely gesture at some model that you released and say you employ cutting edge AI experts.
yep, that’s why it’s free.
If it was good, then they’d charge for it.
…what they (and everyone) is gonna do is play with smallish models to iterate on the process for relatively small expense and earn karma.
Then pay big $$$ to make a really good model for internal use and/or an api that people have to pay for.
Tldr; it’s free. It’s by sales force. You should expect it to be a) crippled and b) a loss leader for a paid product.
Not judging; it’s a fair strategy. Just saying: salesforce is not a company that just gives hundreds of thousands of dollars away for nothing.
If you want a good free open model, you’re kidding yourself if you think a corporate giant is going to kiss you on the head and give it to your for free.
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#37Earlier quoted context omitted.
The main use case is that it's probably the only size consumers can run on their personal devices. If you don't want your data going into an external platform like OpenAI it's the only solution even if it's not very usuable.
You can run big models on the cloud yourself, or with a 3090/4090 quantized. You don't have to go to openai.
What’re some models and hardware combos we can run now? I am avoiding to go to OpenAI with my office’s stuff and can use some gpu(s)
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#38I have no idea what any of these words mean, but I'd like to. Can someone point me in the direction of an "AI for Dipshits"?
It's an open source language model with seven billion parameters (a measure of its complexity), and a longer than typical sequence length of 8K, which allows the provision of more context when querying the model. For example, it allows you to better generate text in someone else's voice by providing a longer example of their work. https://en.wikipedia.org/wiki/Foundation_models
The number of parameters seems meaningless when the training sets are dogshit and hardened old gum chipped from the shoes of Gregslist and FleaBay.
OTOH, there ought to be a construction in the form of an web app that can pinch out nonrepetitive, coherent ~100 page trashy romance novels in the style of any author given name with open source or specific text(s) or transcripts with enough original input volume: Churchill, The Unabomber, psycho happy kindergarten child development IEP manual writer, The Dude, Walter (agro gun nut), Bob Ross, Grace Hopper, Ayn Rand, LBJ, The Dalai LaMa%, Hitler, Kanye (Ye), Bhad Bhabie, and the King James Bible. Ethical and generational safety features be damned; it'd be generating fucking^2 art for hilarious entertainment purposes. How does one stretch training input to something that might involve human/computer output validation to discard sticking on repetitive nonsense?
% He never saw that one coming Ow^(3 + i).
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#39when will the llm race peak? have we peaked already?
To quote Dennis Reynolds, it hasn't even begun to peak.
Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens
#40From all the experimentation I've done, 7B parameter models just don't seem to be able to produce useful output reliably enough for my use cases. What use cases do people have for these smaller LLM's?
I feel like this would work well as a local LLM for a home assistant type setup. Fully local instead of having Alexa send everything to the cloud.