Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

blog.salesforceairesearch.com

Post reply on HN
Print view View original

61–70 of 96 posts

Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

#61

Post by fragmede »

jrflowers wrote: post #47
wokwokwok wrote: post #36

Earlier quoted context omitted.

yep, that’s why it’s free. If it was good , then they’d charge for it. …what they (and everyone) is gonna do is play with smallish models to iterate on the process for relatively small expense and earn karma. Then pay big $$$ to make a really good model for internal use and/or an api that people have to pay for. Tldr; it’s free. It’s by sales force. You should expect it to be a) crippled and b) a loss leader for a pa…

> If you want a good free open model, you’re kidding yourself if you think a corporate giant is going to kiss you on the head and give it to your for free. Yep! That makes sense! I would love to be the CEO of the company that does give away an actually useful model and little forehead kisses though. The amount of goodwill that one would generate from that would be astronomical and training costs are getting so low th…

OpenAI did this when releasing Whisper, but I mostly hear sneers about how they're not really open, and no gratitude for the "little kiss". Given that, I don't know that as CEO, I'd be very benevolent.

Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

#63

Post by going_ham »

profsummergig wrote: post #62

If someone could elucidate on what these phrases signify, I'd be very grateful: 1) 7B foundational model 2) 8K length 3) 1.5T tokens

1. The trained model has 7B parameters or weights for each neuron.

2. It can handle upto 8k tokens. Tokens are usually some representation for a word. If your tokens are characters then, "h", "e", "y" represent 3 tokens for hey. Most of the algos use byte pair encoding. For example "hand-le" has two tokens "hand" and "le". This is a very crud example which is enough to give the gist but is not accurate. You can look into byte pair encoding for more details.

3. The token size 1.5T token means they have huge variations for input and output. Simply put, it was trained on large data corpus.

I hope this simplifies it. You can research further if you are interested! Hope it helps!

Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

#64

Post by rolisz »

TOMDM wrote: post #4

From all the experimentation I've done, 7B parameter models just don't seem to be able to produce useful output reliably enough for my use cases. What use cases do people have for these smaller LLM's?

I've had very good results with even a 3.5 B model (Fastchat-T5) for retrieval augmented generation (aka putting information into the context window and letting the model rephrase it).

Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

#66

Post by Tostino »

brucethemoose2 wrote: post #21
TOMDM wrote: post #4

From all the experimentation I've done, 7B parameter models just don't seem to be able to produce useful output reliably enough for my use cases. What use cases do people have for these smaller LLM's?

7B LLaMA is a terrible general purpose model, but the finetunes are pretty good at very specific roles, like dialogue/roleplay, a dungeon master bot or even code completion. The metrics are good though, perhaps placing this closer to 13B. And 8K context is huge . When you can stuff that much example text in, it gives the model more to "latch onto," and its also the point where you would start worrying about RAM/VRAM…

You must have missed the memo... It's now super easy to extend the context of 2k llama models to 8k, 16k, or even 32k with just a small fine tune and a tweak to the code.

You still need the memory to be able to go that high, but it's totally doable.

Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

#67

Post by thomasahle »

going_ham wrote: post #63
profsummergig wrote: post #62

If someone could elucidate on what these phrases signify, I'd be very grateful: 1) 7B foundational model 2) 8K length 3) 1.5T tokens

1. The trained model has 7B parameters or weights for each neuron. 2. It can handle upto 8k tokens. Tokens are usually some representation for a word. If your tokens are characters then, "h", "e", "y" represent 3 tokens for hey. Most of the algos use byte pair encoding. For example "hand-le" has two tokens "hand" and "le". This is a very crud example which is enough to give the gist but is not accurate. You can look…

Don't just post ChatGPT answers as comments on hackernews.

This one doesn't even make any sense. Of course it doesn't have 7B parameters _per_ neuron.

Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

#68

Post by thomasahle »

Tostino wrote: post #66
brucethemoose2 wrote: post #21

Earlier quoted context omitted.

7B LLaMA is a terrible general purpose model, but the finetunes are pretty good at very specific roles, like dialogue/roleplay, a dungeon master bot or even code completion. The metrics are good though, perhaps placing this closer to 13B. And 8K context is huge . When you can stuff that much example text in, it gives the model more to "latch onto," and its also the point where you would start worrying about RAM/VRAM…

You must have missed the memo... It's now super easy to extend the context of 2k llama models to 8k, 16k, or even 32k with just a small fine tune and a tweak to the code. You still need the memory to be able to go that high, but it's totally doable.

What memo/paper does this?

Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

#69

Post by raincole »

going_ham wrote: post #63
profsummergig wrote: post #62

If someone could elucidate on what these phrases signify, I'd be very grateful: 1) 7B foundational model 2) 8K length 3) 1.5T tokens

1. The trained model has 7B parameters or weights for each neuron. 2. It can handle upto 8k tokens. Tokens are usually some representation for a word. If your tokens are characters then, "h", "e", "y" represent 3 tokens for hey. Most of the algos use byte pair encoding. For example "hand-le" has two tokens "hand" and "le". This is a very crud example which is enough to give the gist but is not accurate. You can look…

This is a good example on why StackOverflow banned ChatGPT-generated answers.

Re: XGen-7B, a new 7B foundational model trained on up to 8K length for 1.5T tokens

#70

Post by johanvts »

thomasahle wrote: post #67
going_ham wrote: post #63

Earlier quoted context omitted.

1. The trained model has 7B parameters or weights for each neuron. 2. It can handle upto 8k tokens. Tokens are usually some representation for a word. If your tokens are characters then, "h", "e", "y" represent 3 tokens for hey. Most of the algos use byte pair encoding. For example "hand-le" has two tokens "hand" and "le". This is a very crud example which is enough to give the gist but is not accurate. You can look…

Don't just post ChatGPT answers as comments on hackernews. This one doesn't even make any sense. Of course it doesn't have 7B parameters _per_ neuron.

Don’t be too critical dr. Ahle. Maybe it’s a new single-neuron architecture.
Post reply on HN