Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Exponentially faster language modelling

arxiv.org

Post reply on HN
Print view View original

141–150 of 150 posts

Re: Exponentially faster language modelling

#141

Post by wruza »

godelski wrote: post #130
wruza wrote: post #107

Earlier quoted context omitted.

Is there a place where you guys discuss... things? I'm layman interested in this topic akin to pop-physics/maths, but have no chance to just read papers and "get it". On the other hand, immediately available resources focus more on how-to part of it rather than on what's up overall. Also, do you have something like 3b1b/pbs/nph for it? Content that you can watch and say "well, yep, good job".

I don't have any great recommendations and unfortunately my advice may be not what you want to hear. What I tell my students is "You don't need to know math to build good models, but you need to know math to know why your models are wrong." But this is even a contentious statement within the community. (Personally I'm more interested in exploring what we can build and understand rather than focusing on throwing more…

Thank you very much!

Re: Exponentially faster language modelling

#142

Post by WhrRTheBaboons »

lettergram wrote: post #132
cloudking wrote: post #106

Earlier quoted context omitted.

This is why I'm not understanding the excitement around open source models, they pale in comparison to GPT-4 quality, so I have no use for them until we have something comparable.

We utilize a LLama that frankly competes directly with gpt-4, to the point I can't tell which model we are using (we randomly switch between providers to ensure a robust backend). I think it likely depends on the use case, but many llama models can be returned and there are quite literally thousands of free versions available.

What is your use case?

Re: Exponentially faster language modelling

#143

Post by srs_sput »

godelski wrote: post #130
wruza wrote: post #107

Earlier quoted context omitted.

Is there a place where you guys discuss... things? I'm layman interested in this topic akin to pop-physics/maths, but have no chance to just read papers and "get it". On the other hand, immediately available resources focus more on how-to part of it rather than on what's up overall. Also, do you have something like 3b1b/pbs/nph for it? Content that you can watch and say "well, yep, good job".

I don't have any great recommendations and unfortunately my advice may be not what you want to hear. What I tell my students is "You don't need to know math to build good models, but you need to know math to know why your models are wrong." But this is even a contentious statement within the community. (Personally I'm more interested in exploring what we can build and understand rather than focusing on throwing more…

Thanks!

Re: Exponentially faster language modelling

#144

Post by godelski »

jamesblonde wrote: post #136
godelski wrote: post #81

Earlier quoted context omitted.

I wouldn't be so quick to conspiracy. I'm the author of a work and a famous blog post that trains a particular common architecture much faster (don't want to dox myself too much) and with far fewer parameters, but it has been rejected several times and is now arxiv only. Our most common complaint was "who would use this? Why not just take a large model and tune it?" That question alone held us back a year (had over a…

Unless they were very confident of acceptance, a top research prof would rewrite and resubmit before publishing on arxiv so that others could "build on it" (scoop you at a top conference).

Welcome to ML. And idk, I'd feel pretty confident that a paper that gets so many citations gets accepted. The review system is like a slot machine if you aren't a big tech lab

Re: Exponentially faster language modelling

#146

Post by dartos »

Geee wrote: post #75
WithinReason wrote: post #48

Earlier quoted context omitted.

That's an interesting question. It actually provides a nice way to parallelized training: Pretrain e.g. the first 3 branch levels, which effectively fragments the model into 8 separate parts, which you can continue training across 8 independent servers/nodes with no further communication between the nodes. A central server would run the 1st 3 levels and mark parts of the training set that each node has to train on. M…

I am barely understanding, so a stupid question: Does this also mean that it would be possible to train on parallel GPU-poor setup instead of needing lots of GPU memory / bandwidth on one computer?

Probably not. The paper talks about this as an inference time optimization

Re: Exponentially faster language modelling

#148

Post by lettergram »

WhrRTheBaboons wrote: post #142
lettergram wrote: post #132

Earlier quoted context omitted.

We utilize a LLama that frankly competes directly with gpt-4, to the point I can't tell which model we are using (we randomly switch between providers to ensure a robust backend). I think it likely depends on the use case, but many llama models can be returned and there are quite literally thousands of free versions available.

What is your use case?

https://ipcopilot.ai

Re: Exponentially faster language modelling

#150

Post by simne »

qntty wrote: post #40

According to scientists, we only use 0.3% of our neural networks. Imagine if we could use 100%.

Really good humor :)

But in reality, sparse NN is just loose it's performance, mean loose precision and recall. Precision, means, larger probability of errors; recall - if you work with piece of information, which could consist of few, ie predicates, it will see not all predicates.

To be concrete, for good trained full-scale NN, usually considered 70-90% for precision and for recall; but if use small fraction of weights, usually will got drop of performance to about 40-70%, which is good enough for many cases, considering saves on size and computations.

Post reply on HN