Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Non-determinism in GPT-4 is caused by Sparse MoE

152334h.github.io

Post reply on HN
Print view View original

11–20 of 186 posts

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#11

Post by dekhn »

sebzim4500 wrote: post #9
derwiki wrote: post #7

Earlier quoted context omitted.

GPT4 web chat for two hours a day? I buy that. Using the API repeatedly for the same inputs, eg developing a program, and the non-determinism is hard to miss.

I would imagine that most people use nonzero temperature, so they won't need to look for any explanation for non-determinism.

Literally the first thing I did when I had llama.cpp working was set the temperature to 0 and repeat queries.

(but that's mainly because I'm a weird old scientist with lots of experience with nondeterminism in software).

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#12

Post by alpark3 »

_If_ 3.5 is a MoE model, doesn't that give a lot of hope to open source movements? Once a good open source MoE model comes out, maybe even some type of variation of the decoder models available(I don't know whether MoE models have to be trained from scratch), that implies a lot more can be done with a lot less.

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#13

Post by cubefox »

152334H wrote: post #8
refulgentis wrote: post #4

This is _excellent_ work, I've been adamantly against MoE for a set of reasons, this is the first compelling evidence I've seen that hasn't been on Substack or a bare repeating of rumor. I had absolutely no idea GPT4 was nondeterministic and I use it about 2 hours a day. I can see why a cursory looking wasn't cutting it, they "feel" the same in your memory, a lot of similar vocab usage, but are formatted entirely dif…

Thanks. I'm really no expert (:P) on MoE research; I just noticed what was written in the Soft MoE paper and felt a need to check. The non-deterministic outputs are really similar, yeah, if you check the gist examples I linked https://gist.github.com/152334H/047827ad3740627f4d37826c867a... . This part is at least no surprise, since the randomness should be bounded. I suspect OpenAI will figure out some way to reduce…

I don't think this commitment had any plausibility. Token "probabilities" only have a straightforward probabilistic interpretation for base models. In fine-tuned models, they do no longer represent the probability of the next token given the prompt, but rather how well the next token fulfills the ... tendencies induced by SL and RL tuning. Which is presumably pretty useless information. OpenAI has no intention to provide access to the GPT-4 base model, and they in fact removed API access to the GPT-3.5 base model.

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#14

Post by dudus »

Off topic

> 3 months later, reading a paper while on board a boring flight home, I have my answer.

I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share.

Any tips or sites for someone interested in picking up more science papers to read.

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#15

Post by 152334H »

alpark3 wrote: post #12

_If_ 3.5 is a MoE model, doesn't that give a lot of hope to open source movements? Once a good open source MoE model comes out, maybe even some type of variation of the decoder models available(I don't know whether MoE models have to be trained from scratch), that implies a lot more can be done with a lot less.

I agree, and really hope that Meta is doing something in that vein. Reducing the FLOPs:Memory ratio (as in Soft MoE) could also open the door to CPU (or at least Apple Silicon) inference becoming more relevant.

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#17

Post by dylan604 »

dudus wrote: post #14

Off topic > 3 months later, reading a paper while on board a boring flight home, I have my answer. I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share. Any tips or sites for someone interested in picking up more science papers to read.

I want to know what a non-boring flight would be like

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#18

Post by nerdponx »

dudus wrote: post #14

Off topic > 3 months later, reading a paper while on board a boring flight home, I have my answer. I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share. Any tips or sites for someone interested in picking up more science papers to read.

> I noticed people from hacker news routinely read scientific papers.

Do they? I suspect that most don't, and those that do are either in specialized careers or are engaged in some kind of scientific research.

Some interesting research gets disseminated via Twitter and chatrooms. Or maybe you follow a podcast that mentions new research. But you might also be following new publications from a handful of reputable journals, or following an Arxiv category, or looking through new conference papers. It's very easy to get overwhelmed with new research to read, and not knowing what's worth your time, unless you're already very familiar with the field and well-versed in the material.

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#19

Post by jldugger »

dudus wrote: post #14

Off topic > 3 months later, reading a paper while on board a boring flight home, I have my answer. I noticed people from hacker news routinely read scientific papers. This is a habit I envy but don't share. Any tips or sites for someone interested in picking up more science papers to read.

For just getting started I recommend collections:

1. Ideas That Created The Future[1]. It's a collection of fiftyish classic CS papers, with some commentary.

2. Wikipedia's list[2].

3. Test of Time awards[3]. These are papers that have been around for a while and people still think are important.

4. Best paper awards[4]. Less useful than ToT as not every best paper is actually that good or important, and sometimes the award committees can't see past names or brands for novel research.

5. Survey Journals[5]. Students often get their research started with a literature review and some go the extra step to collect dozens of papers into a summary paper. I subscribe to the RSS feed for that one, and usually one or two are interesting enough to read.

6. Citation mining -- As you read all these, consider their citation list as potential new reading material, or if an old paper leaves you wanting more, use Google Scholar to find a papers that cited what you just read.

[1]: https://www.amazon.com/Ideas-That-Created-Future-Computer/dp...

[2]: https://en.wikipedia.org/wiki/List_of_important_publications...

[3]: https://www.usenix.org/conferences/test-of-time-awards

[4]: https://jeffhuang.com/best_paper_awards/

[5]: https://dl.acm.org/journal/csur

Re: Non-determinism in GPT-4 is caused by Sparse MoE

#20

Post by adql »

tehsauce wrote: post #3
osmarks wrote: post #2

I feel like this introduces the potential for weird and hard-to-implement side channel attacks, if the sequences in a batch can affect the routing of others.

I think you’re right. Would be very hard to exploit I imagine though.

Same thing was said about Spectre-like bugs
Post reply on HN