321–330 of 352 posts
The post really reminds me of a component of a platform I’m currently building. The problem really with this is finding not just good questions that do not discriminate individual models but also providing a good sample size (eg not just 60) to get really some meaningful results. And even if you have those, there is a drift in the quality of responses. I'm the founder of Pulze.ai, a B2B SaaS Dynamic LLM Automation Pl…
Playground and account are for free
Earlier quoted context omitted.
[flagged]
When searching for them all I can find is this thread.
The same way people on HN often (ignore the HN guidelines to) say that a comment here belongs on Reddit, not because it's literally copied from Reddit but because it's a low-effort comment that's technically a joke but barely funny.
> Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have? The site reports every LLM as getting this wrong. But GPT4 seems to get it right for me: > Sally has 3 brothers. Since each brother has 2 sisters and Sally is one of those sisters, the other sister is the second sister for each brother. So, Sally has 1 sister.
From the ChatGPT app, using GPT-4 model, using my own custom prompt for such puzzles: “You are better than Marilyn Vos Savant at solving brain teaser logic puzzles step by step. Sally (a girl) has 3 brothers. Each brother has 2 sisters. How many sisters does Sally have?” Certainly! Let's break it down step by step: 1. Sally has 3 brothers. 2. Each of those brothers has two sisters. Since all the brothers share the sa…
"Sally has 2 sisters. The three brothers are all siblings, so when it's said that each brother has 2 sisters, it means they all share the same two sisters - one of them being Sally."
Earlier quoted context omitted.
So I guess here my question is why a GPU would perform accumulations in a nondeterministic way where the non-associativity of FP arithmetic matters. You could require that a + b + c always be evaluated left to right and then you've got determinism, which all things being equal is desirable. Presumably because relaxing that constraint allows for some significant performance benefits, but how? Something like avoiding k…
There has been speculation that GPT4 is a mixture of experts model, where each expert could be hosted on a different machine. As those machines may report their results to the aggregating machine in different orders then the results could be summed in different orders.
Have I completely misunderstood, does Mixture of Experts somehow involve the different experts actually collaborating on the raw computation together?
Could anyone share a recommendation for what to read to learn more about MoE generally? (Ideally that's understandable by someone like me that isn't an expert in LLMs/ML/etc.)
Earlier quoted context omitted.
No doubt. But, considering they all get the question dead wrong, including MPT 30B, I'm inclined to think this question hasn't been entered into training data for most LLMs yet.
It could very well be in the data, but not captured via learning/encoding.
Earlier quoted context omitted.
Floating point math is not associative: (a + b) + c != a + (b + c) This leads to different results from accumulating sums in different orderings. Accumulating in different ordering is common in parallel math operations.
So I guess here my question is why a GPU would perform accumulations in a nondeterministic way where the non-associativity of FP arithmetic matters. You could require that a + b + c always be evaluated left to right and then you've got determinism, which all things being equal is desirable. Presumably because relaxing that constraint allows for some significant performance benefits, but how? Something like avoiding k…
This is sort of a deep topic, so it's hard to give a concise answer but as an example: CuBLAS guarantees determinism, but only for the same arch and same library version (because the best performing ordering of operations depends on arch and implementation details) and does not guarantee it when using multiple streams (because the thread scheduling is non-deterministic and can change ordering).
Determinism is something you have to build in from the ground up if you want it. It can cost performance, it won't give you the same results between different architectures, and it's frequently tricky to maintain in the face of common parallel programming patterns.
Consider this explanation from the pytorch docs (particularly the bit on cuda convolutions):
Earlier quoted context omitted.
When searching for them all I can find is this thread.
I assume they meant that they're lines which are easy to think of and not very funny, not that they're directly copied. The same way people on HN often (ignore the HN guidelines to) say that a comment here belongs on Reddit, not because it's literally copied from Reddit but because it's a low-effort comment that's technically a joke but barely funny.
Earlier quoted context omitted.
> People say that "they are literally Markov chains", but anyone who has looked at the code for LLMs knows that they are more complicated than that. They are literally Markov chains according to the mathematical definition. The code is complicated. Having complicated code doesn't mean it's not literally a Markov chain. > I implemented Markov chains in BASIC in about ten lines of code in the 1980s on a 1 Mhz 64K Apple…
How big is the state space of the Markov chain corresponding to a LLM generating a sequence of tokens? Wouldn't it be (size of the vocabulary)^(size of the context window), i.e. ~ (100k)^(4k)? How useful is it to conceptualize LLMs as Markov chains at that point? For example, is there a result about Markov chains with interesting implications for LLMs?
Maybe you are used to Markov chains being shitty at language so you are confused how an LLM can be a Markov chain even though it's good at language and has some amazing emergent cognitive capabilities. That's a problem with your conception of Markov chains, it's not an argument that LLMs aren't Markov chains.
Finally, a Markov chain with a context space that cannot be practically iterated over (e.g. all possible 10k token contexts) can still be useful in ways that are shared with smaller Markov chains, even though if this weren't true it would still be a Markov chain. For example you can greedily generate tokens from it, calculate likelihoods, do some beam search, select multiple choice tokens, etc.
Earlier quoted context omitted.
I assume they meant that they're lines which are easy to think of and not very funny, not that they're directly copied. The same way people on HN often (ignore the HN guidelines to) say that a comment here belongs on Reddit, not because it's literally copied from Reddit but because it's a low-effort comment that's technically a joke but barely funny.
I think the first one at least was rather funny.
Earlier quoted context omitted.
Eh, it's not D&K gone berserk, it's what happens when you attempt to compress reality down to a single dimension (text). If you're doing a haiku, you will likely subvocalize it to ensure you're saying it correctly. It will be interesting when we get multimodal AI that can speak and listen to itself to detect things like this.
The problem isn’t just that everything is text. It’s that everything is a Fourier transform of text in such a way that it’s not actually possible for an LLM to learn to count syllables.
Imagine you have a lot more computing resources in a multimodal LLM. It sees your request of count the syllables and realizes it can't do them from text alone (hell I can't and have to vocalize it). It then sends your request to a audio module and 'says' the sentence, then another listening module that understand syllables 'hears' the sentence.
This is how it works in most humans, now if you do this every day you'll likely make some kind of mental shortcut to reduce the effort needed, but at the end of the day there is no unsolvable problem on the AI side.