Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

LLMs are not suitable for brainstorming

piaoyang0.wordpress.com

Post reply on HN
Print view View original

81–90 of 99 posts

Re: LLMs are not suitable for brainstorming

#81

Post by little_name »

dragonwriter wrote: post #79
hobobaggins wrote: post #49

Earlier quoted context omitted.

> people do the same thing You're right, most people do that, but that's because they haven't trained themselves to be inventors. This is a skill that definitely needs to be developed more, but finding truly creative (and sometimes backwards-seeming) solutions can sometimes mean thinking about problems in a completely different way. However, all LLM's do this. That's one of the points of the article.

> However, all LLM’s do this. LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.) Can this be leveraged to build a framework around an LLM for brainstorming usefully? I’m not sure. But “Nope, because LLMs just directly adhere to existing patterns” is definitely not a useful answer to that question. It is the kind of answer someone who just p…

> LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.)

I think people often forget the behavior of instruction following is still based on SFT training data. This is exactly "adherence to training patterns". Prompt engineering is not elixir, it's just a way to utilize the patterns seen in training data.

Re: LLMs are not suitable for brainstorming

#82

Post by dragonwriter »

little_name wrote: post #81
dragonwriter wrote: post #79

Earlier quoted context omitted.

> However, all LLM’s do this. LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.) Can this be leveraged to build a framework around an LLM for brainstorming usefully? I’m not sure. But “Nope, because LLMs just directly adhere to existing patterns” is definitely not a useful answer to that question. It is the kind of answer someone who just p…

> LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.) I think people often forget the behavior of instruction following is still based on SFT training data. This is exactly "adherence to training patterns". Prompt engineering is not elixir, it's just a way to utilize the patterns seen in training data.

> Prompt engineering is not elixir

I’m not talking about prompt engineering when I talk about per request tuning of how closely it follows established patterns, I’m talking about inference parameters (temperature, top_p, top_k, are common for most models and ways of calling them, some others may be available, too.)

Re: LLMs are not suitable for brainstorming

#84

Post by [deleted] »

dragonwriter wrote: post #82
little_name wrote: post #81

Earlier quoted context omitted.

> LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.) I think people often forget the behavior of instruction following is still based on SFT training data. This is exactly "adherence to training patterns". Prompt engineering is not elixir, it's just a way to utilize the patterns seen in training data.

> Prompt engineering is not elixir I’m not talking about prompt engineering when I talk about per request tuning of how closely it follows established patterns, I’m talking about inference parameters (temperature, top_p, top_k, are common for most models and ways of calling them, some others may be available, too.)

[deleted]

Re: LLMs are not suitable for brainstorming

#85

Post by little_name »

dragonwriter wrote: post #82
little_name wrote: post #81

Earlier quoted context omitted.

> LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.) I think people often forget the behavior of instruction following is still based on SFT training data. This is exactly "adherence to training patterns". Prompt engineering is not elixir, it's just a way to utilize the patterns seen in training data.

> Prompt engineering is not elixir I’m not talking about prompt engineering when I talk about per request tuning of how closely it follows established patterns, I’m talking about inference parameters (temperature, top_p, top_k, are common for most models and ways of calling them, some others may be available, too.)

There is a thread below discussing tuning temperature and why it's not tied to creativity. I think the point is temperature and other sampling parameters don't have any inherent mechanism to control the adherence to training pattern as a whole (all top-X candidates are all from training data). Deviating from the max probability in next token generation has no relation with thinking out of box. It does give more generation results that may be lucky to hit something, but I doubt the cost effectiveness

Re: LLMs are not suitable for brainstorming

#86

Post by imtringued »

dragonwriter wrote: post #79
hobobaggins wrote: post #49

Earlier quoted context omitted.

> people do the same thing You're right, most people do that, but that's because they haven't trained themselves to be inventors. This is a skill that definitely needs to be developed more, but finding truly creative (and sometimes backwards-seeming) solutions can sometimes mean thinking about problems in a completely different way. However, all LLM's do this. That's one of the points of the article.

> However, all LLM’s do this. LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.) Can this be leveraged to build a framework around an LLM for brainstorming usefully? I’m not sure. But “Nope, because LLMs just directly adhere to existing patterns” is definitely not a useful answer to that question. It is the kind of answer someone who just p…

How will increasing temperature improve the arithmetic performance of the model with numbers it has never seen? When prompted the right way, the model has trouble sticking to the answer that is implied by its chain of thought process. For some strange reason it knows how to perform the task, but when it is time to output the answer, it decides to recalculate it from scratch using attention. Before you complain about me benchmarking arithmetic performance. Consider that I could have defined any algebraic structure and I could have made the exact same complaint. It's just that arithmetic is very easy to verify using a calculator. Dismissing the easy arithmetic also dismisses all operations on any algebra.

The obvious problem is that these logic and arithmetic operations don't really need exhaustive training examples. Training the LLM is supposed to teach it a method or process by which it arrives at the answer, not just the question answer pair itself, which tends to only lead to very good approximate results. That is ok when using language, since your exact words don't really matter if you can express yourself in another way, but when it comes to algebras, accuracy is key.

It really is strange that when one wishes for a more capable LLM that one is told how stupid one is for wanting such a thing. What exactly is so wrong about giving the model a price list and an example of an order at a restaurant and then asking how much the whole order costs and expecting the correct total sum? For some strange reason I'm getting numbers that would make sense if I generated the text in a vacuum without the price list, as if they are following some existing pattern...

Re: LLMs are not suitable for brainstorming

#87

Post by imtringued »

refulgentis wrote: post #62
hobobaggins wrote: post #49

Earlier quoted context omitted.

> people do the same thing You're right, most people do that, but that's because they haven't trained themselves to be inventors. This is a skill that definitely needs to be developed more, but finding truly creative (and sometimes backwards-seeming) solutions can sometimes mean thinking about problems in a completely different way. However, all LLM's do this. That's one of the points of the article.

There was an actual peer-reviewed article from an institution yesterday showing, numerically, robustly, human evaluated, that LLMs are better at creative thinking than humans. I don't think handwaving about exclusively outputting training data and random punditry via article really holds up here. It's wrong for so many reasons, from the training data having more perspectives, to the "Fat Tony" test, empirically, we c…

There is a difference between a verbatim answer and an answer in the shape of a verbatim answer. The latter is not much better if you actually want to do new things.

Re: LLMs are not suitable for brainstorming

#88

Post by dragonwriter »

imtringued wrote: post #86
dragonwriter wrote: post #79

Earlier quoted context omitted.

> However, all LLM’s do this. LLM’s degree of adherence to training patterns can be directly tuned, per request (up to whatever the maximum it is capable of.) Can this be leveraged to build a framework around an LLM for brainstorming usefully? I’m not sure. But “Nope, because LLMs just directly adhere to existing patterns” is definitely not a useful answer to that question. It is the kind of answer someone who just p…

How will increasing temperature improve the arithmetic performance of the model with numbers it has never seen? When prompted the right way, the model has trouble sticking to the answer that is implied by its chain of thought process. For some strange reason it knows how to perform the task, but when it is time to output the answer, it decides to recalculate it from scratch using attention. Before you complain about…

> How will increasing temperature improve the arithmetic performance of the model with numbers it has never seen?

“Brainstorming” and “doing precise arithmetic” are wildly different tasks. It is a complete non-sequitur to ask how a suggestion about the former will help the latter.

Re: LLMs are not suitable for brainstorming

#89

Post by pixelsort »

dahart wrote: post #12

> The reason [LLMs are not a good tool to do truly effective brainstorming] is LLMs are trained to follow existing patterns in the human-produced corpus, and not natively taught to “brainstorm”. The problem with this argument is that people do the same thing, we’re not that great at brainstorming either. When we brainstorm in groups, we’re just bringing multiple points of view together. The more data LLMs are trained…

I came here to say pretty much the same. It is so tiring to hear opinions like these from technically-minded people who clearly lack "better-than-average" creativity and lateral thinking skills. LLMs are garbage-in garbage-out in terms of brainstorming. If you aren't good at brainstorming without LLMs then they won't be much use. So many people are just seeing their own limits reflected back at them.

When I'm brainstorming math or cryptography or philosophy with an LLM it is only because of the connections and leaps that I introduce that leads to novel and amusing discoveries.

You have to develop successful patterns. Coaxing and condensing. Drilling into numbered lists. Asking for tables with progressively more columns and altered sorting. You have to know the spatial reasoning limits of the model. Asking for enumeration and disabling of social niceties. The number of skills needed is probably beyond most people's imaginations.

Re: LLMs are not suitable for brainstorming

#90

Post by gavanwilhite »

infinite-hugs wrote: post #11

I’ll add this here for the fake internet points but also as a general PSA. Most people are not aware that “Brainstorming” although popularized as a process since its origins in the 1960s… is actually Step 2. Most people are unaware that Step 1 is called “Questionstorming”. At the heart of the process is leveraging divergent and convergent modes of thinking which is done both to generate questions (and select the most…

Fantastic context. I love hourglass shaped brainstorming, and the “questions” prompt is a solid one!
Post reply on HN