I’ll add this here for the fake internet points but also as a general PSA. Most people are not aware that “Brainstorming” although popularized as a process since its origins in the 1960s… is actually Step 2. Most people are unaware that Step 1 is called “Questionstorming”. At the heart of the process is leveraging divergent and convergent modes of thinking which is done both to generate questions (and select the most promising ones) and answers… you’re welcome :D
LLMs are not suitable for brainstorming
11–20 of 99 posts
Re: LLMs are not suitable for brainstorming
#12> The reason [LLMs are not a good tool to do truly effective brainstorming] is LLMs are trained to follow existing patterns in the human-produced corpus, and not natively taught to “brainstorm”.
The problem with this argument is that people do the same thing, we’re not that great at brainstorming either. When we brainstorm in groups, we’re just bringing multiple points of view together. The more data LLMs are trained on, the more viewpoints it might be able to bring that you haven’t considered.
That said, LLMs and all NNs so far are built to interpolate, and they are bad and have unbounded error when extrapolating outside their training examples. That is a good reason to not expect today’s AI to come up with new ideas.
Re: LLMs are not suitable for brainstorming
#13This is utterly wrong. The strength of LLMs is that they have very wide knowledge, so while your specialist knowledge might surpass them in a particular area it will know more than you across other topics.
When brainstorming this wide knowledge is what you want. The trick is (as always) better prompting to push it hard - so things like "consider parallels in similar situations in other fields" are useful.
Re: LLMs are not suitable for brainstorming
#14I find LLMs to be useful tools for brainstorming. I have often inadvertently found myself brainstorming just by explaining something challenging I am working on to a colleague with no expertise in my area. Just explaining forces new perspectives and new ideas. Similarly, LLMs are captive audiences for throwing out ideas and playing with them. The interaction creates something more than just talking to myself or a whi…
I agree with this, but also get so irritated with the endless praise. Like, tell me I'm wrong and tell me why. If I present a bad idea, tell me that. Or even like "huh? explain" Looking forward to ramping up the honesty parameter. Hoping OpenAI's new voice model trivializes this so I don't need to prompt engineer.
Curious what happens if you set custom instructions to correct that. Does it work?
Re: LLMs are not suitable for brainstorming
#15>However, here I would like to argue that (especially in cutting edge scenarios) LLMs are not a good tool to do truly effective brainstorming. Great title, they baited and switched. 95% of the time I don't need effective brainstorming, I need a bunch of ideas, let me pick the best, and move on. If its a real engineering problem, then I need >truly effective brainstorming.
I recently spoke with a cousin about their current project decommissioning an oil and gas platform over the next four years (ie. real engineering (in reverse?)).
Their comment was that ChatGPT can't write final reports but it's extremely useful for seeding brainstorming sessions with real human engineers.
It can generate a list of health, safety, and environmental concerns related to the project that can be distributed and act as a "starter dot point list" that engineers can agree to, expand on, reject as being just silly, and add to as tengential thoughts arise.
I'd argue that makes it useful in the context of real engineering brainstorming .. and that it's being used that way on multi million dollar projects in a billion dollar domain.
Re: LLMs are not suitable for brainstorming
#16>However, here I would like to argue that (especially in cutting edge scenarios) LLMs are not a good tool to do truly effective brainstorming. Great title, they baited and switched. 95% of the time I don't need effective brainstorming, I need a bunch of ideas, let me pick the best, and move on. If its a real engineering problem, then I need >truly effective brainstorming.
There's a lot of this with current AI - someone will write a piece titled something like "LLMs are not suitable for brainstorming" or "GPT4 is useless for programmers" or "AI image generation has no use cases" - and what they say in the body is that they're not perfect for brainstorming, GPT4 can't write an entire app perfectly from one prompt, and Midjourney doesn't let you take a picture from your mind's eye and put it on paper.
Re: LLMs are not suitable for brainstorming
#17not suitable = better-than-average?
Re: LLMs are not suitable for brainstorming
#18I find LLMs to be useful tools for brainstorming. I have often inadvertently found myself brainstorming just by explaining something challenging I am working on to a colleague with no expertise in my area. Just explaining forces new perspectives and new ideas. Similarly, LLMs are captive audiences for throwing out ideas and playing with them. The interaction creates something more than just talking to myself or a whi…
I agree with this, but also get so irritated with the endless praise. Like, tell me I'm wrong and tell me why. If I present a bad idea, tell me that. Or even like "huh? explain" Looking forward to ramping up the honesty parameter. Hoping OpenAI's new voice model trivializes this so I don't need to prompt engineer.
A simple "Be brutally honest" at the end works surprisingly well.
Re: LLMs are not suitable for brainstorming
#19Counterpoint: LLMs are very suitable for brainstorming if prompted appropriately
LLMs are going to give you the consensus reality (average opinion? average facts?) but you can easily steer it into offbeat, controversial and esoteric areas of it's training with the right prompts
Re: LLMs are not suitable for brainstorming
#20I find LLMs to be useful tools for brainstorming. I have often inadvertently found myself brainstorming just by explaining something challenging I am working on to a colleague with no expertise in my area. Just explaining forces new perspectives and new ideas. Similarly, LLMs are captive audiences for throwing out ideas and playing with them. The interaction creates something more than just talking to myself or a whi…
I agree with this, but also get so irritated with the endless praise. Like, tell me I'm wrong and tell me why. If I present a bad idea, tell me that. Or even like "huh? explain" Looking forward to ramping up the honesty parameter. Hoping OpenAI's new voice model trivializes this so I don't need to prompt engineer.
Me: “I’d like to start a business harvesting cat shit to fight global warming. People would pay me to take their cat shit and bury it because maybe it has carbon in it or something IDK I’m not a scientist. I would give them carbon credits in the form of a cat-shit-sequestration crypto token my business created. I’d pick up the cat shit by bicycle to save carbon emissions. Please help me develop a business plan.”
Chargpt: “that’s a great idea for a business! Here are some suggestions…”
Not a real exchange, but god, it does feel that way sometimes. One reason you can’t trust the damn thing is it’s too positive.