91–99 of 99 posts
1. Ask agent to write a few paragraphs.
2. Notice that it's terrible and it would be easier to rewrite it myself than fix it.
3. Actually be motivated to write.Earlier quoted context omitted.
That's not brainstorming, that's rubber ducking.
No, its better. The reason is the LLM will say or add things you wouldnt have said or added on your own. Wether right or wrong, what it adds, it adds novel input to your thought stream and helps prevent you getting stuck. Ive found myself surprised from time to time with questions an LLM asked, or the justifications provided when pushing for a different path. If you've never talked with a highly opinionated LLM, I un…
Continually asking the LLM to create exhaustive lists of responses to new ideas (big or small) as they first appear, remaining apropos to the running context, is wonderful.
Only one unexpected connection out of 20 or 50 is fantastic, when you can iterate quickly. Brainstorming is the tradeoff of accepting lots of "failure" ideas, in order to generate more serendipitous and creative ideas.
As I use it, the LLM accelerates, augments, and documents my brainstorming session.
--
For some reason, I find it easy to push LLMs to be helpful in ways other people often say "LLMs can't do that".
It is just a matter of setting up the right context, being the right guide, and iterating. They are often faster, aware of more concepts, and more versatile (in terms of fulfilling roles), than any single human.
Their biggest weakness is having a short context. With some work, multiple sessions can mitigate that. Like getting help on a problem from a series of people, where you have to communicate the progression of context to each.
The second weakness is their default to conventional responses. But that is easily overcome by iterative/patient pushing for more creativity responses. Within a session, you can see the model transitioning to more creative as originality goes up, and it gets increasingly "emotionally excited".
They start having "fun", and get more adventurous, as epiphanies emerging from either of you go up. This is not just a funny artifact of how we behave, but also a great barometer for achieving LLM "creative mode".
No, they won’t come up with a totally new set of concepts and axioms that are totally unconnected to any human set… and you don’t want them to. Because those are the actual hallucinations. New ideas need to be novel, but they still need to fit into the rest of the set.
It’s remarkable just how much people race to be pessimistic about LLMs, even when the pessimistic assertions are obviously and factually incorrect. Like yes we know LLMs have “hallucinations”, which is a loaded term to begin with, because the pessimists literally never shut up about it, even though it’s a fairly trivial overall problem given the immense overall utility.
Some people are clearly so very very emotionally invested in there being nothing productive or useful about AI if it’s not AGI or if it needs the slightest amount of correction or oversight.
> LLMs won’t behave in a more creative and independent way (as we hoped), but more susceptible to issues like hallucination. What is creativity if not what people call hallucination? Horribly named phenomenon, btw.
I tried asking an LLM for assistance with Chef Policyfiles today (more specific than that of course), and the response was as if it was for a non-existent product rather than Chef. It even included a code block that it claimed I could run on repl.it if I had an account, which resembled a resource block in chef (policies are not a kind of resource). If a coworker confidently gave me code that is not only wrong, but re…
I’ll add this here for the fake internet points but also as a general PSA. Most people are not aware that “Brainstorming” although popularized as a process since its origins in the 1960s… is actually Step 2. Most people are unaware that Step 1 is called “Questionstorming”. At the heart of the process is leveraging divergent and convergent modes of thinking which is done both to generate questions (and select the most…
Fantastic context. I love hourglass shaped brainstorming, and the “questions” prompt is a solid one!
Earlier quoted context omitted.
Fantastic context. I love hourglass shaped brainstorming, and the “questions” prompt is a solid one!
With LLMs, having questions is more important than having answers. When you run out of questions, the exploration stops.
Earlier quoted context omitted.
With LLMs, having questions is more important than having answers. When you run out of questions, the exploration stops.
I just keep prompting with "continue" when that happens.
This causes the LLM to orbit around some point in the latent space, but doesn't cause it to explore. You have to tell it to pick a direction and there is no better direction than a question.
Earlier quoted context omitted.
My bad, looked it up and meant to edit it in, dunno what happened: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC10858891/
The methodology of this paper is seriously suspect. First they have a very small sample size of around n = 150. Second, they don't represent average husband, they use people working on mechanical Turk for $9 an hour. Finally, they don't have any standard measures of divergent thought - they just use 4 different types of question answering problems, and those are supposed to measure creativity. Whether they do or don'…