Large Language Models Are Human-Level Prompt Engineers
openreview.net
Large Language Models Are Human-Level Prompt Engineers
1–10 of 54 posts
Re: Large Language Models Are Human-Level Prompt Engineers
#2Capable enough LLMs are human level for lots of things. Reinforcement learning from ai feedback is a thing (the anthropic claude models use that). Strictly speaking, it's not necessary to have humans in the loop for a lot of these things.
Some are hesitant to admit we've created human level general intelligence but saying otherwise doesn't really hold up to scrutiny.
Re: Large Language Models Are Human-Level Prompt Engineers
#3Capable enough LLMs are human level for lots of things. Reinforcement learning from ai feedback is a thing (the anthropic claude models use that). Strictly speaking, it's not necessary to have humans in the loop for a lot of these things. Some are hesitant to admit we've created human level general intelligence but saying otherwise doesn't really hold up to scrutiny.
Maybe for oversight and liability.
Re: Large Language Models Are Human-Level Prompt Engineers
#4This is pretty alarming tbh. Anyone already making a pivot out of SWE?
Re: Large Language Models Are Human-Level Prompt Engineers
#5This is pretty alarming tbh. Anyone already making a pivot out of SWE?
Article is about prompt engineers (3 month old job type), not swes.
Re: Large Language Models Are Human-Level Prompt Engineers
#6Capable enough LLMs are human level for lots of things. Reinforcement learning from ai feedback is a thing (the anthropic claude models use that). Strictly speaking, it's not necessary to have humans in the loop for a lot of these things. Some are hesitant to admit we've created human level general intelligence but saying otherwise doesn't really hold up to scrutiny.
I see people saying things like this but I have yet to see anyone show data for a non-trivial workflow with human-level accuracy over a wide range of inputs, without a human in the loop.
Re: Large Language Models Are Human-Level Prompt Engineers
#7Capable enough LLMs are human level for lots of things. Reinforcement learning from ai feedback is a thing (the anthropic claude models use that). Strictly speaking, it's not necessary to have humans in the loop for a lot of these things. Some are hesitant to admit we've created human level general intelligence but saying otherwise doesn't really hold up to scrutiny.
I see people saying things like this but I have yet to see anyone show data for a non-trivial workflow with human-level accuracy over a wide range of inputs, without a human in the loop.
Counter argument: this may be a matter of incremental improvement. The breakthroughs may all be behind us.
It’s like saying you haven’t yet seen a 1000 mile range EV for under $100k. No you can’t buy such a thing now but it’s clearly possible and we know how to get there by just continuing to grind on battery technology and scale manufacturing.
AGI may be at the place a moon landing was in 1950, not where it was in 1900 or 1850.
Re: Large Language Models Are Human-Level Prompt Engineers
#8I can’t find the link to the paper right now, but after reading about how LLMs perform better with task breakdowns, I vastly improved my integrations by having ChatGPT generate prompts that decompose a general task into a series of tasks based on a sample input and output. I haven’t needed to make a self-refining system (one or two rounds of task decomposition and refinement resulted in the expected result for all inputs), but I would assume this is fairly trivial and that AIs can do it better than humans.
This is also an area where I expect OpenAI will continue to demolish the competition. The ability to recursively generate and process large prompts is truly nuts. I tried swapping in some of the “high-performing” LLama models and they all choked on anything more than a paragraph.
Re: Large Language Models Are Human-Level Prompt Engineers
#9Earlier quoted context omitted.
I see people saying things like this but I have yet to see anyone show data for a non-trivial workflow with human-level accuracy over a wide range of inputs, without a human in the loop.
Counter argument: this may be a matter of incremental improvement. The breakthroughs may all be behind us. It’s like saying you haven’t yet seen a 1000 mile range EV for under $100k. No you can’t buy such a thing now but it’s clearly possible and we know how to get there by just continuing to grind on battery technology and scale manufacturing. AGI may be at the place a moon landing was in 1950, not where it was in 1…
You can actually buy a 1000 km range EV for $160k now (MB EQXX). Just as a by the way. :)
At this price point it actually has nothing to do with grinding on battery tech and scale manufacturing, the limiting factor is physics. You can only make it so aerodynamic before you hit diminishing returns or it stops looking like a car. You can only make it so lightweight. And so forth.
This is vaguely as good as it can get and we can say that because we understand how it all works.
LLMs on the other hand invite all kinds of magical thinking around unlimited potential because we poked them with a stick and something interesting comes out it must mean that if we poke it just right we will get an AGI. That just doesn't logically follow from what we know of it so far.
Re: Large Language Models Are Human-Level Prompt Engineers
#10This is pretty alarming tbh. Anyone already making a pivot out of SWE?
I am a SWE currently making a pivot into business owner.
The future I see is that everyone is about to become a CEO with a personal assistant that can run a business.
So I'm going to start building something of my own starting now.