Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Tech’s hottest new job: Prompt engineer

washingtonpost.com

Post reply on HN
Print view View original

81–90 of 123 posts

Re: Tech’s hottest new job: Prompt engineer

#81

Post by super256 »

TrackerFF wrote: post #50
superdisk wrote: post #8

I just can't believe how idiotic such a thing is. It's like calling a person who uses a toaster a Toast Engineer with a salary of 300k.

Not too different from analysts writing SQL queries all day long.

Complex SQL requires knowledge about relational algebra (Cartesian products, set theory, domain relational calculus yada yada) and understanding of how RDBMS and their query planners work. At least, if performance is important to you.

I don't see this in this prompt engineering. In my limited experience (I played a few hours with Stable Diffusion and more hours with the OAI davinci-003 model), you can get good at it within a few days.

Re: Tech’s hottest new job: Prompt engineer

#82

Post by dijit »

deeviant wrote: post #49
dijit wrote: post #45

Earlier quoted context omitted.

I dislike that analogy greatly. Lawyers have a bad reputation, sure, but there’s a lot of education about the interpretation of our law and the absurdly large corpus of legal documentation that must be read in order to even become a lawyer is far and above anything you describe.

So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?

>So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?

No.

A LLM has more opportunity itself to replace a Lawyer, the person typing the prompt is not necessarily required to be as educated. Though a case can be made that you need to validate the information.

As it happens we have an opportunity to tell how this works. Software engineering has seen many abstractions of which each comes with its own complexity in verification.

What tends to happen is that people don't really do a lot of verification, we are just "mostly right" very fast and leave an immense amount of inefficiency and indirection behind us.

Re: Tech’s hottest new job: Prompt engineer

#83

Post by vba616 »

thfuran wrote: post #67
vba616 wrote: post #64

Earlier quoted context omitted.

>You've got a huge blind spot if you think prompt engineer isn't already a thing. It may be a "thing", because generating BS is a viable business model and ChatGPT makes it more efficient. ..but I submit as a working hypothesis, that it is completely impossible to gain knowledge you do not already possess from a language model, no matter how clever your prompting. I'm very interested in counter-examples, but I have s…

That's so obviously false that I literally can't imagine how you could believe it. GPT-3 certainly isn't 100% accurate, but neither is it so perfectly unreliable that no one could ever get it to produce a relevant fact not in the prompt. And even if it were, it would probably still be potentially useful for learning languages.

> GPT-3 certainly isn't 100% accurate, but neither is it so perfectly unreliable that no one could ever get it to produce a relevant fact not in the prompt.

I think I understand the sense in which you claim it produces relevant facts not in the prompt.

It's not that we differ on easily observable behavior of the system.

It's that I question if GPT-3 is "producing" these identifiable facts, and if the user is "producing" them instead, whether they can possibly be "relevant".

Re: Tech’s hottest new job: Prompt engineer

#84

Post by vba616 »

ThrowawayTestr wrote: post #70
vba616 wrote: post #64

Earlier quoted context omitted.

>You've got a huge blind spot if you think prompt engineer isn't already a thing. It may be a "thing", because generating BS is a viable business model and ChatGPT makes it more efficient. ..but I submit as a working hypothesis, that it is completely impossible to gain knowledge you do not already possess from a language model, no matter how clever your prompting. I'm very interested in counter-examples, but I have s…

ChatGPT taught me about vectors and other ways to dynamically create an array.

How do you know?

Re: Tech’s hottest new job: Prompt engineer

#85

Post by Syntonicles »

barking_biscuit wrote: post #35
SillyUsername wrote: post #7

We don't need a new job position, what we really need is a LLM to generate good prompts...

It has been amazing to watch the feedback loop with tools like Stable Diffusion where you see two types of strategy for solving problems of missing functionality. The first strategy is coming up with some long winded manual process involving disparate tools and several steps. The other is simply train a model to do what it is you want automatically. It has fast taught me that there is no competing with this new parad…

Could you give me some concrete examples that come to mind? I'm new to Stable Diffusion but have been using it a lot lately.

Re: Tech’s hottest new job: Prompt engineer

#86

Post by vba616 »

eurasiantiger wrote: post #79
vba616 wrote: post #64

Earlier quoted context omitted.

>You've got a huge blind spot if you think prompt engineer isn't already a thing. It may be a "thing", because generating BS is a viable business model and ChatGPT makes it more efficient. ..but I submit as a working hypothesis, that it is completely impossible to gain knowledge you do not already possess from a language model, no matter how clever your prompting. I'm very interested in counter-examples, but I have s…

Synthetic knowledge is still novel knowledge if you haven’t put the pieces together before.

Can you unpack "synthetic knowledge", cause I don't know what that really means.

Re: Tech’s hottest new job: Prompt engineer

#87

Post by p1esk »

dijit wrote: post #82
deeviant wrote: post #49

Earlier quoted context omitted.

So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?

>So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM? No. A LLM has more opportunity itself to replace a Lawyer, the person typing the prompt is not necessarily required to be as educated. Though a case can be made that you need to validate the information. As it happens we…

the person typing the prompt is not necessarily required to be as educated.

If I need someone to help me interact with a legal LLM I will want to (and probably be able to, for 300k) hire someone with a law degree. In fact I anticipate many lawyers in the future will effectively become “prompt engineers” for legal LLMs.

Re: Tech’s hottest new job: Prompt engineer

#88

Post by deeviant »

vba616 wrote: post #76
deeviant wrote: post #49

Earlier quoted context omitted.

So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?

A LLM is a generator of misinformation that is maximally difficult to distinguish from real information. How do you use this as a lawyer? I mean, as a stereotypical evil lawyer in a world of naïve people who don't learn from experience, you could maybe use it to win cases until you destroy the justice system. But other than that...

Speaking as somebody who spent thousands of dollars for a lawyer on a matter, with basically no results, and also who used chatGPT, personal research, and common sense to solve this same matter for free, I can only say if a LLM is "a generator of misinformation that is maximally difficult to distinguish from real information" a lawyer is simply "a human who has been trained to maximally drain your wallet without regard for any other matter", of course, neither is true and there exists far more nuance for both.

Sure, there are matters I would only trust a lawyer to handle, but there are a great many I wouldn't.

Further, the average quality of a human lawyer will likely remain the same tomorrow as it is today, while AI will only get better. LLM today, perhaps some hybrid stack tomorrow, it's only a matter of time before an AI lawyer is the way to go for just about any legal matter. And let me be clear, that time might be 10 years, or may be 100+, but it is coming.

Re: Tech’s hottest new job: Prompt engineer

#89

Post by p1esk »

vba616 wrote: post #76
deeviant wrote: post #49

Earlier quoted context omitted.

So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?

A LLM is a generator of misinformation that is maximally difficult to distinguish from real information. How do you use this as a lawyer? I mean, as a stereotypical evil lawyer in a world of naïve people who don't learn from experience, you could maybe use it to win cases until you destroy the justice system. But other than that...

A LLM is a generator of misinformation

This is a strange statement. No one is training LLMs to generate “misinformation”. It’s the opposite - it’s trained to generate the most likely next word, given the preceding 2000 words - using billions of examples from a real world training corpus. So it will try to generate as much information as what’s present in the corpus. Maybe even more, but that’s debatable.

Re: Tech’s hottest new job: Prompt engineer

#90

Post by p1esk »

super256 wrote: post #81
TrackerFF wrote: post #50

Earlier quoted context omitted.

Not too different from analysts writing SQL queries all day long.

Complex SQL requires knowledge about relational algebra (Cartesian products, set theory, domain relational calculus yada yada) and understanding of how RDBMS and their query planners work. At least, if performance is important to you. I don't see this in this prompt engineering. In my limited experience (I played a few hours with Stable Diffusion and more hours with the OAI davinci-003 model), you can get good at it…

I spent a few hours learning SQL. I can get “good” at writing SQL queries within a few days. Do you want to hire me as someone who’s primary role is to write SQL queries?
Post reply on HN