I'm currently tinkering on a customer support bot with langchain and gpt3. The bot can answer questions about services and their terms, it can use tools to make bookings and perform some taks like scheduling appointments, in a conversational manner.
It's becoming clear to me that subtle changes in the prompt can lead to bullshit answers and gpt making up facts, despite being specifically told not to do so.
If the prompt reaches some complexity threshold, the output quality goes down visibly. I learned that I have to split the bot into subtasks, each having different, smaller, prompts.
So, yeah, I believe prompt engineering can be a thing. At least for a while, until the models become smarter at understanding what we want from them :)
Tech’s hottest new job: Prompt engineer
41–50 of 123 posts
Re: Tech’s hottest new job: Prompt engineer
#42I just can't believe how idiotic such a thing is. It's like calling a person who uses a toaster a Toast Engineer with a salary of 300k.
The naming might be flawed but it's like a batista I guess. You can spend a lot of money on the machine but without an expert to use it you will not get the best out of it.
Just train an LLM on LLM prompts and automate the prompt engineer process.
Re: Tech’s hottest new job: Prompt engineer
#43I just can't believe how idiotic such a thing is. It's like calling a person who uses a toaster a Toast Engineer with a salary of 300k.
No more idiotic than a lawyer - someone who knows the right words to convince a jury (model) to produce a desired outcome.
Re: Tech’s hottest new job: Prompt engineer
#44I just can't believe how idiotic such a thing is. It's like calling a person who uses a toaster a Toast Engineer with a salary of 300k.
The naming might be flawed but it's like a batista I guess. You can spend a lot of money on the machine but without an expert to use it you will not get the best out of it.
Yes but you can teach most of what there is to know in a couple of weeks tops. You don't call baristas Coffee Engineers.
Re: Tech’s hottest new job: Prompt engineer
#45I just can't believe how idiotic such a thing is. It's like calling a person who uses a toaster a Toast Engineer with a salary of 300k.
No more idiotic than a lawyer - someone who knows the right words to convince a jury (model) to produce a desired outcome.
I dislike that analogy greatly.
Lawyers have a bad reputation, sure, but there’s a lot of education about the interpretation of our law and the absurdly large corpus of legal documentation that must be read in order to even become a lawyer is far and above anything you describe.
Re: Tech’s hottest new job: Prompt engineer
#46I just can't believe how idiotic such a thing is. It's like calling a person who uses a toaster a Toast Engineer with a salary of 300k.
I still remeber when Google appeared there were people offering search services for a fee (maybe they were called Search Engineers but I don't recal seeing that).
Re: Tech’s hottest new job: Prompt engineer
#47Earlier quoted context omitted.
No more idiotic than a lawyer - someone who knows the right words to convince a jury (model) to produce a desired outcome.
I dislike that analogy greatly. Lawyers have a bad reputation, sure, but there’s a lot of education about the interpretation of our law and the absurdly large corpus of legal documentation that must be read in order to even become a lawyer is far and above anything you describe.
To be an effective prompt engineer you need to have expertise in two different domains - large generative ML models, and in whatever it is people want to generate (e.g. art).
Re: Tech’s hottest new job: Prompt engineer
#48you can automatically learn soft prompts with backprop so.. the job "prompt engineering" isn't going to stick around for too long given it's automatable. https://arxiv.org/abs/2302.06541 That is not to say, that integrating LLMs won't create a lot of jobs. Think of it as systems engineering. Knowing how computers work, as well as a software engineer does, will always be useful.
You still have to come up with the original prompt, though.
Cool paper, BTW.
Re: Tech’s hottest new job: Prompt engineer
#49Earlier quoted context omitted.
No more idiotic than a lawyer - someone who knows the right words to convince a jury (model) to produce a desired outcome.
I dislike that analogy greatly. Lawyers have a bad reputation, sure, but there’s a lot of education about the interpretation of our law and the absurdly large corpus of legal documentation that must be read in order to even become a lawyer is far and above anything you describe.
So you are presenting the fact that lawyers must digest an absurdly large corpus of legal documentation, but also maintaining that is something a human lawyer has some sort of advantage over a LLM?
Re: Tech’s hottest new job: Prompt engineer
#50I just can't believe how idiotic such a thing is. It's like calling a person who uses a toaster a Toast Engineer with a salary of 300k.
Not too different from analysts writing SQL queries all day long.