Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Brex’s Prompt Engineering Guide

github.com

Post reply on HN
Print view View original

81–90 of 112 posts

Re: Brex’s Prompt Engineering Guide

#81

Post by jollybot »

rpastuszak wrote: post #73
jollybot wrote: post #70

Earlier quoted context omitted.

Engineering, the action of working artfully to bring something about.

What's the difference between engineering and design?

Design is part of engineering, but engineers will go on to build for example. Many folks like to gatekeep the use of engineer as a word. Mostly it just comes down to the fact that a displine hasn't matured yet and isnt' taught in a rigourous and formal manner. It is still engineering. See Network Engineering for example. Plenty of people building complex systems at scales never before seen. Even more that just about know how traceroute works but still building networks. They are all engineers of a young field of engineering.

Re: Brex’s Prompt Engineering Guide

#82

Post by jaredsohn »

One thing I haven't heard much discussion about is the fact that ChatGPT is constantly being updated.

This means that if you build a prompt for classification and become confident that you've whacked all of the moles so that it is pretty solid with all of the edge cases, it can later start breaking again.

Some solutions I can think of are 1) choose a fixed model to test against but they become deprecated over time or 2) perhaps fine-tuning might help.

Re: Brex’s Prompt Engineering Guide

#83

Post by dakom »

sgk284 wrote: post #80
dakom wrote: post #74

Earlier quoted context omitted.

That's extremely broad. I know how to push the buttons on my washing machine. So now I'm a laundering engineer? Maybe the answer is "yes", but I think it's a bit silly, and detracts from the engineering work that goes into building the machine, i.e. the work of understanding the connections and flow of electricity from button press to pumps and gears turning.

Tweaked my comment a little bit to clarify. Surely we'd agree that not everyone who uses software is a software engineer, but writing software is generally agreed upon as "software engineering". If you consider my comment as a whole, and the linked document, then the application of the definition makes sense. Large Language Models are incredibly similar to black-box non-deterministic interpreters. And prompt engineer…

I've never heard of non-deterministic programming languages. Is that really a thing?

(I mean - cue the JS jokes and all, but, really?)

Fwiw, to me, writing prompts feels most similar to management, directing, maybe psychotherapy (though I never worked in that, just guessing). Knowing how to adapt and use language to guide the "other" to achieve a goal, but without having insight into the exact way they go about it.

I feel like when I talk to ChatGPT I am directing it. "guide to directing AI bots" has its share of problems too, and "directing prompts" is just flat out unsexy.

Re: Brex’s Prompt Engineering Guide

#84

Post by tiborsaas »

jaredsohn wrote: post #82

One thing I haven't heard much discussion about is the fact that ChatGPT is constantly being updated. This means that if you build a prompt for classification and become confident that you've whacked all of the moles so that it is pretty solid with all of the edge cases, it can later start breaking again. Some solutions I can think of are 1) choose a fixed model to test against but they become deprecated over time or…

That's a valid concern, I think just like with any other software you need to write tests for the AI model to constantly check if your prompts are working as intended. Basic unit tests would work well in this case.

Re: Brex’s Prompt Engineering Guide

#85

Post by Veen »

game_the0ry wrote: post #2

I wonder if linguistic and English majors would end up benefiting in this trend of "prompt engineering."

I've occasionally thought that prompt engineering is better described as a type of rhetoric that aims to persuade LLMs to do what you want.

Re: Brex’s Prompt Engineering Guide

#86

Post by jaredsohn »

tiborsaas wrote: post #84
jaredsohn wrote: post #82

One thing I haven't heard much discussion about is the fact that ChatGPT is constantly being updated. This means that if you build a prompt for classification and become confident that you've whacked all of the moles so that it is pretty solid with all of the edge cases, it can later start breaking again. Some solutions I can think of are 1) choose a fixed model to test against but they become deprecated over time or…

That's a valid concern, I think just like with any other software you need to write tests for the AI model to constantly check if your prompts are working as intended. Basic unit tests would work well in this case.

The differences here compared to unit tests are that the breaking is outside of your control and the updating process is tedious. Also, testing requires making real API calls rather than using stubs so it requires additional infrastructure.

Re: Brex’s Prompt Engineering Guide

#87

Post by tiborsaas »

jaredsohn wrote: post #86
tiborsaas wrote: post #84

Earlier quoted context omitted.

That's a valid concern, I think just like with any other software you need to write tests for the AI model to constantly check if your prompts are working as intended. Basic unit tests would work well in this case.

The differences here compared to unit tests are that the breaking is outside of your control and the updating process is tedious. Also, testing requires making real API calls rather than using stubs so it requires additional infrastructure.

That's all true, but sometimes things break because some package is bumped. I'd still like to know if my app is basically broken if an LLM has changed somehow.

You are right that fine tuning would probably help to minimize the risks, but it probably never can be zero. New tests will also be needed when customers find new edge cases that break our assumptions.

Testing LLM prompts is a new paradigm that we'll have to learn to deal with.

Re: Brex’s Prompt Engineering Guide

#88

Post by TrueDuality »

xyzzy123 wrote: post #13
jeron wrote: post #4

Earlier quoted context omitted.

I always thought this problem was better thought of as software engineer/farmer problem - it's easer to teach a software engineer about agriculture than the other way around

Now I'm really wondering if that's true or not. Software engineering can be self-studied very cheaply, lots of free resources, mostly you just need time and motivation. Failure is usually cheap. Farming on the other hand requires more local, implicit and hands-on knowledge, capital requirements are high, feedback cycles are slower and failure is expensive.

I would argue the same is true of farming as well, you're just comparing differences in final outcomes. You can start learning basic farming techniques with a few pots, some seeds, water and a sunny spot in the same way you can get a raspberry pi and a keyboard sourcing information for both from YouTube fairly effectively.

There might be an argument for cost at the highest scale in each field which few in either profession really make it to but I'd bet even then its pretty on par.

I've been a huge fan of comparing the complexity or cost of a profession. We as a species specialize because doing these tasks in the most efficient way requires managing large number of details that are not obvious at first glance.

Re: Brex’s Prompt Engineering Guide

#89

Post by pwillia7 »

sgk284 wrote: post #80
dakom wrote: post #74

Earlier quoted context omitted.

That's extremely broad. I know how to push the buttons on my washing machine. So now I'm a laundering engineer? Maybe the answer is "yes", but I think it's a bit silly, and detracts from the engineering work that goes into building the machine, i.e. the work of understanding the connections and flow of electricity from button press to pumps and gears turning.

Tweaked my comment a little bit to clarify. Surely we'd agree that not everyone who uses software is a software engineer, but writing software is generally agreed upon as "software engineering". If you consider my comment as a whole, and the linked document, then the application of the definition makes sense. Large Language Models are incredibly similar to black-box non-deterministic interpreters. And prompt engineer…

I think here, like in a lot of places in our societies, there's a gap in language. Take the group of someone's grandma, john carmack, a 3rd year uni student and like a guy that writes YAML to control his home automation system.

2 words surely aren't enough. There's no way to draw one line through that group and get any real meaning (unless maybe it's above the grandma)

So, like in other parts of society, if we don't make new words, people will start using the words we already have.

I want someone to write a sci-fi book where our need for new language snowballs with increased technology and ends up being the great filter for life since it grows until no groups can understand each other.

Re: Brex’s Prompt Engineering Guide

#90

Post by sharemywin »

jatins wrote: post #53
voz_ wrote: post #3

Earlier quoted context omitted.

Perhaps, but this space suffers from the Armageddon astronaut/fire figher problem. It is easier to teach a Computer Science Major good english, than it is to teach an English Major computer science.

Don't think prompting LLMs require particularly "good" English in the first place. You can say a half baked sentence with typos and it'll still make sense of it. Plus when you go meta and ask LLMs to generate prompts for themselves, you own language proficiency becomes even less important. I do think English/language proficiency will help with Generative image AIs like Midjourney. Like if someone could describe a sce…

f*cking Meta. it took me a second to parse your sentence. at first I thought you meant using Llama or something. instead of prompts generating prompts.
Post reply on HN