Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ask HN: Where to learn the cutting edge of prompt engineering?

news.ycombinator.com

Post reply on HN
Print view View original

11–20 of 20 posts

Re: Ask HN: Where to learn the cutting edge of prompt engineering?

#11

Post by muzani »

Do customer support for OpenAI, lol. https://community.openai.com/

Answer enough questions, stay active enough, and you'll see the same patterns emerge. You'll probably make a lot of mistakes. You'll be corrected by other regulars and people you try to help will send you angry messages saying your prompt didn't work when utilised in the industry. It's a good way to learn. As a little bonus, if you do it constantly enough, OpenAI will give you this little "Regular" rank with a secret forum and such.

Langchain feels a little outdated IMO. I feel like OpenAI's in built tools might be a little ahead of it. It was originally designed to handle memory on the old completion API, but since OpenAI's chat API was released, it's not as useful. There's still good reason to use their completion models though - it performs higher quality responses for some creative uses. Agents built on them don't seem very impressive and OpenAI has their own "assistants" for agent-like stuff: https://platform.openai.com/docs/assistants/how-it-works

Re: Ask HN: Where to learn the cutting edge of prompt engineering?

#12

Post by muzani »

janalsncm wrote: post #8
danielmarkbruce wrote: post #7

Earlier quoted context omitted.

two cents: any situation involving billions/trillions of variables looks pretty pseudo-sciency because you can't reduce it down or isolate components very well. People can do studies, add things and take things out, and sort of hint at things and explain things sort of. It is what it is.

Real science is reproducible and provides testable, falsifiable hypotheses. In practice, some models (ChatGPT in particular) are not deterministic. This makes reproducing things harder. Not impossible, but harder.

I'd expect science in this field to look more like economics than physics. You're probably not looking at lab results or anything, but at experiments already done "on the field" with controlled variables. Like an economist assessing minimum wage laws might compare employment in two neighbouring states over a period of time.

Re: Ask HN: Where to learn the cutting edge of prompt engineering?

#14

Post by catlover76 »

muzani wrote: post #11

Do customer support for OpenAI, lol. https://community.openai.com/ Answer enough questions, stay active enough, and you'll see the same patterns emerge. You'll probably make a lot of mistakes. You'll be corrected by other regulars and people you try to help will send you angry messages saying your prompt didn't work when utilised in the industry. It's a good way to learn. As a little bonus, if you do it constantly en…

> Langchain feels a little outdated IMO.

That's being too generous lol

Re: Ask HN: Where to learn the cutting edge of prompt engineering?

#15

Post by _andrei_ » 

  - https://arxiv.org/
  - https://www.microsoft.com/en-us/research/group/dynamics-insights-apps-artificial-intelligence-machine-learning/articles/prompt-engineering-improving-our-ability-to-communicate-with-an-llm/
  - https://cloud.google.com/blog/products/ai-machine-learning/how-to-use-grounding-for-your-llms-with-text-embeddings
  - https://amatriain.net/blog/hallucinations

  and general resources:
  - https://learnprompting.org
  - https://www.promptingguide.ai
  - https://github.com/dair-ai/Prompt-Engineering-Guide

Re: Ask HN: Where to learn the cutting edge of prompt engineering?

#17

Post by danielmarkbruce »

muzani wrote: post #12
janalsncm wrote: post #8

Earlier quoted context omitted.

Real science is reproducible and provides testable, falsifiable hypotheses. In practice, some models (ChatGPT in particular) are not deterministic. This makes reproducing things harder. Not impossible, but harder.

I'd expect science in this field to look more like economics than physics. You're probably not looking at lab results or anything, but at experiments already done "on the field" with controlled variables. Like an economist assessing minimum wage laws might compare employment in two neighbouring states over a period of time.

Certainly not physics - but shouldn't it be more like biology than economics? You can do experiments in biology (and with LLMs), the problem is you can't quite get to a cause-effect. The situation is just so complex. In both cases we are desperate to get an "explanation" that fits in a human brain and makes us feel like we understand, but it's out of reach.

Re: Ask HN: Where to learn the cutting edge of prompt engineering?

#19

Post by muzani »

danielmarkbruce wrote: post #17
muzani wrote: post #12

Earlier quoted context omitted.

I'd expect science in this field to look more like economics than physics. You're probably not looking at lab results or anything, but at experiments already done "on the field" with controlled variables. Like an economist assessing minimum wage laws might compare employment in two neighbouring states over a period of time.

Certainly not physics - but shouldn't it be more like biology than economics? You can do experiments in biology (and with LLMs), the problem is you can't quite get to a cause-effect. The situation is just so complex. In both cases we are desperate to get an "explanation" that fits in a human brain and makes us feel like we understand, but it's out of reach.

Yeah, biology does seem like a better comparison, especially with "strands" of AI performing differently than what they used to after patches.

Re: Ask HN: Where to learn the cutting edge of prompt engineering?

#20

Post by arthurcolle »

catlover76 wrote: post #9
arthurcolle wrote: post #3

Earlier quoted context omitted.

Prompt Engineering is clearly "a thing" irrespective of whether or not one trains or build models. LLMs clearly have a wide range of possible outputs given a particular prompt (even with just tuning temperature, top_p, top_k) but then, modification of a prompt can lead to significant improvements in the output. it's not a science. It's not really an art either. Certain prompts lead to better outputs than other prompt…

What kind of eval set do you use?

homegrown and full of love, like a carefully pruned garden of bonsai trees
Post reply on HN