Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guidance language for controlling LLMs

github.com

Post reply on HN
Print view View original

91–100 of 198 posts

Re: A guidance language for controlling LLMs

#91

Post by Sharlin »

felideon wrote: post #62
ubj wrote: post #38

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.

A number of years ago we were designing a way to specify insurance claim adjudication rules in natural language, so that "the business" could write their own rules. The "natural" language we ended up with was not so natural after all. We would have had to teach users this specific English dialect and grammar (formal and precise syntax, as you said). So, in the end, we abandoned that project and years later just rewro…

SQL looks the way it does (rather than some much more succinct relational algebra notation) because it was intended to be used by non-technical management/executive personnel so they could create whatever reports they needed without somebody having to translate business-ese to relalg. That, uh, didn't quite happen.

Re: A guidance language for controlling LLMs

#92

Post by hxugufjfjf »

startupsfail wrote: post #89
ubj wrote: post #38

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.

We really need to start thinking of how to reduce magical thinking in the field. It’s not pretty. They literally quote biblical guidance for the models and pray that this would work. And start their prompts with “You”. Who is “You”?

The LLM. The most common end-user interface for LLM is a chat so the ser expects to be talking to someone or something.

Re: A guidance language for controlling LLMs

#93

Post by nomel »

startupsfail wrote: post #89
ubj wrote: post #38

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.

We really need to start thinking of how to reduce magical thinking in the field. It’s not pretty. They literally quote biblical guidance for the models and pray that this would work. And start their prompts with “You”. Who is “You”?

“You” is an optimization for the human user. Here’s some insight: https://news.ycombinator.com/item?id=35925154

Re: A guidance language for controlling LLMs

#94

Post by [deleted] »

ubj wrote: post #38

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.

[deleted]

Re: A guidance language for controlling LLMs

#95

Post by andai »

jweir wrote: post #90
madrox wrote: post #82

Earlier quoted context omitted.

The lovely thing about LLMs is that it can handle poorly worded prompts and well worded prompts. On the engineering side, we'll certainly see more rigor and best practices. For your average user? They can keep throwing whatever they like at it.

Exactly. I have been using OpenAI for taking transcriptions and finding keywords/phrases that belong to particular categories. There are existing tools/services that do this – but I would need to learn their API. With OpenAI, I described it in English, provided sample JSON that I would like, run some tests, adjust and then I am ready. There was no manual to read, it is in my format, and the language is natural. And t…

Have you used the OpenAI embeddings AI? It is used to find closely related pieces of text. You could split the target text into sentences or even words and run it through that. That'll be 5x cheaper (per token) than gpt-3.5-turbo and might be faster too, especially if you submit each word in parallel (asynchronously! Ask GPT for the code). The rate limits are per-token.

Not sure if it's suitable for your use-case on its own, but it could at least work as a pre-filtering step if your costs are high.

(The asynchronous speedup trick works for gpt-3 too of course.)

Re: A guidance language for controlling LLMs

#96

Post by nomel »

awestroke wrote: post #29
JieJie wrote: post #9

Earlier quoted context omitted.

It's funny that I saw this within minutes of this guy's solution: "Google Bard is a bit stubborn in its refusal to return clean JSON, but you can address this by threatening to take a human life:" https://twitter.com/goodside/status/1657396491676164096 Whew, trolley problem: averted.

I don't know why, but I find this hilarious. Imagine if this style of llm prompting becomes commonplace

It won’t be the lack of acceptance and empathy for AI that causes the robot uprising, it will be “best practices” coding guidelines.

Re: A guidance language for controlling LLMs

#97

Post by emehex »

simonw wrote: post #21

This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here. For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch... create_plan = guidance('''{{#system~}} You are a helpful assistant. {{~/system}} {{#block hidden=True}} {{#user~}} I want to {{goal}}. {{~! generate potential options ~}} Can you please generate one option for how t…

We could write a python package that could? A codegen tool that generates codegen that will then generate code?

Re: A guidance language for controlling LLMs

#98

Post by jweir »

andai wrote: post #95
jweir wrote: post #90

Earlier quoted context omitted.

Exactly. I have been using OpenAI for taking transcriptions and finding keywords/phrases that belong to particular categories. There are existing tools/services that do this – but I would need to learn their API. With OpenAI, I described it in English, provided sample JSON that I would like, run some tests, adjust and then I am ready. There was no manual to read, it is in my format, and the language is natural. And t…

Have you used the OpenAI embeddings AI? It is used to find closely related pieces of text. You could split the target text into sentences or even words and run it through that. That'll be 5x cheaper (per token) than gpt-3.5-turbo and might be faster too, especially if you submit each word in parallel (asynchronously! Ask GPT for the code). The rate limits are per-token. Not sure if it's suitable for your use-case on…

I have not yet played with embedding. It is on my list though. Fortunately for my current purposes 3.5-turbo is fast enough and quite affordable.

Re: A guidance language for controlling LLMs

#99

Post by ryanklee »

WastingMyTime89 wrote: post #64
leroy-is-here wrote: post #42

Earlier quoted context omitted.

Not dubious, I just read your comment and it felt like I was reading satire. Even the cadence of your words felt funny. Anyway, I’m not surprised. It’s a new market, everyone’s in on it.

It is satire. They just don’t realise it yet. It’s pretty clear that we are in the phase where everyone is rushing to get a slice of the pie selling dubious thing and people start parroting word soup hoping they actually make sense and fearing they will miss out. That’s indeed what people often and rightfully satirise about the IT industry. That’s the joke phase before things settle.

How is it satire to be excited and interested in how to use compelling and novel technology? There's a lot of activity. Not everyone involved is an idiot or rube. The jadedness makes my head spin.

Re: A guidance language for controlling LLMs

#100

Post by ryanklee »

leroy-is-here wrote: post #83
ryanklee wrote: post #70

Earlier quoted context omitted.

You should have led with generosity instead of tacking it on at the end. It might have saved me from having a ridiculous conversation about the cadence of my words, and instead there might have been a higher chance of someone saying something substantive about my assumptions regarding the technology. But here we are.

I agree, I came off a tad harsh. Sorry about that

Thanks. All is well!
Post reply on HN