Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guidance language for controlling LLMs

github.com

Post reply on HN
Print view View original

21–30 of 198 posts

Re: A guidance language for controlling LLMs

#21

Post by simonw »

This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here.

For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch...

    create_plan = guidance('''{{#system~}}
    You are a helpful assistant.
    {{~/system}}
    {{#block hidden=True}}
    {{#user~}}
    I want to {{goal}}.
    {{~! generate potential options ~}}
    Can you please generate one option for how to accomplish this?
    Please make the option very short, at most one line.
    {{~/user}}
    {{#assistant~}}
    {{gen 'options' n=5 temperature=1.0 max_tokens=500}}
    {{~/assistant}}
    {{/block}}
    {{~! generate pros and cons and select the best option ~}}
    {{#block hidden=True}}
    {{#user~}}
    I want to {{goal}}.
    ''')
How about something like this instead?

    create_plan = guidance([
        system("You are a helpful assistant."),
        hidden([
            user("I want to {{goal}}."),
            comment("generate potential options"),
            user([
                "Can you please generate one option for how to accomplish this?",
                "Please make the option very short, at most one line."
            ]),
            assistant(gen('options', n=5, temperature=1.0, max_tokens=500)),
        ]),
        comment("generate pros and cons and select the best option"),
        hidden(
            user("I want to {{goal}}"),
        )
    ])

Re: A guidance language for controlling LLMs

#22

Post by itake »

simonw wrote: post #21

This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here. For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch... create_plan = guidance('''{{#system~}} You are a helpful assistant. {{~/system}} {{#block hidden=True}} {{#user~}} I want to {{goal}}. {{~! generate potential options ~}} Can you please generate one option for how t…

My guess is you can store the DLS as a file (or in a db). With your example, you have to execute the code stored in your db.

Re: A guidance language for controlling LLMs

#23

Post by slundberg »

simonw wrote: post #21

This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here. For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch... create_plan = guidance('''{{#system~}} You are a helpful assistant. {{~/system}} {{#block hidden=True}} {{#user~}} I want to {{goal}}. {{~! generate potential options ~}} Can you please generate one option for how t…

You can serialize and ship the DSL to a remote server for high speed execution. (without trusting raw Python code)

Re: A guidance language for controlling LLMs

#24

Post by marcotcr »

simonw wrote: post #21

This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here. For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch... create_plan = guidance('''{{#system~}} You are a helpful assistant. {{~/system}} {{#block hidden=True}} {{#user~}} I want to {{goal}}. {{~! generate potential options ~}} Can you please generate one option for how t…

I think the DSL is nice when you want to take part of the generation and use it later in the prompt, e.g. this (in the same notebook).

---

prompt = guidance('''{{#system~}}

You are a helpful assistant.

{{~/system}}

{{#user~}}

From now on, whenever your response depends on any factual information, please search the web by using the function query before responding. I will then paste web results in, and you can respond.

{{~/user}}

{{#assistant~}}

Ok, I will do that. Let's do a practice round

{{~/assistant}}

{{>practice_round}}

{{#user~}}

That was great, now let's do another one.

{{~/user}}

{{#assistant~}}

Ok, I'm ready.

{{~/assistant}}

{{#user~}}

{{user_query}}

{{~/user}}

{{#assistant~}}

{{gen "query" stop=""}}{{#if (is_search query)}}{{/if}}

{{~/assistant}}

{{#if (is_search query)}}

{{#user~}}

Search results: {{#each (search query)}}

{{this.title}}

{{this.snippet}}

{{/each}}

{{~/user}}

{{#assistant~}}

{{gen "answer"}}

{{~/assistant}}

{{/if}}''')

---

You could still write it without a DSL, but I think it would be harder to read.

Re: A guidance language for controlling LLMs

#25

Post by leroy-is-here »

ryanklee wrote: post #11

I'm personally starting with learning Guidance and LMQL rather than LangChain just in order to get a better grasp of the behaviors that I've gathered LangChain papers over. Even after that, I'm likely to look at Haystack before LangChain. Just getting the feeling that LangChain is going to end up being considered a kitchen sink solution full of anti patterns so might as well spend time a little lower level while I se…

If this comment performative comedy? Are these real technologies ?

Re: A guidance language for controlling LLMs

#27

Post by foota »

slundberg wrote: post #23
simonw wrote: post #21

This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here. For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch... create_plan = guidance('''{{#system~}} You are a helpful assistant. {{~/system}} {{#block hidden=True}} {{#user~}} I want to {{goal}}. {{~! generate potential options ~}} Can you please generate one option for how t…

You can serialize and ship the DSL to a remote server for high speed execution. (without trusting raw Python code)

There's prior art for pythonic DSLs that aren't actual python code.

Re: A guidance language for controlling LLMs

#28

Post by Der_Einzige »

There has been a huge explosion of awesome tooling which utilizes constrained text generation.

Awhile ago, I tried my own hand at constraining the output of LLMs. I'm actively working on this to make it better, especially with the lessons learned from repos like this and from guidance

https://github.com/hellisotherpeople/constrained-text-genera...

Re: A guidance language for controlling LLMs

#29

Post by awestroke »

JieJie wrote: post #9
ntonozzi wrote: post #3

How does this work? I've seen a cool project about forcing Llama to output valid JSON: https://twitter.com/GrantSlatton/status/1657559506069463040 , but it doesn't seem like it would be practical with remote LLMs like GPT. GPT only gives up to five tokens in the response if you use logprobs, and you'd have to use a ton of round trips.

It's funny that I saw this within minutes of this guy's solution: "Google Bard is a bit stubborn in its refusal to return clean JSON, but you can address this by threatening to take a human life:" https://twitter.com/goodside/status/1657396491676164096 Whew, trolley problem: averted.

I don't know why, but I find this hilarious. Imagine if this style of llm prompting becomes commonplace
Post reply on HN