Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guidance language for controlling LLMs

github.com

Post reply on HN
Print view View original

51–60 of 198 posts

Re: A guidance language for controlling LLMs

#51

Post by intelVISA »

ubj wrote: post #38

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.

Hear me out, just incubated a hot new lang that's about to capture the market and VC hearts:

SELECT * FROM llm

Re: A guidance language for controlling LLMs

#53

Post by m3kw9 »

There should be a standard template/language to structurally prompt LLMs. Once that is good, all good LLMs should use the doc to fine tune it to take in that standard. Right now each model has their own little way to best prompt it and you end up needing programs like this to sit in between and handle it for you

Re: A guidance language for controlling LLMs

#54

Post by crooked-v »

slundberg wrote: post #23
simonw wrote: post #21

This is pretty fascinating, but I'm not sure I understand the benefit of using a Handlebars-like DSL here. For example, given this code from https://github.com/microsoft/guidance/blob/main/notebooks/ch... create_plan = guidance('''{{#system~}} You are a helpful assistant. {{~/system}} {{#block hidden=True}} {{#user~}} I want to {{goal}}. {{~! generate potential options ~}} Can you please generate one option for how t…

You can serialize and ship the DSL to a remote server for high speed execution. (without trusting raw Python code)

Why not just use JSON instead, though? Then you can just rely on all the preexisting JSON tooling out there for most stuff to do with it.

Re: A guidance language for controlling LLMs

#55

Post by bjackman »

Wow I think there are details here I'm not fully understanding but this feels like a bit of a quantum leap* in terms of leveraging the strengths while avoiding the weaknesses of LLMs.

It seems like anything that provides access to the fuzzy "intelligence" in these systems while minimizing the cost to predictability and efficiency is really valuable.

I can't quite put it into words but it seems like we are gonna be moving into a more hybrid model for lots of computing tasks in the next 3 years or so and I wonder if this is a huge peek at the kind of paradigms we'll be seeing?

I feel so ignorant in such an exciting way at the moment! That tidbit about the problem solved by "token healing" is fascinating.

*I'm sure this isn't as novel to people in the AI space but I haven't seen anything like it before myself.

Re: A guidance language for controlling LLMs

#56

Post by Der_Einzige »

killthebuddha wrote: post #48
rain1 wrote: post #35

Earlier quoted context omitted.

This looks incredible. Wow.

I agree, it looks great. A couple similar projects you might find interesting: - https://github.com/newhouseb/clownfish - https://github.com/r2d4/rellm The first one is JSON only and the second one uses regular expressions, but they both take the same "logit masking" approach as the project GP linked to.

I love the love from you two - I am trying right now to significantly improve CTGS. I'm not actually using the "Logitsprocessor" from Huggingface, and I really ought to as it will massively speed up inference performance. Unfortunately, fixing up my current code to work with that will take quite awhile. I've started working on it but I am extremely busy these days and would really love for other smart people to help me on this project.

If not here, I really want proper access to the constraints APIs (LogitsProcessor and the Constraints classes in Huggingface) in the big webUIs for LLMs like oogabooga. I'd love to make that an extension.

I'm also upset at the "undertooling" in the world of LLM prompting. I wrote a snarky blog post about this: https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

Re: A guidance language for controlling LLMs

#57

Post by homarp »

leroy-is-here wrote: post #42
ryanklee wrote: post #39

Earlier quoted context omitted.

Not quite sure what the spirit of your comment is. But, yes, they are real technologies. Very confused as to why you would even find that dubious.

Not dubious, I just read your comment and it felt like I was reading satire. Even the cadence of your words felt funny. Anyway, I’m not surprised. It’s a new market, everyone’s in on it.

LangChain: https://news.ycombinator.com/item?id=34422627

LQML: https://news.ycombinator.com/item?id=35956484

Haystack: https://news.ycombinator.com/item?id=29501045 or more recently https://news.ycombinator.com/item?id=35430188

Re: A guidance language for controlling LLMs

#58

Post by lcnPylGDnU4H9OF »

ubj wrote: post #38

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.

It won't necessarily turn into some that is fundamentally the same as a current programming language. Rather than a "VM" or "interpreter" or "compiler" we have this "LLM".

Even if it requires a lot of domain knowledge to program using an "LLM-interpreted" language, the means of specification (in terms of how the software code is interpreted) may be different enough that it enables easier-to-write, more robust, (more Good Thing) etc. programs.

Re: A guidance language for controlling LLMs

#59

Post by Der_Einzige »

bjackman wrote: post #55

Wow I think there are details here I'm not fully understanding but this feels like a bit of a quantum leap* in terms of leveraging the strengths while avoiding the weaknesses of LLMs. It seems like anything that provides access to the fuzzy "intelligence" in these systems while minimizing the cost to predictability and efficiency is really valuable. I can't quite put it into words but it seems like we are gonna be mo…

A lot of this is because there was and still is systemic undertooling in NLP around how to prompt and leverage the wonderful LLMs that they built.

We have to let the Stable Diffusion community guide us, as the waifu generating crowd seems to be quite good at learning how to prompt models. I wrote a snarky github gist about this - https://gist.github.com/Hellisotherpeople/45c619ee22aac6865c...

Post reply on HN