Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guidance language for controlling LLMs

github.com

Post reply on HN
Print view View original

121–130 of 198 posts

Re: A guidance language for controlling LLMs

#121

Post by pxtail »

quenix wrote: post #116
startupsfail wrote: post #114

Earlier quoted context omitted.

It is not a step towards greater rigor. They literally have magical thinking and “biblical” quotes from GPT 11:4 all other the place, mixing code and religion. And starting prompts with “You”? Seriously. Can we at least drop that as a start?

> And starting prompts with “You”? Seriously. Can we at least drop that as a start? What is wrong with this?

I'm not interested in pleasant, formal "conversation" with the thing roleplaying as human and wasting, time, keystrokes and money, I want data as fast and condensed as possible without dumb fluff. Yes, it's funny for few first times but not much after that

Re: A guidance language for controlling LLMs

#122

Post by june_twenty »

evanmays wrote: post #67
m3kw9 wrote: post #50

I’m not understanding how Guidence Accelerating works. It says “ This cuts this prompt's runtime in half vs. a standard generation approach.” and it gives an example of it asking LLM to generate json. I don’t see anywhere how it accelerates anything because it’s a simple json completion call. How can you accelerate that?

The interface makes it look simple, but under the hood it follows a similar approach to jsonformer/clownfish [1] passing control of generation back and forth between a slow LLM and relatively fast python Let's say you're halfway through a generation of a json blob with a name field and a job field and have already generated { "name": "bob" At this point, guidance will take over generation control from the model to ge…

Thanks for that example. Very helpful

Re: A guidance language for controlling LLMs

#123

Post by conradev »

DonaldPShimoda wrote: post #119
conradev wrote: post #107

Earlier quoted context omitted.

I don’t think formal languages are going anywhere because we need the guarantees that they can provide. From Dijkstra: https://www.cs.utexas.edu/users/EWD/transcriptions/EWD06xx/E... You need to be able to define all of the possible edge cases so there isn’t any Undefined Behavior: that’s the formal part Humans can use LLMs to manipulate these languages to achieve specific goals. I can imagine designing formal langua…

> LLMs, like humans, can manipulate these languages Absolutely not. LLMs do not "manipulate" language. They do not have agency. They are extremely advanced text prediction engines. Their output is the result of applying the statistics harvested and distilled from existing uses of natural language. They only "appear" human because they are statistically geared toward producing human-like sequences of words. They canno…

I made some edits, does that satisfy your constraints? Humans are the agent, LLMs the tool

Re: A guidance language for controlling LLMs

#124

Post by yinser »

WastingMyTime89 wrote: post #64
leroy-is-here wrote: post #42

Earlier quoted context omitted.

Not dubious, I just read your comment and it felt like I was reading satire. Even the cadence of your words felt funny. Anyway, I’m not surprised. It’s a new market, everyone’s in on it.

It is satire. They just don’t realise it yet. It’s pretty clear that we are in the phase where everyone is rushing to get a slice of the pie selling dubious thing and people start parroting word soup hoping they actually make sense and fearing they will miss out. That’s indeed what people often and rightfully satirise about the IT industry. That’s the joke phase before things settle.

Why don’t you try shaking your fist at your computer instead of wasting everyone’s time in this thread. Touch grass

Re: A guidance language for controlling LLMs

#126

Post by yinser »

ryanklee wrote: post #68
jameshart wrote: post #43

Earlier quoted context omitted.

Consider how similar your comment reads, for an outsider, to this explanation of AWS InfiniDash: https://twitter.com/TartanLlama/status/1410959645238308866

I'm not considering outsiders. Why should I. It's a reasonable assumption that readers of HN are accustomed to ridiculous sounding tech product names. Further, this is a comment on a thread regarding a particularly new technology in a particularly newly thriving domain. The expectation should therefore be that there will be references to tech even more esoteric than normal. The commenter should have instead thought:…

There are more of us that read and shake our heads with you than these cretins who want to tear down - on a tech news site of all places. Don’t be dismayed, and there are more of us with you than against you.

Re: A guidance language for controlling LLMs

#127

Post by ryanklee »

jameshart wrote: post #117
ryanklee wrote: post #68

Earlier quoted context omitted.

I'm not considering outsiders. Why should I. It's a reasonable assumption that readers of HN are accustomed to ridiculous sounding tech product names. Further, this is a comment on a thread regarding a particularly new technology in a particularly newly thriving domain. The expectation should therefore be that there will be references to tech even more esoteric than normal. The commenter should have instead thought:…

I was responding to your confusion as to why someone might think you were writing a parody. You ran into the tech equivalent of poe’s law. You said something that makes perfect sense in your technical sphere, but it read as indistinguishable from parody to an audience unfamiliar with the technologies in question.

Apologies for misunderstanding the intention of your comment. That does make sense.

Re: A guidance language for controlling LLMs

#128

Post by jameshart »

joe_the_user wrote: post #106
ubj wrote: post #38

I like this step towards greater rigor when working with LLM's. But part of me can't help but feel like this is essentially reinventing the concept of programming languages: formal and precise syntax to perform specific tasks with guarantees. I wonder where the final balance will end up between the ease and flexibility of everyday language, and the precision / guarantees of a formally specified language.

But is it a step to greater rigor? Or is it an illusion of rigor? They talk about improving tokenization but I don't believe that's the fundamental problem of controlling LLMs. The problem with LLMs is all the data comes in as (tokenized) language and the result is nothing but in-context predicted output. That's where all the "prompt-injection" exploits come from - as well as the hallucinations, "temper tantrums" and…

The result is actually richer than ‘predicted output’ - it’s a probability distribution over all possible output.

Having richer ways to consume that probability distribution than just ‘take the most likely thing, after adding some noise’ is more conducive to using LLMs to generate output that can be further processed - in rigorous ways. Like by running it through a compiler.

Think about how when you’re coding, autocomplete suggestions help you pick the right ‘next token’ with greater accuracy.

Re: A guidance language for controlling LLMs

#129

Post by kurisufag »

startupsfail wrote: post #120
quenix wrote: post #116

Earlier quoted context omitted.

> And starting prompts with “You”? Seriously. Can we at least drop that as a start? What is wrong with this?

“You” is completely unnecessary. What needs to be defined is the content of the language being modeled, not the model itself. And if there is an attempt to define the model itself, then this definition should be correct, should not contradict anything and should be useful. Otherwise it’s just dead code, waiting to create problems.

> Otherwise it’s just dead code, waiting to create problems

it's very possible that the pretense improves results: most recorded interactions /are/ between two people, after all.

Re: A guidance language for controlling LLMs

#130

Post by hitchstory »

tomduncalf wrote: post #76
felideon wrote: post #62

Earlier quoted context omitted.

A number of years ago we were designing a way to specify insurance claim adjudication rules in natural language, so that "the business" could write their own rules. The "natural" language we ended up with was not so natural after all. We would have had to teach users this specific English dialect and grammar (formal and precise syntax, as you said). So, in the end, we abandoned that project and years later just rewro…

> but it wasn't something the stakeholders outside of engineering even wanted Ha this reminds me of the craze for BDD/Cucumber type testing. Don’t think I ever once saw a product owner take interest in a human readable test case haha

Ive had product owners take an interest in docs autogenerated from tests. Especially with artrfacts embedded. They like tuff like this:

https://github.com/hitchdev/hitchstory/blob/master/examples/...

And can be persuaded to look at the (YAML) source.

Gherkin isnt really a suitable language for writing test cases in - it's verbose, lacks inheritance, has clunky syntax and is stringly typed.

Post reply on HN