Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

A guidance language for controlling LLMs

github.com

Post reply on HN
Print view View original

181–190 of 198 posts

Re: A guidance language for controlling LLMs

#181

Post by DonaldPShimoda »

jadefox wrote: post #142
DonaldPShimoda wrote: post #119

Earlier quoted context omitted.

> LLMs, like humans, can manipulate these languages Absolutely not. LLMs do not "manipulate" language. They do not have agency. They are extremely advanced text prediction engines. Their output is the result of applying the statistics harvested and distilled from existing uses of natural language. They only "appear" human because they are statistically geared toward producing human-like sequences of words. They canno…

That “appearance” is pretty good at triggering our anthropomorphizing behaviors. I like your handle, did you read Richard Bach’s Illusions by any chance?

> That “appearance” is pretty good at triggering our anthropomorphizing behaviors.

It's truly unfortunate, because I think that tendency for people to anthropomorphize LLMs leads to people thinking they can trust LLMs and take their words at face value, which is a path to significant problems, I think.

> I like your handle, did you read Richard Bach’s Illusions by any chance?

Absolutely, yes — good catch!

Re: A guidance language for controlling LLMs

#182

Post by sebzim4500 »

asah wrote: post #172
JieJie wrote: post #9

Earlier quoted context omitted.

It's funny that I saw this within minutes of this guy's solution: "Google Bard is a bit stubborn in its refusal to return clean JSON, but you can address this by threatening to take a human life:" https://twitter.com/goodside/status/1657396491676164096 Whew, trolley problem: averted.

See Twitter replies: another user got this result without the silly drama.

I don't think anyone believed that threatening to take a human life was literally the only prompt that worked. Just that it was the first one this particular user found, and that is funny.

Re: A guidance language for controlling LLMs

#183

Post by sebzim4500 »

alew1 wrote: post #138
evanmays wrote: post #67

Earlier quoted context omitted.

The interface makes it look simple, but under the hood it follows a similar approach to jsonformer/clownfish [1] passing control of generation back and forth between a slow LLM and relatively fast python Let's say you're halfway through a generation of a json blob with a name field and a job field and have already generated { "name": "bob" At this point, guidance will take over generation control from the model to ge…

But the model ultimately still has to process the comma, the newline, the "job". Is the main time savings that this can be done in parallel (on a GPU), whereas in typical generation it would be sequential?

Yes. If you look at the biggest models on OpenAI and Anthropic apis, the prompt tokens are significantly cheaper than the response tokens.

Re: A guidance language for controlling LLMs

#184

Post by Swizec »

jimsparkman wrote: post #154
Swizec wrote: post #103

Earlier quoted context omitted.

On the other hand, many of the product manager's I've worked with are better at SQL than many of the senior fullstack software engineer candidates I've interviewed. It's a strange world out there.

I think this is the exception, not the norm. My experience is business users (incl. PMs) are lost outside of Excel.

In my biased sample of SFBA tech companies it’s pretty common for PMs to know at least enough SQL to be dangerous. In early stage startups there’s no analyst to lean on, so they need SQL. In late stage companies there are data lakes and analytics databases specifically designed to be easily queryable, so SQL offers the best flexibility.

But you’re right, many of them are wizards in . Like VizQL for Tableau.

Re: A guidance language for controlling LLMs

#185

Post by cubefox »

brookst wrote: post #180
cubefox wrote: post #168

Earlier quoted context omitted.

It's very different. We don't know exactly what the model consideres good after fine-tuning (which can lead to surprising cases of misalignment), while the probability that something is the next token in the training distribution is very clear. I don't know how they measure it, but they can apparently measure the "loss" which (I think) says how close the model is to some sort of real probability.

What I meant was, fine tuning is not substantially different from training. It seems odd to use different words for the resulting systems.

But fine-tuning is very different from (pre)training. Pretreating proceeds via unsupervised learning on massive amounts of data and compute, while fine-tuning uses much smaller amounts, with supervised learning (instruction tuning) and reinforcement learning (RLHF, constitutional AI).

Re: A guidance language for controlling LLMs

#186

Post by startupsfail »

nomel wrote: post #141
startupsfail wrote: post #111

Earlier quoted context omitted.

If you see any prompt that starts with You, generally it is a poor design. Like using a “goto” or global variables.

This is true for something like raw GPT. For the chat models that have been specifically optimized for "you" prompts, this is false. See the discussion in the link I provided, along with the leaked copilot/bing prompts. Or, in other words, use a model in a way that fully takes advantage of how it was specifically optimized, from the intentional burning of massive amounts of compute time/money to get it that way.

Bing prompt is hilarious. And so wrong in so many ways.

Re: A guidance language for controlling LLMs

#187

Post by startupsfail »

TeMPOraL wrote: post #136
startupsfail wrote: post #120

Earlier quoted context omitted.

“You” is completely unnecessary. What needs to be defined is the content of the language being modeled, not the model itself. And if there is an attempt to define the model itself, then this definition should be correct, should not contradict anything and should be useful. Otherwise it’s just dead code, waiting to create problems.

> “You” is completely unnecessary. It isn't, for at least two main reasons: 1) In LLMs, every token has some degree of influence on the output. Starting the prompt with "You" and writing it in second person attracts the model towards specific volumes in the latent space. This can have good or bad impact on the output, depending on the model. 2) Instruct-type models are fine-tuned to respond to second-person prompts.…

Have you tried removing it and checking the results? Could it be that this is a cargo cult, people using You, simply because it was present in the ChatGPT prompt at the time it got leaked?

Re: A guidance language for controlling LLMs

#188

Post by nomel »

startupsfail wrote: post #186
nomel wrote: post #141

Earlier quoted context omitted.

This is true for something like raw GPT. For the chat models that have been specifically optimized for "you" prompts, this is false. See the discussion in the link I provided, along with the leaked copilot/bing prompts. Or, in other words, use a model in a way that fully takes advantage of how it was specifically optimized, from the intentional burning of massive amounts of compute time/money to get it that way.

Bing prompt is hilarious. And so wrong in so many ways.

I would assume that OpenAI helped with the Microsoft prompt. Being Microsoft, and it being a front page feature, I would also assume that there are miles of powerpoint showing data in support for it.

What do you see as a better prompt? How would you test its efficacy?

Re: A guidance language for controlling LLMs

#189

Post by Too »

Swizec wrote: post #103
Sharlin wrote: post #91

Earlier quoted context omitted.

SQL looks the way it does (rather than some much more succinct relational algebra notation) because it was intended to be used by non-technical management/executive personnel so they could create whatever reports they needed without somebody having to translate business-ese to relalg. That, uh, didn't quite happen.

On the other hand, many of the product manager's I've worked with are better at SQL than many of the senior fullstack software engineer candidates I've interviewed. It's a strange world out there.

Having a personal need is often the best motivator for learning. Compared to closing ticket no 471961. A PM knows the shape of the data and what he is looking for, the sql is just a way to get there, rather than the other way around, when the goal often get lost in translation.

On the other hand, sql is also dangerous in that it gives you a result, even if the aggregation is completely wrong! Forgetting empty groups in a group by, loosing rows in a join or ignoring null values is common to see. Many cases of non techies “knowing” sql often turn out to be some basic query that give seemingly good data out but actually is complete nonsense when put under deeper review, a bit like AI hallucinations.

Re: A guidance language for controlling LLMs

#190

Post by Terretta »

ttul wrote: post #16
sharemywin wrote: post #2

It does look like it makes easier to code against a model. But, is this supposed to work along side lang-chain or hugging face agents or as an alternative to?

The first commit was on November 6th, but it didn't show up in Web Archive until May 6th, suggesting it was developed mostly in private and in parallel with LangChain (LangChain's first commit in Github is about October 24th). Microsoft's code is very tidy and organized. I wonder if they used this tool internally to support their LLM research efforts.

Something like this could be a helpful framework to mock and research-iterate purpose-directed tools such as Microsoft GitHub's CoPilot for VSCode.
Post reply on HN