Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Building LLM Applications for Production

huyenchip.com

Post reply on HN
Print view View original

21–30 of 99 posts

Re: Building LLM Applications for Production

#21

Post by visarga »

imranq wrote: post #12

I like a lot of the LLM use cases mentioned here. A couple more are: - conducting literature reviews (stay sane while researching LLMs!) - Talking to textbooks / AI teaching assistants - language learning with a companion tailored to your level and interested LLMs are so hyped and written about these days that it would be hilarious if the next version of GPT trained on todays internet would be biased towards praising…

Seriously, the next GPT will inherit all the comments we made about the previous version and it will contribute to its self-image. It will define and see itself through our words. Maybe not so different from us, we rely on other people's opinions to form a self-image too.

Re: Building LLM Applications for Production

#22

Post by visarga »

namaria wrote: post #19

The novelty is wearing off and the reality of parsing hundreds of TBs into hundreds of GB memory blobs you can query by the Kb is setting in.

What's interesting is that each token goes and visits all the model. Basically each token touches the synthesis of the whole human culture before being fully formed.

Re: Building LLM Applications for Production

#23

Post by visarga »

rvz wrote: post #4

So now almost all the low hanging fruit programming books have instantly become redundant and off-shored to ChatGPT, and will stay on the shelves to collect dust. New here comes the race to create prompt engineering books and courses in. 24 hours to sell to other AI bros who think that they are prompting it wrong, not prompting hard enough or the prompting the wrong way.

> New here comes the race to create prompt engineering books

Let's call it "Language [based] Programming", LP for short, as opposed to "prompt engineering" and "programming language". It's programming, in language. Not just prompting, it can be multi-step, involve multiple models and plugins, have branches and loops. And it's not just a new programming language, it's the Language itself.

Re: Building LLM Applications for Production

#25

Post by jimsimmons »

visarga wrote: post #22
namaria wrote: post #19

The novelty is wearing off and the reality of parsing hundreds of TBs into hundreds of GB memory blobs you can query by the Kb is setting in.

What's interesting is that each token goes and visits all the model. Basically each token touches the synthesis of the whole human culture before being fully formed.

To bake a cake first you have to invent the universe

Re: Building LLM Applications for Production

#27

Post by stareatgoats »

visarga wrote: post #21
imranq wrote: post #12

I like a lot of the LLM use cases mentioned here. A couple more are: - conducting literature reviews (stay sane while researching LLMs!) - Talking to textbooks / AI teaching assistants - language learning with a companion tailored to your level and interested LLMs are so hyped and written about these days that it would be hilarious if the next version of GPT trained on todays internet would be biased towards praising…

Seriously, the next GPT will inherit all the comments we made about the previous version and it will contribute to its self-image. It will define and see itself through our words. Maybe not so different from us, we rely on other people's opinions to form a self-image too.

> Maybe not so different from us

There is a phenomenon in history where people have identified with their artifacts: in the machine age humans were seen as nothing but advanced machines, in the computer age we became advanced computers. No doubt there is now a budding trend to see us as hardly anything more than advanced LLMs...

None of these perspectives were ever totally wrong however, only somewhat reductionist.

Re: Building LLM Applications for Production

#28

Post by vsnf »

For a lot of the usecases that involve summarizing some form of input data (for instance the article mentions book summaries, math walkthroughs etc), how can I trust the output to not be hallucinated? How can I reasonably judge that what it tells me is factual with respect to the input and not just made-up nonsense?

This is the problem I have with the GPT models. I don't think I can trust them for anything actually important.

Re: Building LLM Applications for Production

#29

Post by visarga »

vsnf wrote: post #28

For a lot of the usecases that involve summarizing some form of input data (for instance the article mentions book summaries, math walkthroughs etc), how can I trust the output to not be hallucinated? How can I reasonably judge that what it tells me is factual with respect to the input and not just made-up nonsense? This is the problem I have with the GPT models. I don't think I can trust them for anything actually i…

In open-book mode it does not hallucinate. That only happens in closed-book mode. So if you put a piece of text in the prompt you can trust the summary will be factual. You can also use it for information extraction - text to JSON.

Re: Building LLM Applications for Production

#30

Post by chaoz_ »

vsnf wrote: post #28

For a lot of the usecases that involve summarizing some form of input data (for instance the article mentions book summaries, math walkthroughs etc), how can I trust the output to not be hallucinated? How can I reasonably judge that what it tells me is factual with respect to the input and not just made-up nonsense? This is the problem I have with the GPT models. I don't think I can trust them for anything actually i…

True, "amount of hallucination" (very confident, but factually wrong) is probably something they can decrease in the next versions tho.

I also would not trust it with anything important, but there can be good applications for something that works 9/10 times.

Post reply on HN