Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What we know about LLMs

willthompson.name

Post reply on HN
Print view View original

51–60 of 173 posts

Re: What we know about LLMs

#51

Post by jawerty »

I run through a lot of these concepts, specifically RLHF, in my latest coding stream where I finetune LLama 2 if anyone's interested in getting a LLM deep dive https://www.youtube.com/watch?v=TYgtG2Th6fI&t=4002s

Long story short, the size of the model and reward mechanisms used in validating off of human annotating/feedback are the main differences between what we can do as independents in OSS vs OpenAI. BigCode's StarCoder (https://huggingface.co/bigcode/starcoder) has some human labor backing it (I believe correct me if I'm wrong) but at the end of the day a company will always be able to gather people better.

Not knocking Starcoder, in fact I streamed how to fine tune it the other day. However, it's important to mention some of the limitations in the OSS space now (big reason Meta pushing LLama 2 is a nice to have)

Re: What we know about LLMs

#52

Post by haswell »

killernap wrote: post #38
throwuwu wrote: post #22

Earlier quoted context omitted.

lol, now who’s demented? Everyone I know uses it. It even diagnosed a problem with my pool filter among dozens of other uses I find for it. I like it and use it more than Google and stack overflow now. Losing the school crowd for the summer isn’t the beginning of the end, it just means there’s a cohort that doesn’t need it as much for a few months while they’re out having fun instead of stuck inside writing papers an…

It is great that everyone you know uses it but the traffic to ChatGPT is decreasing and has been for over two months now. If pointing this fact out makes me demented consider that perhaps you are emotionally invested in this new toy/brand. I guess we can wait and see what kind of usage trends will emerge long term. My anecdotal evidence (which is not worth much, same as yours) is that many normies tried it a few time…

> the traffic to ChatGPT is decreasing and has been for over two months now

This seems entirely unsurprising, and isn’t by itself enough to support your general thesis.

Interacting with these LLMs was extremely novel for most people when the tech first dropped, and those earlier months were the peak of the viral growth/expansion into public awareness.

As the novelty dies down, it’s not surprising that there would be less traffic. Early on, I had all sorts of ridiculous conversations just to see what would happen. Now, I only use it when I have some task in mind.

That transition points to this being the opposite of a toy - after the fun dies down, the real work begins.

> My anecdotal evidence…is that many normies tried it a few times and it was a topic of conversation but is no longer mentioned much.

This has not been my experience at all. Most non-technical folks I know who are interested in ChatGPT see value in its ability to expand their technical capabilities/knowledge.

People who are motivated to learn will continue to use this to their advantage.

If some subset of that population has no such interest, this has no bearing on the usefulness of the tech, nor is it representative of the population.

And even if the “normie” population (this is pretty reductive…) abandons it entirely, this again says nothing about the value/utility of LLMs, and hints at a product/market fit issue.

We don’t say programming languages are useless because they’re not adopted by the general public.

Re: What we know about LLMs

#53

Post by carlossouza »

killernap wrote: post #13

ChatGPT was announced November, 2022 - 8 months ago. Time flies. Question for HN: Where are we in the hype cycle on this? We can run shitty clones slowly on Raspberry Pi's and your phone. The educational implementations demonstrate the basics in under a thousand lines of brisk C. Great. At some point you have to wonder... well, so what? Not one killer app has emerged. I for one am eager to be all hip and open minded…

Isn’t Copilot a killer app?!

Re: What we know about LLMs

#54

Post by IKantRead »

killernap wrote: post #13

ChatGPT was announced November, 2022 - 8 months ago. Time flies. Question for HN: Where are we in the hype cycle on this? We can run shitty clones slowly on Raspberry Pi's and your phone. The educational implementations demonstrate the basics in under a thousand lines of brisk C. Great. At some point you have to wonder... well, so what? Not one killer app has emerged. I for one am eager to be all hip and open minded…

> Where are we in the hype cycle on this?

Can we stop acting like the Gartner "hype cycle" is anything more than a marketing gimmick created Gartner to validate their own consulting/research services?

While you can absolutely find cases that map to the "hype cycle", there is nothing whatsoever to validate this model as remotely accurate or valid for describing technology trends.

Where is crypto in the "hype cycle"? It went through at least 3 rounds of "peak of inflated expectation" and I'm not confident it will ever reach a meaningful "plateau of productivity".

Did mobile ever have "inflated expectation"? Yes there was a lot of hype in the early days but those people hyped about it, rushing to build mobile versions of their websites... were correct.

The "hype cycle" is a neat idea but doesn't really map to reality in a way that makes it useful. It's only useful for Gartner to create an illusion of credibility and sell their services.

Re: What we know about LLMs

#55

Post by TigeriusKirk »

killernap wrote: post #38
throwuwu wrote: post #22

Earlier quoted context omitted.

lol, now who’s demented? Everyone I know uses it. It even diagnosed a problem with my pool filter among dozens of other uses I find for it. I like it and use it more than Google and stack overflow now. Losing the school crowd for the summer isn’t the beginning of the end, it just means there’s a cohort that doesn’t need it as much for a few months while they’re out having fun instead of stuck inside writing papers an…

It is great that everyone you know uses it but the traffic to ChatGPT is decreasing and has been for over two months now. If pointing this fact out makes me demented consider that perhaps you are emotionally invested in this new toy/brand. I guess we can wait and see what kind of usage trends will emerge long term. My anecdotal evidence (which is not worth much, same as yours) is that many normies tried it a few time…

You asked for examples of impactful uses, you have been provided with some.

Arguing with people who provided what you asked for is a common but unproductive habit.

Re: What we know about LLMs

#56

Post by zirgs »

blitzar wrote: post #26
dsco wrote: post #4

Earlier quoted context omitted.

As a matter of fact, there’s even more developers making a hard left into AI who have never touched crypto. The interesting follow up question is: what will they actually spend time on? Training new models? Copy pasting front ends on ChatGPT? Fine tuning models? I think many of them will be scared by how much of a hard science ML is vs just spinning up old CRUD apps

> Training new models? Copy pasting front ends on ChatGPT? Fine tuning models? The stable diffusion community is probably 2 years more mature than the GPT, there we see gui tools of a kind (in colab notebooks) to abstract away from code and thenlots of fine tuning. On the professional side, adobe have plugged these tools into their products. https://www.adobe.com/sensei/generative-ai/firefly.html

It's a lot easier to run Stable Diffusion locally. Meanwhile only the dumbest LLMs work on ordinary consumer GPUs. Datacenter GPUs with 80 GB vram are ridiculously expensive.

Re: What we know about LLMs

#57

Post by Jevon23 »

killernap wrote: post #13

ChatGPT was announced November, 2022 - 8 months ago. Time flies. Question for HN: Where are we in the hype cycle on this? We can run shitty clones slowly on Raspberry Pi's and your phone. The educational implementations demonstrate the basics in under a thousand lines of brisk C. Great. At some point you have to wonder... well, so what? Not one killer app has emerged. I for one am eager to be all hip and open minded…

>Not one killer app has emerged.

Surely the “killer app” is ChatGPT itself?

ChatGPT has already put some copywriters and journalists out of work, or at least reduced their hours. The app is quite literally “killing” something, i.e. people’s jobs. For those people, it’s not just empty hype. It’s very real. Certainly it’s already more real than anything having to do with blockchain/crypto.

Re: What we know about LLMs

#58

Post by tmpz22 »

mjr00 wrote: post #50
killernap wrote: post #13

ChatGPT was announced November, 2022 - 8 months ago. Time flies. Question for HN: Where are we in the hype cycle on this? We can run shitty clones slowly on Raspberry Pi's and your phone. The educational implementations demonstrate the basics in under a thousand lines of brisk C. Great. At some point you have to wonder... well, so what? Not one killer app has emerged. I for one am eager to be all hip and open minded…

> Not one killer app has emerged. I for one am eager to be all hip and open minded and pretend like I use LLMs all the time for everything and they are "the future" but novelty aside it seems like so far we have a demented clippy and some sophomoric arguments about alignment and wrong think. In my mind I divide LLM usage into two categories, creation and ingestion. Creation is largely a parlor trick that blew the min…

Isn’t the summarization of text like legal documents where the notion of hallucinations come in as a huge blocker?

Is the industry making progress on fixing such hallucinations? Or for that matter the privacy implications of sharing such documents with entities like OpenAI that don’t respect IP?

Until hallucinations and IP/PII are fixed I don’t want this technology anywhere near my legal or personal documents.

Re: What we know about LLMs

#59

Post by bilsbie »

jawerty wrote: post #51

I run through a lot of these concepts, specifically RLHF, in my latest coding stream where I finetune LLama 2 if anyone's interested in getting a LLM deep dive https://www.youtube.com/watch?v=TYgtG2Th6fI&t=4002s Long story short, the size of the model and reward mechanisms used in validating off of human annotating/feedback are the main differences between what we can do as independents in OSS vs OpenAI. BigCode's St…

When you’re doing RLHF are you actually modifying the weights of llama itself?

Or is something on top?

Re: What we know about LLMs

#60

Post by JustBreath »

FishInTheWater wrote: post #10

Given a set of instructions, an instruction fine-tuned/aligned LLM is able (conditional on size and training quality) to reason through a set of steps to produce a desired output. This is plainly wrong. The model's growing size makes it better at guessing the outcome of a reasoning task, but little to no actual reasoning is performed. It's trivial to prove this as well, as LLMs will still fail miserably at (larger) m…

There's some argument to be made that a form of reasoning happens in a roundabout way when the AI is told to explain it's reasoning.

For example if you tell it "Do " and then open a new context and say "Do , explain your reasoning beforehand." you will often get a more accurate response.

Granted, it's not that any "Hmm, let me think about that." Deep Thought reasoning occurs, but simply that predicting what the reasoning would look like and then predicting what comes after that reasoning results in a more accurate - and ironically, reasoned - response.

Kinda funny actually, it's a bit like how in Hitchiker's Guide they just had to tell the probability machine to calculate the odds of an improbability drive in order to create it.

Post reply on HN