Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

What we know about LLMs

willthompson.name

Post reply on HN
Print view View original

121–130 of 173 posts

Re: What we know about LLMs

#121

Post by DanHulton »

api wrote: post #34
DanHulton wrote: post #27

Earlier quoted context omitted.

I feel like it took practically no time for custom ASICs for bitcoin mining to show up, as soon as it was determined there was real money involved. Given that there's already definitely real money involved here, I wonder what's holding up the custom AI ASICs?

It takes a while to design an ASIC and it hasn't been that long since the hype wave really arrived for these things. I would bet on LLM chips showing up in 2024-2025. There's also a run on foundries right now which might delay things further. The new foundries in the US being built under the Chips act in Arizona and Ohio won't be online until probably 2025.

Ah that's an excellent point about the foundries, I hadn't considered that. Also, in reflection, I'm being pretty handwavy about the "as soon as it was determined there was real money involved."

Honestly, I'd probably be pretty unsurprised to learn if the time difference between "big commercial interest" and "custom chips appear" is probably pretty similar for both cases.

Re: What we know about LLMs

#122

Post by jasmer »

bastawhiz wrote: post #110
killernap wrote: post #13

ChatGPT was announced November, 2022 - 8 months ago. Time flies. Question for HN: Where are we in the hype cycle on this? We can run shitty clones slowly on Raspberry Pi's and your phone. The educational implementations demonstrate the basics in under a thousand lines of brisk C. Great. At some point you have to wonder... well, so what? Not one killer app has emerged. I for one am eager to be all hip and open minded…

> Not one killer app has emerged. I'll say that I pretty firmly disagree with this. I've been using Github Copilot for about six months for my own work and it has fundamentally changed how I write code. Ignoring the ethics of Copilot, if I just need to read a file with some data, parse it, and render that data on screen, Copilot just _does_ most of that for me. I write a chunky comment explaining what I want, it writ…

[dead]

Re: What we know about LLMs

#123

Post by jasmer »

DebtDeflation wrote: post #108
killernap wrote: post #13

ChatGPT was announced November, 2022 - 8 months ago. Time flies. Question for HN: Where are we in the hype cycle on this? We can run shitty clones slowly on Raspberry Pi's and your phone. The educational implementations demonstrate the basics in under a thousand lines of brisk C. Great. At some point you have to wonder... well, so what? Not one killer app has emerged. I for one am eager to be all hip and open minded…

The killer app for large enterprises is Q&A against the corporate knowledgebase(s). Big companies have an insane amount of tribal knowledge locked away in documents sitting on Sharepoint, on Box, on file servers, etc. Best case scenario, their employees can do keyword search against a subset of those documents. Chunk those docs, run them through an embedding process, store the embeddings in a vector store, let employ…

[dead]

Re: What we know about LLMs

#124

Post by EEMac »

Arkhaine_kupo wrote: post #37
killernap wrote: post #13

ChatGPT was announced November, 2022 - 8 months ago. Time flies. Question for HN: Where are we in the hype cycle on this? We can run shitty clones slowly on Raspberry Pi's and your phone. The educational implementations demonstrate the basics in under a thousand lines of brisk C. Great. At some point you have to wonder... well, so what? Not one killer app has emerged. I for one am eager to be all hip and open minded…

> Not one killer app has emerged. I think the microsoft gpt integration on Office is probably that app. Ability to ask to have your email's summarised, or getting your excel sheets formulas configured with natural language, etc are increidbly useful tools to lower the floor of entry to tools that already speed up humans so much. I don't think the use of this tools is some life redefining feature, but a friend of mine…

There was a science fiction story about this, with phone auto-message and auto-answer systems connecting with each other long after all the humans were dead.

Re: What we know about LLMs

#125

Post by apsurd »

bastawhiz wrote: post #110
killernap wrote: post #13

ChatGPT was announced November, 2022 - 8 months ago. Time flies. Question for HN: Where are we in the hype cycle on this? We can run shitty clones slowly on Raspberry Pi's and your phone. The educational implementations demonstrate the basics in under a thousand lines of brisk C. Great. At some point you have to wonder... well, so what? Not one killer app has emerged. I for one am eager to be all hip and open minded…

> Not one killer app has emerged. I'll say that I pretty firmly disagree with this. I've been using Github Copilot for about six months for my own work and it has fundamentally changed how I write code. Ignoring the ethics of Copilot, if I just need to read a file with some data, parse it, and render that data on screen, Copilot just _does_ most of that for me. I write a chunky comment explaining what I want, it writ…

My friend made a great comparison that seems to agree with your take: chatGPT for coding is like when ruby on rails came out. Or wordpress. felt magical and boosted (a certain kind of) productivity through the roof.

We don't think of rails as the second coming though.

same with code editors. of course a rails for all of code is cool. but iono, it's a code editor. i still use sublime.

Re: What we know about LLMs

#126

Post by samstave »

RC_ITR wrote: post #70
bilsbie wrote: post #59

Earlier quoted context omitted.

When you’re doing RLHF are you actually modifying the weights of llama itself? Or is something on top?

RLHF does change the parameters. The way to think about it is that backpropagation changes the parameters of a model so they get closer to some sort of desired output. In pre-training and SFT, the parameters are changed so the model does a better job of replicating the next word in the training data, given the words it has already seen. In RLHF, the parameters are changed so the model does a better job of outputting…

This implies that any RLHF is introducing human bias into any "thoughts" the model may have?

Re: What we know about LLMs

#127

Post by hallqv »

jawerty wrote: post #63
bilsbie wrote: post #59

Earlier quoted context omitted.

When you’re doing RLHF are you actually modifying the weights of llama itself? Or is something on top?

So I'm not doing RLHF that's how LLama is pre-trained. It's in the loss/optimization phase in their training I believe. For the finetuning i'm using LoRA to freeze most of the layers for parameter optimization. Using PEFT from huggingface

RLHF is not part of LLaMa pretraining, or pretraning of any other models for that matter. RLHF comes after pretraining. https://twitter.com/Jeande_d/status/1661833563069620247/phot...

Re: What we know about LLMs

#128

Post by esafak »

jannyfer wrote: post #7

Great summary. I’ve been reading a pop neuroscience book called Incognito (2011). In it, the author talks about how the brain is a group of competing sub-brains of many forms, and the brain might have several ways of doing the same thing (e.g. recognizing an object). The author also posited that the lack of AI progress back then was due to the fact that there are no constantly competing sub-brains. Our brains are alw…

> The author also posited that the lack of AI progress back then was due to the fact that there are no constantly competing sub-brains.

That became popular in neural networks after the introduction of dropout regularization, which forced neurons to "co-adapt" and learn to do each others' jobs. Large, over-specified models also provide a natural setting for co-adaptation.

Re: What we know about LLMs

#129

Post by api »

DanHulton wrote: post #121
api wrote: post #34

Earlier quoted context omitted.

It takes a while to design an ASIC and it hasn't been that long since the hype wave really arrived for these things. I would bet on LLM chips showing up in 2024-2025. There's also a run on foundries right now which might delay things further. The new foundries in the US being built under the Chips act in Arizona and Ohio won't be online until probably 2025.

Ah that's an excellent point about the foundries, I hadn't considered that. Also, in reflection, I'm being pretty handwavy about the "as soon as it was determined there was real money involved." Honestly, I'd probably be pretty unsurprised to learn if the time difference between "big commercial interest" and "custom chips appear" is probably pretty similar for both cases.

I recall it taking years for BTC ASICS that were generally available.

Re: What we know about LLMs

#130

Post by gardenhedge »

killernap wrote: post #116
IKantRead wrote: post #54

Earlier quoted context omitted.

> Where are we in the hype cycle on this? Can we stop acting like the Gartner "hype cycle" is anything more than a marketing gimmick created Gartner to validate their own consulting/research services? While you can absolutely find cases that map to the "hype cycle", there is nothing whatsoever to validate this model as remotely accurate or valid for describing technology trends. Where is crypto in the "hype cycle"? I…

Gartner are so great at their job you think they own the concept of hype cycles and rage against them being mentioned while being the one to introduce them to the conversation in the first place :)

Just google 'hype cycles' and it's all about gartner so I think GP is fine with bringing them up.
Post reply on HN