What we know about LLMs • Hacker News Board

Re: What we know about LLMs

#171

Post by bradfox2 » Thu, Jul 27, 2023, 1:43 PM UTC

bilsbie wrote: post #106

bradfox2 wrote: post #94

Earlier quoted context omitted.

It's still loss being backproped, but the loss is calculated over a different criteria

Ok that makes a lot of sense. Why do they call it reinforcement learning then? Is it not traditional RE such as Q learning?

LLM output is scored by another model that produces a reward for the entire sequence emitted by the LLM. The reward model is trained on human preferences or some other metric usually. It's RL because we train on the reward and not some language modeling objective.

The LLM is trained to increase this reward score (or minimize the inverse), which is what makes it RL.

Re: What we know about LLMs

#172

Post by lacrimacida » Fri, Jul 28, 2023, 9:54 PM UTC

samwillis wrote: post #20

go_elmo wrote: post #11

Earlier quoted context omitted.

In the end - an AI should have these competing subsystems in one system - just as our brains are one system. What I find extremely interesting is how perception and thinking differs from person to person too - it was a "taboo" topic to call this neurodiversity - just as other genetic traits, but AI makes this relevant more than ever imo. Sure, its complicated and much comes from nurture (Nurture vs nature.. as exposu…

You touch in an important topic here, how our understanding of AI/ML/LLMs will influence our "understanding" of the human brain and intelligence. My fear is that we will ascribe too much human behaviour to that we see in and understand of our AI inventions, and that this will result in the dehumanisation of people. So essentially my fear is what we justify doing to each other due to AI, rather than what "AGI" could d…

I started seeing this dehumanisation spring up even here on HN, comparing LLMs with human brains and human thoughts as similar regurgitation to LLMs. I’m afraid it will get worse as this technology advances

Re: What we know about LLMs

#173

Post by obiefernandez » Sun, Aug 06, 2023, 4:00 PM UTC

llm_nerd wrote: post #100

obiefernandez wrote: post #96

Earlier quoted context omitted.

I can assure you that the smart folks running daily AI newsletters are currently raking in money hand over fist. It’s awe inspiring.

Such as? Can you give some examples? There are some very smart people working incredibly hard, yielding some rewards from it. It is a tiny, tiny set.

Recently paid 500 usd to TheresAnAIforThat for one link in their newsletter. They feature at least 5-10 per day. Decent money for a one man operation that’s “generated by AI”