The upcoming GPT-3 moment for RL • Hacker News Board

Re: The upcoming GPT-3 moment for RL

#101

Post by xoralkindi » Mon, Jul 14, 2025, 5:15 PM UTC

jasim wrote: post #92

xnorswap wrote: post #91

Earlier quoted context omitted.

Audit is at the heart of accounting, and LLMs are the antithesis of an audit trail.

I'm sorry I don't follow. The fact that you use an LLM to classify a transaction does not mean there is no audit trail for the fact. There should also be a manual verifier who's ultimately responsible for the entries, so that we do not abdicate responsibility to black boxes.

Now instead of having accountants audit transactions you will have accountants audit LLM output for possible hallucinations. Seems counter productive.

Re: The upcoming GPT-3 moment for RL

#102

Post by cjblomqvist » Mon, Jul 14, 2025, 8:41 PM UTC

kevindamm wrote: post #58

cjblomqvist wrote: post #46

Earlier quoted context omitted.

> Please please no vibe accounting. Funny you mention; There are multiple companies in Sweden working on AI/ML based accounting. It's not so different from AI/ML based automated driving.

I've seen some of those but all of the ones I've looked at also had a panel of experts who could give it a once-over (or re-work) before sending it back to the client. I'd compare it more to cruise control or driver-assist but not quite automated driving.

Well, at the core of accounting there is an accountant reviewing your accounts (annually at least) - regardless how your books are done (with exceptions for small/tiny companies). So it doesn't seem so far off to do the base work done by an AI.

I guess it's like automated driving a few years back - monitored by humans able to take over control. Step by step it'll become better and better until good enough for handle certain chosen areas.

Re: The upcoming GPT-3 moment for RL

#103

Post by [deleted] » Mon, Jul 14, 2025, 9:50 PM UTC

Shorel wrote: post #87

kuruczgy wrote: post #16

Earlier quoted context omitted.

> but language models frequently outperform us in reasoning what 99% of the time their reasoning is laughable. Or even if their reasoning is on the right track, they often just ignore it in the final answer, and do the stupid thing anyway.

Yes, if a LLM outperforms you, you have never reasoned in your life. I will assume you passed high-school based on your looks and not on your abilities.

[deleted]

Re: The upcoming GPT-3 moment for RL

#104

Post by sva_ » Tue, Jul 15, 2025, 7:03 PM UTC

physix wrote: post #93

sva_ wrote: post #72

Earlier quoted context omitted.

> From my understanding, RL is a tuning approach on LLMs, What you're referring to is actually just one application of RL (RLHF). RL itself is much more than that

Actually I didn't. Correct me if I am wrong, but my understanding is that RL is still an LLM tuning approach, i.e. an optimization of its parameter set, no matter if it's done at scale or via HF.

RL is a lot more general than that, it is basically a way in which an agent learns to make optimal decisions by learning from experience to maximize rewards. So you can do all kinds of stuff other than finetuning LLMs with it, like controlling a robotic arm, playing/mastering videogames, etc. For example, AlphaGo was also RL.