Live data from Hacker News

Learning to Reason with LLMs

openai.com

551–560 of 1001 posts

Re: Learning to Reason with LLMs

#551

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

Semi-retired software/hardware engineer here. After my recent experiences with various coding LLMs (similar to the experience of the OP with the bluetooth fan protocol) I'm really glad I'm in a financial position such that I'm able to retire. The progress of these LLMs at coding has been astonishing over the last 18 months. Will they entirely replace humans? No. But as they increase programmer productivity fewer devs will be required. In my case the contract gig I was doing over this last summer I was able to do about 3 to 4X faster than I could've done it without LLMs. Yeah, they were generating a lot of boiler plate HDL code for me, but that still saved me several days of work at least. And then there was the test code that they generated which again saved me days of work. And their ability to explain old undocumented code that was part of the project was also extremely helpful. I was skeptical 18 months ago that any of this would be possible. Not anymore. I wasn't doing a project in which there would've been a lot of training examples. We're talking Verilog testbench generation based on multiple input Verilog modules, C++ code generation for a C program analyzer using libclang - none of this stuff would've worked just a few months back.

Re: Learning to Reason with LLMs

#552
So, it’s good at hard-logic reasoning (which is great, and no small feat.)

Does this reasoning capability generalize outside of the knowledge domains the model was trained to reason about, into “softer” domains?

For example, is O1 better at comedy (because it can reason better about what’s funny)?

Is it better at poetry, because it can reason about rhyme and meter?

Is it better at storytelling as an extension of an existing input story, because it now will first analyze the story-so-far and deduce aspects of the characters, setting, and themes that the author seems to be going for (and will ask for more information about those things if it’s not sure)?

Re: Learning to Reason with LLMs

#553
post #465

Earlier quoted context omitted.

Did you edit the message? I cannot see anything now in the screenshot, too low resolution

You need to click on the image for the high res version to load. Sorry, it’s awkward.

The website seems to redirect me to a low resolution image, the first time I clicked on the link it worked as you are saying.

Re: Learning to Reason with LLMs

#554

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

The calculator didn’t eliminate math majors. Excel and accounting software didn’t eliminate accountants and CPAs. These are all just tools. I spend very little of my overall time at work actually coding. It’s a nice treat when I get a day where that’s all I do. From my limited work with Copilot so far, the user still needs to know what they’re doing. I have 0 faith a product owner, without a coding background, can us…

Agreed. The sweet spot is people who have product owner skills _and_ can code. They are quickly developing superpowers. The overhead of writing tickets, communicating with the team and so on is huge. If one person can do it all, efficiency skyrockets.

I guess it's always been true to some extent that single individuals are capable of amazing things. For example, the guy who's built https://www.photopea.com/. But they must be exceptional - this empowers more people to do things like that.

Re: Learning to Reason with LLMs

#555
My point of view: this is a real advancement. I’ve always believed that with the right data allowing the LLM to be trained to imitate reasoning, it’s possible to improve its performance. However, this is still pattern matching, and I suspect that this approach may not be very effective for creating true generalization. As a result, once o1 becomes generally available, we will likely notice the persistent hallucinations and faulty reasoning, especially when the problem is sufficiently new or complex, beyond the “reasoning programs” or “reasoning patterns” the model learned during the reinforcement learning phase. https://www.lycee.ai/blog/openai-o1-release-agi-reasoning

Re: Learning to Reason with LLMs

#556

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

Hey, kid.

My name is Rachel. I'm the founder of company whose existence is contingent on the continued existence, employment, and indeed competitive employment of software engineers, so I have as much skin in this game as you do.

I worry about this a lot. I don't know what the chances are that AI wipes out developer jobs [EDIT: to clarify, in the sense that they become either much rarer or much lower-paid, which is sufficient] within a timescale relevant to my work (say, 3-5 years), but they aren't zero. Gun to my head, I peg that chance at perhaps 20%. That makes me more bearish on AI than the typical person in the tech world - Manifold thinks AI surpasses human researchers by the end of 2028 at 48% [1], for example - but 20% is most certainly not zero.

That thought stresses me out. It's not just an existential threat to my business over which I have no control, it's a threat against which I cannot realistically hedge and which may disrupt or even destroy my life. It bothers me.

But I do my work anyway, for a couple of reasons.

One, progress on AI in posts like this is always going to be inflated. This is a marketing post. It's a post OpenAI wrote, and posted, to generate additional hype, business, and investment. There is some justified skepticism further down this thread, but even if you couldn't find a reason to be skeptical, you ought to be skeptical by default of such posts. I am an abnormally honest person by Silicon Valley founder standards, and even I cherry pick my marketing blogs (I just don't outright make stuff up for them).

Two, if AI surpasses a good software engineer, it probably surpasses just about everything else. This isn't a guarantee, but good software engineering is already one of the more challenging professions for humans, and there's no particular reason to think progress would stop exactly at making SWEs obsolete. So there's no good alternative here. There's no other knowledge work you could pivot to that would be a decent defense against what you're worried about. So you may as well play the hand you've got, even in the knowledge that it might lose.

Three, in the world where AI does surpass a good software engineer, there's a decent chance it surpasses a good ML engineer in the near future. And once it does that, we're in completely uncharted territory. Even if more extreme singularity-like scenarios don't come to pass, it doesn't need to be a singularity to become significantly superhuman to the point that almost nothing about the world in which we live continues to make any sense. So again, you lack any good alternatives.

And four: *if this is the last era in which human beings matter, I want to take advantage of it!* I may be among the very last entrepreneurs or businesswomen in the history of the human race! If I don't do this now, I'll never get the chance! If you want to be a software engineer, do it now, because you might never get the chance again.

It's totally reasonable to be scared, or stressed, or uncertain. Fear and stress and uncertainty are parts of life in far less scary times than these. But all you can do is play the hand you're dealt, and try not to be totally miserable while you're playing it.

-----

[1] https://manifold.markets/Royf214/will-ai-surpass-humans-in-c...

Re: Learning to Reason with LLMs

#557
post #533

Here's an unpopular take on this: "We had the chance to make AI decision-making auditable but are locking ourselves out of hundreds of critical applications by not exposing the chain of thought." One of the key blockers in many customer discussions I have is that AI models are not really auditable and that automating complex processes with them (let alone debug things when "reasoning" goes awry) is difficult if not i…

I suspect that actually reading the "chain of thought" would reveal obvious "logic" errors embarrassingly often.

Re: Learning to Reason with LLMs

#558

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

I think CS skills will remain valuable, but you should try to build some domain specific knowledge in addition. Perhaps programmer roles will eventually merge with product owner / business person type of roles.

Re: Learning to Reason with LLMs

#559

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

I went from economics dropout waiter who built a app startup with $0 funding and $1M a year in revenue by midway through year 1, sold it a few years later, then went to Google for 7 years, and last year I left. I'm mentioning that because the following sounds darn opinionated and brusque without the context I've capital-S seen a variety of people and situations. Sit down and be really honest with yourself. If your go…

That's more well balanced opinion comparing to others I seen here. I also believe that the golden age with 250k+ salaries with solving easy problems will be gone in 5-10 years. Most people look at this AI improvements at current state and forget that you are supposed to have a profession for 40 years until retirement. 250k+ jobs will still exist 10 years from now but expectations will be much higher and competition much bigger.

On the other hand now is the best time to build your own product as long you are not interested only in software as craftmanship but in product development in general. Probably in the future expectation will be your are not only monkey coder or craftman but also project lead/manager (for AI teams), product developer/designer and maybe even UX/designer if you will be working for some software house, consulting or freelancing.

Re: Learning to Reason with LLMs

#560

Student here. Can someone give me one reason why I should continue in software engineering that isn't denial and hopium?

I don't want to lean into negativity here, and I'm far from an "AI Doomer".

But... I will say I think the question you ask is a very fair question, and that there is, indeed, a LOT of uncertainty about what the future holds in this regard.

So far the best reason we have for optimism is history: so far the old adage has held up that "technology does destroy some jobs, but on balance it creates more new ones than it destroys." And while that's small solace to the buggy-whip maker or steam-engine engineer, things tend to work out in the long-run. However... history is suggestive, but far from conclusive. There is the well known "problem of induction"[1] which points out that we can't make definite predictions about the future based on past experience. And when those expectations are violated, we get "black swan events"[2]. And while they be uncommon, they do happen.

The other issue with this question is, we don't really know what the "rate of change" in terms of AI improvement is. And we definitely don't know the 2nd derivative (acceleration). So a short-term guess that "there will be a job for you in 1 year's time" is probably a fairly safe guess. But as a current student, you're presumably worried about 5 years, 10 years, 20 years down the line and whether or not you'll still have a career. And the simple truth is, we can't be sure.

So what to do? My gut feeling is "continue to learn software engineering, but make sure to look for ways to broaden your skill base, and position yourself to possibly move in other directions in the future". Eg, don't focus on just becoming a skilled coder in a particular language. Learn fundamentals that apply broadly, and - more importantly - learn about how business work, learn "people skills"[3], develop domain knowledge in one or more domains, and generally learn as much as you can about "how the world works". Then from there, just "keep your head on a swivel" and stay aware of what's going on around you and be ready to make adjustments as needed.

It might not also hurt to learn a thing or two about something that requires a physical presence (welding, etc.). And just in case a full-fledged cyberpunk dystopia develops... maybe start buying an extra box or two of ammunition every now and then, and study escape and evasion techniques, yadda yadda...

[1]: http://en.wikipedia.org/wiki/Problem_of_induction

[2]: https://en.wikipedia.org/wiki/Black_swan_theory

[3]: https://www.youtube.com/watch?v=hNuu9CpdjIo

Post reply on HN