61–70 of 383 posts
Earlier quoted context omitted.
I would say yes, everyone should care about it. There is plenty of material on the topic. See for example https://ai-2027.com/ or https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...
fear mongering science fiction, you may as well cite Dune or Terminator
I have programmed 30K+ hours. Do LLMs make bad code: yes all the time (at the moment zero clue about good architecture). Are they still useful: yes, extremely so. The secret sauce is that you'd know exactly what to do without them.
> There are certain tasks, like improving a given program for speed, for instance, where in theory the model can continue to make progress with a very clear reward signal for a very long time. Super skeptical of this claim. Yes, if I have some toy poorly optimized python example or maybe a sorting algorithm in ASM, but this won’t work in any non-trivial case. My intuition is that the LLM will spin its wheels at a loc…
https://github.com/algorithmicsuperintelligence/openevolve
I have programmed 30K+ hours. Do LLMs make bad code: yes all the time (at the moment zero clue about good architecture). Are they still useful: yes, extremely so. The secret sauce is that you'd know exactly what to do without them.
> The fundamental challenge in AI for the next 20 years is avoiding extinction. That's a weird thing to end on. Surely it's worth more than one sentence if you're serious about it? As it stands, it feels a bit like the fearmongering Big Tech CEOs use to drive up the AI stocks. If AI is really that powerful and I should care about it, I'd rather hear about it without the scare tactics.
Oil companies: we are causing global warming with all this carbon emissions, are you scared yet? so buy our stock
Pharma companies: our drugs are unsafe, full of side effects, and kill a lot of people, are you scared yet? so buy our stock
Software companies: our software is full of bugs, will corrupt your files and make you lose money, are you scared yet? so buy our stock
Classic marketing tactics, very effective.
Could not agree more. I myself started 2025 being very skeptical, and finished it very convinced about the usefulness of LLMs for programming. I have also seen multiple colleagues and friends go through the same change of appreciation.
I noticed that for certain task, our productivity can be multiplied by 2 to 4. So hence comes my doubts: are we going to be too many developers / software engineers ? What will happen for the rests of us ?
I assume that other fields (other than software-related) should also benefits from the same productivity boosts. I wonder if our society is ready to accept that people should work less. I think the more likely continuation is that companies will either hire less, or fire more, instead of accepting to pay the same for less hours of human-work.
What also happens and it's irrelevant of AGI: global RL Around the world people ask an LLM and get a response. Just grouping and analysing these questions and solving them once centrally and then making the solution available again is huge. Linearly solving the most asked questions and then the next one then the next will make, whatever system is behind it, smarter every day.
Exactly. The singularity is already here. It's just "programmers + AI" as a whole, rather than independent self-improvements of the AI. I wonder how a "programmers + AI" self-improving loop is different from an "AI only" one.
AGI will also be generic.
LLM is already very impressive though
> * Programmers resistance to AI assisted programming has lowered considerably. Even if LLMs make mistakes, the ability of LLMs to deliver useful code and hints improved to the point most skeptics started to use LLMs anyway: now the return on the investment is acceptable for many more folks. Could not agree more. I myself started 2025 being very skeptical, and finished it very convinced about the usefulness of LLMs f…
I propose that we should raise the bar for the quality of software now.