961–970 of 1001 posts
Am I the only one becoming less impressed by LLMs as time passes? I will admit, when Copilot first became a thing in 2021, I had my own “I’m about to become obsolete” moment. However, it’s become clear to me, both through my own experience and through research that has been conducted, that modern LLMs are fundamentally flawed and are not on the path to general intelligence. We are stuck with ancient (in AI terms) tec…
Today's big tech companies might seem invincible, but they’re not. Their edge? It's going to wear thin, especially once everyone else starts jumping in with custom solutions. Instagram with just 13 people gave Facebook a run for its money. That was before the era of LLMs. Imagine now – anyone could potentially build the next big thing in an afternoon.
And don’t get me started on the legal stuff. There are so many laws, no one person can know them all. But LLMs? They can handle it. What happens when the legal system gets flooded with cases about loopholes no one knew existed? Or when everyone can have the world's best lawyer for free? It's wild to think about.
ChatGPT came out of nowhere for most of us, and robotics is going to hit us harder and sooner than we realize. No politician or law can keep up with this tech freight train we're on. We’re talking about more progress in the next 20 years than in the last 20,000. Jobs and whether we can stay employed? That's going to be the least of our worries.
Or AI will hit a plateau and we'll all get a little bit more productive.
Earlier quoted context omitted.
It is especially sad that VC money is currently being spent on developing AI to eliminate good jobs rather than on developing robots to eliminate bad jobs.
Many machinists, welders, etc would have asked the same question when we shipped most of American manufacturing overseas. There was a generation of experienced people with good jobs that lost their jobs and white collar workers celebrated it. Just Google “those jobs are never coming back”, you’ll find a lot of heartless comparisons to the horse and buggy. Why should we treat these office jobs any differently?
Interestingly at the same time as all those jobs disappeared and got automated there were surges of people into the workforce. Women started to be routinely employed for all but a few years of child birth and care, and many workers came from overseas. Yet, white collar unemployment didn't spike. The driver for this was that the effective size of the economy boomed with the inclusion of Russia, China, Indonesia, India and many other smaller countries in the western sphere/economy post cold war... and growth from innovation.
Earlier quoted context omitted.
AI might help programmers become more rigorous by lowering the cost of formal methods. Imagine an advanced language where simply writing a function contract, in some kind of Hoare logic or using a dependently-typed signature, yields provably correct code. These kinds of ideas are already worked on, and I believe are the future.
I think the beauty of our craft on a theoretical level is that it very quickly outgrows all of our mathematics and what can be stated based on that (e.g. see the busy beaver problem). It is honestly, humbling and empowering at the same time. Even a hyper-intelligent AI will be unable to reason about any arbitrary code. Especially that current AI - while impressive at many things - is a far cry from being anywhere nea…
And yet we have free will (ok, within bounds, I cannot fly to the moon etc, but maybe my path integral allows it), we see processes like the expansion of the universe that we cannot account for and infer them like quantum gravity as well.
Earlier quoted context omitted.
Not necessarily. Every hockey stick is just the beginning of an s-curve. It will saturate, probably sooner than you think.
Some parts of AI will necessarily asymptote to human-level intelligence because of a fixed corpus of training data. It's hard to think AI will become a better creative writer than the best human creative writers, because the AI is trained on their output and you can't go much further than that. But in areas where there's self-play (e.g. Chess, and to a lesser extent, programming), there is no good reason to think it'…
The language model "creativity" is a total fraud. It is not creative at all but it takes time to see the edges. It is like AI art. AI art is mind blowing until you have seen the same 2000th variation on basically the same theme because it is so limited in what it can do.
To compare the simple game of chess to the entire space of what can be programmed on a computer is utterly absurd. You just don't know what you are talking about.
I have a simple front-end test that I give to junior devs. Every few months I see if ChatGPT can pass it. It hasn’t. It can’t. It isn’t even close. It answers questions confidently but with subtle inaccuracies. The code that it produces is the same kind of non-sense that you get from recent bootcamp devs who’ve “mastered” the 50 technologies on their eight page résumé. If it’s gotten better, I haven’t noticed. Self-d…
Earlier quoted context omitted.
i wonder how many of the commenters who bemoan how GPT-4 can't do this or that task, could have predicted 2 or 5 years ago what GPT-4 can do easily today
Yes I also only find people to be credible if they have a solid track record of magically predicting events 2 to 5 years in the future.
Earlier quoted context omitted.
My niche is in video game programming, and I am very specialized in a specific area. So I might ask things like how would one architect a certain game system with a number of requirements, to meet certain player expectations, and be compatible with a number of things. Unfortunately, it hasn’t been helpful once, and often due to the same reason - when the question gets specific enough, it hallucinates because it doesn…
Interesting. I also work in game development, and I tend to work on project-specific optimization problems, and I've had the opposite experience. If I have to solve a hairy problem specific to our game's architecture, obviously I'm not going to ask ChatGPT to solve that for me. It's everything else that it works so well for. The stuff that I could do, but it's not really worth my time to actually do it when I can be…
there might simply not be enough literature for LLM's to properly write this stuff in certain domains. I'm sure a graphics programmer would consider a lot of shader and DirectX API calls to be busy work, but I'm not sure if GPT can get more than a basic tutorial renderer working. Simply because there really isn't that much public literature to begin with, especially for DX12 and Vulkan. That part of games has tons of tribal knowledge kept in-house at large studios and Nvidia/intel/AMD so there's not much to go on.
But I can see it replacing various kinds of tools programming or even UI work soon, if not right now. It sounds like GPT works best for scripting tasks and there's tons of web literature to go off of (and many programmers hate UI work to begin with).
Earlier quoted context omitted.
AI might help programmers become more rigorous by lowering the cost of formal methods. Imagine an advanced language where simply writing a function contract, in some kind of Hoare logic or using a dependently-typed signature, yields provably correct code. These kinds of ideas are already worked on, and I believe are the future.
I'm not convinced about that. Writing a formal contract for a function is incredibly hard, much harder than writing the function itself. I could open any random function in my codebase and with high probability get a piece of code that is By "as correct", I mean that such a function may have bugs, but the same is true for an AI-generated function derived from a formal contract, if the contract has a loophole. And in…
Contracts can be quite easy to use, see e.g. Dafny by MS Research.
Earlier quoted context omitted.
HN’s culture has changed somewhat and downvotes are now used more often to signal disagreement, sadly. But also “use common sense” and “but the trendline” are only partially compelling arguments as presented if you already believe what is being argued. They’re not compelling enough to those who aren’t convinced yet
The trendline is the only argument. What other predictor of the future is there? Given the available information there is no condition where one would bet against the trendline. Common sense is basically trendline following. It's the basis of our existence. You get out of bed without worrying about whether or not there is no ground under your feet because the trendline points to a reality where the ground is always t…
Typically, experts actually thinking about how technology works, on a deep level, does a pretty good job.
Consider for example moore's law. A trendline everyone in the know knew couldn't continue long before it eventually failed. It wasn't a case of "well there are always some naysayers, they're right sometimes", it was anyone with any reasonable experience in building chips knew that each innovation is more hard won than the last, with physical barriers looming.
Is AI like that? inevitably. There are invisible physical barriers to all fields and all technologies. The only way to find them is to try. But the discussion here is essentially hypothesising where and when they will show up. We may well be able to run an AGI. Using current techniques, it will need to be trained on a vastly more powerful compute stack than gpt4. Its difficult to impart to you just how big their current one is. They are going to have to mobilise non-trivial segments of entire industries and supply chains just to be big enough for gpt5. There will also be neat tricks that will be found to reduce requirements. But eventually, some wall will be hit, gains will slow. The bet is whether we get to AGI or whatever before then