Meanwhile $employer is continuing to migrate individual tasks to in-house AI tooling, and has licensed an off-the-shelf coding agent for all of us developers to put in our IDEs.
Let's be honest, Generative AI isn't going all that well
141–150 of 346 posts
Re: Let's be honest, Generative AI isn't going all that well
#142A year ago I would have agreed wholeheartedly and I was a self confessed skeptic. Then Gemini got good (around 2.5?), like I-turned-my-head good. I started to use it every week-ish, not to write code. But more like a tool (as you would a calculator). More recently Opus 4.5 was released and now I'm using it every day to assist in code. It is regularly helping me take tasks that would have taken 6-12 hours down to 15-3…
I would strongly recommend this podcast episode with Andrej Karpathy. I will poorly summarize it by saying his main point is that AI will spread like any other technology. It’s not going to be a sudden flash and everything is done by AI. It will be a slow rollout where each year it automates more and more manual work, until one day we realize it’s everywhere and has become indispensable. It sounds like what you are s…
Though I think it's a very steep sigmoid that we're still far on the bottom half of.
For math it just did its first "almost independent" Erdos problem. In a couple months it'll probably do another, then maybe one each month for a while, then one morning we'll wake up and find whoom it solved 20 overnight and is spitting them out by the hour.
For software it's been "curiosity ... curiosity ... curiosity ... occasionally useful assistant ... slightly more capable assistant" up to now, and it'll probably continue like that for a while. The inflection point will be when OpenAI/Anthropic/Google releases an e2e platform meant to be driven primarily by the product team, with engineering just being co-drivers. It probably starts out buggy and needing a lot of hand-holding (and grumbling) from engineering, but slowly but surely becomes more independently capable. Then at some point, product will become more confident in that platform than their own engineering team, and begin pushing out features based on that alone. Once that process starts (probably first at OpenAI/Anthropic/Google themselves, but spreading like wildfire across the industry), then it's just a matter of time until leadership declares that all feature development goes through that platform, and retains only as many engineers as is required to support the platform itself.
Re: Let's be honest, Generative AI isn't going all that well
#143Earlier quoted context omitted.
Because the likes of Altman have set short term expectations unrealistically high.
I mean that's every tech company. I made a joke once after the first time I watched one of those Apple announcement shows in 2018, where I said "it's kind of sad, because there won't be any problems for us to solve because the iPhone XS Max is going to solve all of them". The US economy is pretty much a big vibes-based Ponzi scheme now, so I don't think we can single-out AI, I think we have to blame the fact that the…
"Ponzi" requires records fraud and is popularly misused, sort of like if people started describing every software bug as "a stack overflow."
I'd rather characterize it as extremes of Greater Fool Theory.
Re: Let's be honest, Generative AI isn't going all that well
#144I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…
This comment or something very close always appears alongside a Gary Marcus post.
I think it’s for good reason. I’m a bit at a loss as to why every time this guy rages into the ether of his blog it’s considered newsworthy. Celebrity driven tech news is just so tiresome. Marcus was surpassed by others in the field and now he’s basically a professional heckler on a university payroll. I wish people could just be happy for the success of others instead of fuming about how so and so is a billionaire and they are not.
Re: Let's be honest, Generative AI isn't going all that well
#145Earlier quoted context omitted.
Yes they can. The size of many codebases is much larger and LLMs can handle those. Consider also that they can generate summaries and tackle the novel piecemeal, just like a human would. Re: movies. Get YouTube premium and ask YouTube to summarize a 2hr video for you.
Novel is different from a codebase. In code you can have a relationship between files and most files can be ignored depending on what you're doing. But for a novel, its a sequential thing, in most cases A leads to B and B leads to C and so on. > Re: movies. Get YouTube premium and ask YouTube to summarize a 2hr video for you. This is different from watching a movie. Can it tell what suit actor was wearing? Can it tel…
Yes, it is possible to do those things and there are benchmarks for testing multimodal models on their ability to do so. Context length is the major limitation but longer videos can be processed in small chunks whose descriptions can be composed into larger scenes.
https://github.com/JUNJIE99/MLVU
https://huggingface.co/datasets/OpenGVLab/MVBench
Ovis and Qwen3-VL are examples of models that can work with multiple frames from a video at once to produce both visual and temporal understanding
Re: Let's be honest, Generative AI isn't going all that well
#146Earlier quoted context omitted.
There’s a subtle point a moment when you HAVE to take the driver wheel from the AI. All issues I see are from people insisting to use far beyond the point it stops being useful. It is a helper, a partner, it is still not ready go the last mile
As someone else said in this thread: > The whole discourse around LLMs is so utterly exhausting. If I say I don't like them for almost any reason, I'm a luddite. If I complain about their shortcomings, I'm just using it wrong. If I try and use it the "right" way and it still gets extremely basic things wrong, then my expectations are too high. I’m perfectly happy to write code, to use these tools. I do use them, and…
There's no silver bullet. I’m not a researcher, but I’ve done my best to understand how these systems work—through books, video courses, and even taking underpaid hourly work at a company that creates datasets for RLHF. I spent my days fixing bugs step-by-step, writing notes like, “Hmm… this version of the library doesn’t support protocol Y version 4423123423. We need to update it, then refactor the code so we instantiate ‘blah’ and pass it to ‘foo’ before we can connect.”
That experience gave me a deep appreciation for how incredible LLMs are and the amazing software they can power—but it also completely demystified them. So by all means, let’s use them. But let’s also understand there are no miracles here. Go back to Shannon’s papers from the ’60s, and you'll understand that what seems to you like "emerging behaviors" are quite explainable from an information theory background. Learn how these models are built. Keep up with the latests research papers. If you do, you’ll recognize their limitations before those limitations catch you by surprise.
There is no silver bullet. And if you think you’ve found one, you’re in for a world of pain. Worse still, you’ll never realize the full potential of these tools, because you won’t understand their constraints, their limits, or their pitfalls.
Re: Let's be honest, Generative AI isn't going all that well
#147Earlier quoted context omitted.
Because the likes of Altman have set short term expectations unrealistically high.
I maintain that most anti-AI sentiment is actually anti-lying-tech-CEO sentiment misattributed. The technology is neat, the people selling it are ghouls.
This is how I felt about Bitcoin.
Re: Let's be honest, Generative AI isn't going all that well
#148Earlier quoted context omitted.
Not quite the same analysis. The human is better, no surprise. But the NotebookLM output links back to the original book in a very useful way. If you think about it as fuzzy semantic search it's amazing. If you want an essay or even just creativity, yes it's lacking.
It doesn't have to be the same analysis to put it in a partially overlapping vector space. Not saying it wasn't a useful perspective shuffling in the vector space, but it definitely wasn't original. LLMs haven't solved any of the 2029 predictions as they were posited. But I expect some will be reached by 2029. The AI hype acts like all this is easy. Not by 2029 doesn't mean impossible or even most of the way there.
LLMs will never achieve anything as long as any victory can be hand waved away with "in the training set". Somehow these models have condensed the entire internet down to a few TB's, yet people aren't backing up their terabytes of personal data down to a couple MB using this same tech...wonder why
Re: Let's be honest, Generative AI isn't going all that well
#149Earlier quoted context omitted.
lol same. I just wrote a bunch of diagrams with mermaid that would legit take me a week, also did a mock of an UI for a frontend engineer that would take me another week to do .. or some designers. All of that in between meetings... Waiting for it to actually go well to see what else I can do !
The more I have this experience and read people maligning AI for coding, the more I think the junior developers are actually not the ones in danger.
Oh I've thought this for years. As an L7, basically my primary role is to serve as someone to bounce ideas off of, and to make recommendations based on experience. A chatbot, with its virtually infinite supply of experience, could ostensibly replace my role way sooner than it could a solid junior/mid-level coder. The main thing it needs is a consistent vision and direction that aligns with the needs of nearby teams, which frankly sounds not all that hard to write in code (I've been considering doing this).
Probably the biggest gap would be the ability to ignite, drive, and launch new initiatives. How does an AI agent "lead" an engineering team? That's not something you can code up in an agent runtime. It'd require a whole culture change that I have a hard time seeing in reality. But of course if there comes a point where AI takes all the junior and mid-level coding jobs, then at that point there's no culture to change, so staff/principal jobs would be just as at risk.
Re: Let's be honest, Generative AI isn't going all that well
#150I believe Gary Marcus is quite well known for terrible AI predictions. He's not in any way an expert in the field. Some of his predictions from 2022 [1] > In 2029, AI will not be able to watch a movie and tell you accurately what is going on (what I called the comprehension challenge in The New Yorker, in 2014). Who are the characters? What are their conflicts and motivations? etc. > In 2029, AI will not be able to r…
> Many of these have already been achieved, and it's only early 2026.
I'm quite sure people who made those (now laughable) predictions will tell you none of these has been achieved, because AI isn't doing this "reliably" or "bug-free."
Defending your predictions is like running an insurance company. You always win.