Where's the product that everyone is using and loves? Um, Chatgpt. And Copilot.
I use them both. A lot. I'd hate to me without them.
The Generative AI Con
151–160 of 503 posts
Re: The Generative AI Con
#152Earlier quoted context omitted.
LLMs are pretty good at the aspects of coding that I consider to be "the fun part". Using them has made me more productive, but also made my job less fun, because I can't justify spending time using my own brain to do "the fun part" on my employer's dime. And that was something I was particularly good at, which is why I was able to be paid well to do it. So now my company makes more money, and the work gets done fast…
It's kind of analogous to the old taxi drivers who took pride in having a sixth sense knowing which route to take you, vs uber drivers who just blindly follow their navigation
Some of them might have had a really good mental map; but the majority would just take inefficient routes (and charge you some random price that they put into their counter) — plenty of reasons to dislike Uber but having a pre-set price, vetted/rated drivers, and clear routing for a taxi service is a massive plus in my opinion.
Re: The Generative AI Con
#153Earlier quoted context omitted.
Are they? I find Agentic mode on most editors barely useful. Autocomplete and inline editing is great though. To use these tools properly, you need to know how to build the same thing precisely.
Nah you don't to know to build the same thing precisely. Just the other day I wanted to write a vanilla JS component that could let you select a picture from something like a carousel and be able to blow up the selected picture when clicked. I know JS / HTML but am not used to working with vanilla JS. Copilot didn't write it all by itself but it did teach me things I didn't know like making a custom tag in vanilla JS…
It’s not a common use case though, dipping into an unfamiliar tech stack only to dip out after committing the code. Typically, when you learn a new stack (eg. for a job), you’ll be living in it for at least a few months, and at that point, you’d be better served by perusing the docs and getting deeply familiar with the API.
The copilots get you going quick at the expense of your learning, which is great for one-offs, but not lasting work quality.
Re: The Generative AI Con
#154I’m a little shocked at how much negativity there is around LLMs among developers. It’s a new tool that requires some learning, and it’s sometimes not so great, but if you’ve used an IDE with real coding assistance built in (eg. VS Code in Edit with Copilot mode - NOT Chat mode, using Claude 3.5), it’s honestly not much worse than a junior dev and 100x faster. And if the code is bad you throw it away and try again 10…
LLMs are pretty good at the aspects of coding that I consider to be "the fun part". Using them has made me more productive, but also made my job less fun, because I can't justify spending time using my own brain to do "the fun part" on my employer's dime. And that was something I was particularly good at, which is why I was able to be paid well to do it. So now my company makes more money, and the work gets done fast…
Huh, for me it's the opposite. It does the boring bit, writing pedestrian method bodies. Writing import statements. Closing tags.
I do the fun bit: having creative ideas and trying them out.
Re: The Generative AI Con
#155Ed occasionally makes good points, but he's very very angry at Big Tech, and his anger often gets in the way of his message. Reading his latest rant reminds me of Karl Denninger railing against Google around the time of their IPO, claiming they would never make enough money to justify an $85 share price (a $1000 investment then would be worth around $375,000 today).
Fortunately his message is still big enough to get through.
Re: The Generative AI Con
#156I could only skim to verify an automated summary, but if the takeaway is "these AI giants are doomed" then he's right.
The future, probably within 10 years, is most tasks being handled by small on-portable-device models (7B parameters or so; see Apple's Intelligence thing), a middle ground of workhorse models (pushing closer to 30s and 70s) running on more capable ML-focused chips in laptops and workstations, and home and office servers for the biggest professional users running on dedicated servers.
And then there's the apps. Whoever makes the "Stripe for generative AI" with multiple models with different levels of data provenance, security, SLA, etc for different use cases tied together with support for custom fine tuning stands a good chance of sweeping the market post-collapse.
Re: The Generative AI Con
#157> And even then, we still don't have a killer app! There is no product that everybody loves, and there is no iPhone moment! I would strongly argue that coding assistants are AI’s first killer app. Copilot, Cursor, Windsurf etc.
The killer app is entertainment. Since LLMs emerged people have consistently loved getting them to say whatever they want or roleplay with them. Once integrated into games, it will be very fun to have natural conversations with the inhabitants of game worlds. Imagine a goomba talking to Super Mario before he stomps its head or delivering your pithy one liner in response to some final boss’s villainous monologue.
Re: The Generative AI Con
#158> And even then, we still don't have a killer app! There is no product that everybody loves, and there is no iPhone moment! I would strongly argue that coding assistants are AI’s first killer app. Copilot, Cursor, Windsurf etc.
Say you reduced their revenue to only that application. Would it be sustainable? Would it be worth the billions upon billions of dollars that have been shoveled at it? Would it add more than the billions upon billions of dollars in the end?
By your logic I could claim a quantum computer with qubits on the scale of the mass of the sun is a killer app for doing RSA encryption breaking. And I would be making an equally useless statement.
Re: The Generative AI Con
#159Earlier quoted context omitted.
Cost as in, cost to you? Or cost to serve? If the cost-to-serve is subsidized by VC money, they aren't getting cheaper, they're just leading you on.
> Cost as in, cost to you? Or cost to serve? This. IIUC to serve an LLM is to perform an O(n^2) computation on the model weights for every single character of user input. These models are 40+GB so that means I need to provision about 40GB RAM per concurrent user and perform hundreds of TB worth of computations per query. How much would I have to charge for this? Are there any products where the users would actually g…
> IIUC to serve an LLM is to perform an O(n^2) computation on the model weights for every single character of user input.
The computations are not O(n^2) in terms of model weights (parameters), but linear. If it were quadratic, the number would be ludicrously large. Like, "it'll take thousands of years to process a single token" large.
(The classic transformers are quadratic on the context length, but that's a much smaller number. And it seems pretty obvious from the increases in context lengths that this is no longer the case in frontier models.)
> These models are 40+GB so that means I need to provision about 40GB RAM per concurrent user
The parameters are static, not mutated during the query. That memory can be shared between the concurrent users. The non-shared per-query memory usage is vastly smaller.
> How much would I have to charge for this?
Empirically, as little as 0.00001 cents per token.
For context, the Bing search API costs 2.5 cents per query.
Re: The Generative AI Con
#160> And even then, we still don't have a killer app! There is no product that everybody loves, and there is no iPhone moment! I would strongly argue that coding assistants are AI’s first killer app. Copilot, Cursor, Windsurf etc.
The killer app is entertainment. Since LLMs emerged people have consistently loved getting them to say whatever they want or roleplay with them. Once integrated into games, it will be very fun to have natural conversations with the inhabitants of game worlds. Imagine a goomba talking to Super Mario before he stomps its head or delivering your pithy one liner in response to some final boss’s villainous monologue.
and strangely in two years no one has demonstrated anything like this that people found of value. In fact for all the "entertainment" sectors it has been injected into, we have gotten poisoning of self-publishing and soulless generic pornography. Remember the twitch channels that had AI generated content? Where did those go? Surely by your rationale the market would have taken over by now. Surely there would be something.
It's almost like entertainment requires some humanity and thought and true creativity behind it.