331–340 of 373 posts
One time, it said "You were right; I corrected the regressions and committed them."
It was still not fixed.
I do admit not hitting the five hour limit in two prompts is nice. However, I wonder if becoming an angry person because of cheap models is worth it.
Earlier quoted context omitted.
At work i only had early copilot which was hysterically bad at everything. As i wanted it to do the same task repeatedly and could spot wrong results instantly i kept evolving a prompt that attempted to correct all ways it found to do it wrong. It kept inventing new ways to get it wrong until it eventually got it right 90% of the time. My theory is that an avanced model that has no issues with a task could do the pro…
Early Copilot was tab-complete in editors and was honestly the best version of LLM-assisted development I've used, because it was intentionally small in scope, trivial to verify output from at a glance and easy to opt in and opt out of. I have yet to see anything more useful in terms of code generation; very small-scale code generation (think function-level) comes close, but is a lot more tedious. For analysis, bug h…
Recently my employer hooked Claude into a bunch of SaaS services like a ticketing system and an asset inventory system, and I feel a big productivity boost is that I don't have to dig for their open browser tabs, remember their product name/URL/where I put the bookmark, find that my sessions have logged me out "for my protection" since I last looked, find my MFA code, approve sign-in on my phone, then use their mismashed/sluggish/poor-UX interfaces and slow searches to loko for things, and can instead ask Claude "search these systems for anything about " and it just does. It correlates between systems, summarizes things, and gives me references which system and where to look to check what it found. Internal search on a mess of acquired and disjoint systems.
This feeling reminds me of two classic Joel Spolsky blog posts below, and your comment here about the surprisingly useful tab-complete gives me a similar feeling. I wonder how many smaller candidates there are where LLMs are more helpful and less hyped, against the big vibe-coding solve-everything hype which are currently less helpful than their hype?
References [1]: "A lot of us thought in the 1990s that the big battle would be between procedural and object oriented programming, and we thought that object oriented programming would provide a big boost in programmer productivity. I thought that, too. Some people still think that. It turns out we were wrong. Object oriented programming is handy dandy, but it’s not really the productivity booster that was promised. The real significant productivity advance we’ve had in programming has been from languages which manage memory for you automatically. [...] Whenever you hear someone bragging about how productive their language is, they’re probably getting most of that productivity from the automated memory management, even if they misattribute it. Sidebar: Why does automatic memory management make you so much more productive? 1) Because you can write f(g(x)) without worrying about how to free the return value from g, which means you can use functions which return interesting complex data types and functions which transform interesting complex data types, in turn allowing you to work at a higher level of abstraction"
and [2]: "I have seen many language and programming fads come and go. But there’s only ONE, that’s right, ONE language feature I’ve ever seen that actually improves your productivity significantly. No, it’s not object oriented programming; no, it’s not intentional programming or assertions or programming by example or CASE or UML or XML or Java. The only thing that improves your programming productivity is using managed code – that is, using a language in which memory management is automatic."
[1] https://www.joelonsoftware.com/2004/06/13/how-microsoft-lost...
[2] https://www.joelonsoftware.com/2001/10/17/working-on-citydes...
Earlier quoted context omitted.
At work i only had early copilot which was hysterically bad at everything. As i wanted it to do the same task repeatedly and could spot wrong results instantly i kept evolving a prompt that attempted to correct all ways it found to do it wrong. It kept inventing new ways to get it wrong until it eventually got it right 90% of the time. My theory is that an avanced model that has no issues with a task could do the pro…
I've done this. Ask Claude to permutate a prompt and run Claude Code or a subagent to observe effects until success. Claude will iterate on the prompt faster than I could and likely catch more edge cases, too. It's one of those tasks where the end goal is very clear and the agent just needs to iterate on permutations. Arguably this is a perfect match, too, because prompt engineering is really just text generation, so…
Earlier quoted context omitted.
Early Copilot was tab-complete in editors and was honestly the best version of LLM-assisted development I've used, because it was intentionally small in scope, trivial to verify output from at a glance and easy to opt in and opt out of. I have yet to see anything more useful in terms of code generation; very small-scale code generation (think function-level) comes close, but is a lot more tedious. For analysis, bug h…
> " I have yet to see anything more useful in terms of code generation " Recently my employer hooked Claude into a bunch of SaaS services like a ticketing system and an asset inventory system, and I feel a big productivity boost is that I don't have to dig for their open browser tabs, remember their product name/URL/where I put the bookmark, find that my sessions have logged me out "for my protection" since I last lo…
Earlier quoted context omitted.
> What would consumer AI company even be? Something that helps homeowners build anything on their property, for example https://myhomecat.com
What purpose does AI serve here apart from lowering costs by generating a plan? Especially if I need to ‘consult an expert’ isn’t this just a customer company using AI?
You can say what you want, and the tool has appropriate guardrails to make sure what you're going to build is safe.
It makes sense that we’ll see “room at the bottom” strategies. Currently, large parameter counts seem to be slush funds of world knowledge, language skills (because language’s nuances and open vocabulary make it high-dimensional), and reasoning primitives, the general belief being that the latter takes up the least space in the model. There are many applications where world knowledge is unnecessary or even a negative…
Everyone wants this to be it but over and over we discover that the bigger a model is the better it is at all tasks, even ones far outside the domain it was optimized for. IE claude fable is better at writing both code and prose than smaller code- and prose-specific models. The way vision and language models converge into the same geometric space should be extremely alarming for the "you don't need global knowledge f…
It feels like claiming that because a current generation laptop CPU is slower than a current generation HEDT CPU, we should ignore that the laptop CPU is faster and more efficient than the previous generation HEDT CPU.
Of course a huge system could do more, but these systems are constantly evolving and just in this month we've seen comparatively smaller models approach capabilities that were frontier AI less than a year ago. What you're saying would be true when we eventually find the limits of how much models can compress knowledge. Until then, all I think you're saying is that companies that are willing to burn more money can afford to ignore efficiency.
A friend of mine told me earlier today that they had a discussion at work (a coding shop) about "downgrading" to luna from sol for cost reasons and that many were quite unhappy about this because they didn't want inferior tech to be forced upon them. Do they have a point? Is sol actually worth the extra cost? Especially if you ramp up the effort level?
I don’t love the “forced upon them” framing; if that’s really how people are thinking about it then maybe they should pause and reflect for a moment yhat it isn’t their money being spent. Amd the default isn’t always having the latest and greatest, it’s not paying for anything at all. Now, if the debate is really about which option is more cost effective, then we could easily run an A/B test to find out. Though TBH m…
I'd object to "easily". It's already hard to measure whether AI is generally worth the cost. Let alone compare models in such detailled ways. It's mostly handwaving and gut feeling Doesn't mean the conclusions are wrong, but biases are strong.