321–330 of 742 posts
Earlier quoted context omitted.
This might be the measurable "some" non deceptive time saving, whereas most of it is still deceptive in terms of time saved
Except actual studies objectively show efficiency gains, more with junior devs, which make sense. So no, it's not a "deception" but it is often overstated in popular media.
Earlier quoted context omitted.
> Most people recognize that LLM's are not currently super intelligent, I think calling it intelligent is being extremely generous. Take a look at the following example which is a spelling and grammar checker that I wrote: https://app.gitsense.com/?doc=f7419bfb27c89&temperature=0.50... When the temperature is 0.5, both Claude 3.5 and GPT-4o can't properly recognize that GitHub is capitalized. You can see the response…
> I think calling it intelligent is being extremely generous ... can't properly recognize that GitHub is capitalized. Wouldn't this make chimpanzees and ravens and dolphins unintelligent too? You're asking it to do a task that's (mostly) easy for humans . It's not a human though. It's an alien intelligence which "thinks" in our language, but not in the same way we do. If they could, specialized AI might think we're u…
Animals have the ability to learn and grow by themselves. LLMs are not intelligent and I don't see how they can be since they just follow the most likely path with randomness (temperature) sprinkled in.
I use cursor and its tab completion; while what it can do is mind blowing, in practice I’m not noticing a productivity boost. I find that ai can help significantly with doing plumbing, but it has no problems with connecting the pipes wrong. I need to double and triple check the updated code - or fix the resulting errors when I don’t do that. So: boilerplate and outer app layers, yes; architecture and core libraries,…
I'm actually very curious why AI use is such a bi-modal experience. I've used AI to move multi thousand line codebases between languages. I've created new apps from scratch with it. My theory is the willingness to baby sit and the modality. I'm perfectly fine telling the tool I use its errors and working side by side with it like it was another person. At the end of the day it can belt out lines of code faster than I…
I use cursor and its tab completion; while what it can do is mind blowing, in practice I’m not noticing a productivity boost. I find that ai can help significantly with doing plumbing, but it has no problems with connecting the pipes wrong. I need to double and triple check the updated code - or fix the resulting errors when I don’t do that. So: boilerplate and outer app layers, yes; architecture and core libraries,…
For now, I mostly use AI as a "faster typist". If it wants to complete what I wanted to type anyway, or something extremely similar, I just press tab, otherwise I type my own code. I'd say about 70% of individual lines are obvious enough if you have the surrounding context that this works pretty well in practice. This number is somewhat lower in normal code and higher in unit tests. Another use case is writing one-of…
Earlier quoted context omitted.
Except cursor is the fireworks based on black powder here. It will look good, but as a technology to get you to the moon it seems to look like a dead end. NOTHING (of serious science) seems to indicate LLMs being anything but a dead end with the current hardware capabilites. So then I ask: What, in qualitative terms, makes you think AI in the current form will be capable of this in 5 or 10 years? Other than seeing th…
> NOTHING (of serious science) seems to indicate LLMs being anything but a dead end with the current hardware capabilites. In the same sense that black powder sucks as a rocket propellant - but it's enough to demonstrate that iterating on the same architecture and using better fuels will get you to the Moon eventually. LLMs of today are starting points, and many ideas for architectural improvements are being explored…
I use cursor and its tab completion; while what it can do is mind blowing, in practice I’m not noticing a productivity boost. I find that ai can help significantly with doing plumbing, but it has no problems with connecting the pipes wrong. I need to double and triple check the updated code - or fix the resulting errors when I don’t do that. So: boilerplate and outer app layers, yes; architecture and core libraries,…
It's the subtle errors that are really difficult to navigate. I got burned for about 40 hours on a conditional being backward in the middle of an otherwise flawless method. The apparent speed up is mostly a deception. It definitely helps with rough outlines and approaches. But, the faster you go, the less you will notice the fine details, and the more assumptions you will accumulate before realizing the fundamental e…
Earlier quoted context omitted.
Claude is better than OpenAI for most tasks, and yet OpenAI has enormously more users. What is this, if not first mover advantage?
Claude is only better in some cherry picked standard eval benchmarks, which are becoming more useless every month due to the likelihood of these tests leaking into training data. If you look at the Chatbot Arena rankings where actual users blindly select the best answer from a random choice of models, the top 3 models are all from OpenAI. And the next best ones are from Google and X.
I've been using LLMs as my rubber duck when I get stuck debugging something and have exhausted my standard avenues. GPT-4o tends to give me very general advice that I have almost always already tried or considered, while Claude is happy to say "this snippet looks potentially incorrect; please verify XYZ" and it has gotten me back on track in maybe 4/5 cases.
Earlier quoted context omitted.
It's the subtle errors that are really difficult to navigate. I got burned for about 40 hours on a conditional being backward in the middle of an otherwise flawless method. The apparent speed up is mostly a deception. It definitely helps with rough outlines and approaches. But, the faster you go, the less you will notice the fine details, and the more assumptions you will accumulate before realizing the fundamental e…
That’s the thing, isn’t it? The craft of programming in the small is one of being intimate with the details, thinking things through conscientiously. LLMs don’t do that.
> I un-did your change which made no sense to me and now everything is broken, why is what you did necessary?
I think we just have to ask ourselves what we want it to be good at, and then be diligent about generating decades worth of high quality training material in that domain. At some point, it'll start getting the details right.
I use cursor and its tab completion; while what it can do is mind blowing, in practice I’m not noticing a productivity boost. I find that ai can help significantly with doing plumbing, but it has no problems with connecting the pipes wrong. I need to double and triple check the updated code - or fix the resulting errors when I don’t do that. So: boilerplate and outer app layers, yes; architecture and core libraries,…
I'm actually very curious why AI use is such a bi-modal experience. I've used AI to move multi thousand line codebases between languages. I've created new apps from scratch with it. My theory is the willingness to baby sit and the modality. I'm perfectly fine telling the tool I use its errors and working side by side with it like it was another person. At the end of the day it can belt out lines of code faster than I…
I'm willing to accept that I just have "get off my lawn" syndrome or something.
But the idea of letting an LLM write/move large swaths of code seems so incredibly irresponsible. Whenever I sit down to write some code, be it a large implementation or a small function, I think about what other people (or future versions of myself) will struggle with when interacting with the code. Is it clear and concise? Is it too clever? Is it too easy to write a subtle bug when making changes? Have I made it totally clear that X is relying on Y dangerous behavior by adding a comment or intentionally making it visible in some other way?
It goes the other way too. If I know someone well (or their style) then it makes evaluating their code easier. The more time I spend in a codebase the better idea I have of what the writer was trying to do. I remember spending a lot of time reading the early Redis codebase and got a pretty good sense of how Salvatore thinks. Or altering my approaches to code reviews depending on which coworker was submitting it. These weren't things I were doing out of desire but because all non-trivial code has so much subtlety; it's just the nature of the beast.
So the thought of opening up a codebase that was cobbled together by an AI is just scary to me. Subtle bugs and errors would be equally distributed across the whole thing instead of where the writer was less competent (as is often the case). The whole thing just sounds like a gargantuan mess.
Change my mind.
Earlier quoted context omitted.
Why aren't you writing unit tests just because AI wrote the function? Unit tests should be written regardless of the skill of the developer. Ironically, unit tests are also one area where AI really does help move faster. High level design, rough outlines and approaches, is the worst place to use AI. The other place AI is pretty good is surfacing api call or function calls you might not know about if you're new to the…
I have completely the opposite perspective. Unit tests actually need to be correct, down to individual characters. Same goes with API calls. The API needs to actually exist. Contrast that with "high level design, rough outlines". Those can be quite vague and hand-wavy. That's where these fuzzy LLMs shine. That said, these LLM-based systems are great at writing "change detection" unit tests that offer ~zero value (or…
That’s not at all true in my experience. With minimal guidance they put out pretty sensible tests.