361–370 of 1001 posts
Earlier quoted context omitted.
> For teams you can measure meaningful outcomes and improve team metrics. How? Which metrics?
That is what we pay managers -to figure out- for. They should find out which and how by knowing the team, familiarity with domain knowledge, understanding company dynamics, understanding customer, understanding market dynamics.
I’m gonna bet this is a lie.
Earlier quoted context omitted.
I don't get it either. People will say all sorts of strange stuff about how it writes the code for them or whatever, but even using the new Claude 3.5 Sonnet or whatever variant of GPT4, the moment I ask it anything that isn't the most basic done-to-death boilerplate, it generates stuff that's wrong, and often subtly wrong. If you're not at least pretty knowledgeable about exactly what it's generating, you'll be stuc…
Interesting that you believe your subjective experience outweighs the claims of all others who report successfully using LLMs for coding. Wouldn't a more charitable interpretation be that it doesn't fit the stuff you're doing?
Regardless, I do wonder how accurate those successful reports are. Do people take LLM output, use it verbatim, not notice subtle bugs, and report that as success?
Sometimes I wonder why we would want LLMs spit out human readable code. Wouldn’t be a better future where LLMs generate highly efficient machine code and eventually we read the “source map” for debugging? Wasn’t source code just for humans?
Because you can't trust what the LLM generates, so you have to read it. Of course the question then is whether you can trust your developer or not.
Earlier quoted context omitted.
> if this can be accompanied by an increase in software quality That’s a huge “if”, and by your own admission not what’s happening now. > other LLM agents reviewing code, feeding in compile errors, letting other LLM agents interact with the produced code, etc. What a stupid future. Machines which make errors being “corrected” by machines which make errors in a death spiral. An unbelievable waste of figurative and lit…
> What a stupid future. Machines which make errors being “corrected” by machines which make errors in a death spiral. An unbelievable waste of figurative and literal energy. Humans are machines which make errors. Somehow, we got to the moon. The suggestion that errors just mindlessly compound and that there is no way around it, is what's stupid.
LLMs get their errors fed back into them and become more confident that their wrong code is right.
I'm not saying that's completely unsolvable, but that does seem to be how it works today.
To my experience, AIs can generate perfectly good code relatively easy things, the kind you might as well copy&paste from stackoverflow, and they'll very confidently generate subtly wrong code for anything that's non-trivial for an experienced programmer to write. How do people deal with this? I simply don't understand the value proposition. Does Google now have 25% subtly wrong code? Or do they have 25% trivial code…
I decided to go into programming instead of becoming an Engineer because most Engineering jobs seemed systematic and boring. (Software Engineers weren't really a thing at the time.) For most of my career, Software Engineering was a misnomer. The field was too young, and the tools used changed too quickly, for an appreciable amount of the work to be systematic and boring enough to consider it an Engineering discipline…
I’m an engineer who writes code since 20 years and it’s far away from trivial . Maybe to do web dev for a simple Webshop is. Elsewhere software has often times special requirements. Be them technical or domain wise both make the process complex and not simple IMHO
Earlier quoted context omitted.
I don't see how this is sustainable. We have essentially eaten the seed corn. These current LLMs have been trained by an enormous corpus of mostly human-generated technical knowledge from sources which we already know to be currently being polluted by AI-generated slop. We also have preliminary research into how poorly these models do when training on data generated by other LLMs. Sure, it can coast off of that initi…
The LLM codegen at Google isn't unsupervised. It's integrated into the IDE as both autocomplete and prompt-based assistant, so you get a lot of feedback from a) what suggestions the human accepts and b) how they fix the suggestion when it's not perfect. So future iterations of the model won't be trained on LLM output, but on a mixture of human written code and human-corrected LLM output. As a dev, I like it. It speed…
Earlier quoted context omitted.
> We work very closely with Google DeepMind to adapt Gemini models for Google-scale coding and other Software Engineering usecases. Considering how terrible and frequently broken the code that the public facing Gemini produces, I'll have to be honest that that kind of scares me. Gemini frequently fails at some fairly basic stuff, even in popular languages where it would have had a lot of source material to work from;…
We are sorely lacking a "Make Computer Science a Science" movement, the tech lead's blurb is par for the course, talking about "SWE productivity" with no reference to scientific inquiry and a foundational understanding of safety, correctness, verification, validation of these new LLM technologies.
Earlier quoted context omitted.
Supermaven is an incredible intellisense. Most code IS trivial and I barely write trivial code anymore. My imports appear instantly, with high accuracy. I have lots of embedded SQL queries and it’s able to guess the structure of my database very accurately. As I’m writing a query the suggested joins are accurate probably 80% of the time. I’m significantly more productive and having to type much less. If this is as go…
I don't think that is the signal that I think most people are hoping for here. When I hear that most code is trivial, I think of this as a language design or a framework related issue making things harder than they should be. Throwing AI or generates at the problem just to claim that they fixed it is just frustrating.
This was one of my thoughts too. If the pain of using bad frameworks and clunky languages can be mitigated by AI, it seems like the popular but ugly/verbose languages will win out since there's almost no point to better designed languages/framework. I would rather a good language/framework/etc where it is just as easy to just write the code directly. Similar time in implementation to a LLM prompt, but more deterministic.
If people don't feel the pain of AI slop why move to greener pastures? It almost encourages things to not improve at the code level.