Step 1. Build “AI” (LLM models) that can’t be trusted, doesn’t learn, forgets instructions, and frustrates software engineers
Step 2. Automate the use of these LLMs into “agents”
Step 3. ???
Step 4. Profit
Watching AI drive Microsoft employees insane
101–110 of 580 posts
Re: Watching AI drive Microsoft employees insane
#102Well, the coding agent is pretty much a junior dev at the moment. The seniors are teaching it. Give it a 100k PRs with senior developer feedback and it'll improve just like you'd anticipate a junior would. There is no way that FANG aren't using the comments by the seniors as training data for their next version. It's a long-term play to have pricey senior developers argue with an llm
> Give it a 100k PRs with senior developer feedback
Don't you think it has already been trained with, I don't know, maybe millions of PRs?
Re: Watching AI drive Microsoft employees insane
#103Earlier quoted context omitted.
Depends on team but seems management is pushing it from https://news.ycombinator.com/item?id=44031432 "From talking to colleagues at Microsoft it's a very management-driven push, not developer-driven. Friend on an Azure team had a team member who was nearly put on a PIP because they refused to install the internal AI coding assistant. Every manager has "number of developers using AI" as an OKR, but anecdotally most d…
> Allegedly it's pretty terrible at C# In my experience, LLMs in general are really, really bad at C# / .NET , and it worries me as a .NET developer. With increased LLM usage, I think development in general is going to undergo a "great convergence". There's a positive(1) feedback loop where LLM's are better at Blub, so people use them to write more Blub. With more Blub out there, LLMs get better at Blub. The language…
My impression is also that they are worse at C# than some other languages. In autocomplete mode in particular it is very easy to cause the AI tools to write terrible async code. If you start some autocomplete but didn't put an await in front, it will always do something stupid as it can't add the await itself at that position. But also in other cases I've seen Copilot write just terrible async code.
Re: Watching AI drive Microsoft employees insane
#104Many here don't seem to get it. The AI agent/programmer corpo push is not about the capabilities and whether they match human or not. It's about being able to externalize a majority of one's workforce without having a lot of people on permanent payroll. Think in terms of an infinitely scalable bunch of consultants you can hire and dismiss at your will - they never argue against your "vision", either.
This was already possible with outsourcing and offshoring. I suppose there's a new market of AI "employees" for small businesses that couldn't manage or legally deal with outsourcing their work already.
Re: Watching AI drive Microsoft employees insane
#105rah, we might be in trouble here. The primary issue at play is that we don't have a reliable means of measuring developer performance, outside of subjective judgement like end of year reviews.
This means its probably quite hard to measure the gain or the drag of using these agents. On one side, its a lot cheaper than a junior, but on the other side it pulls time from seniors and doesn't necessarily follow instruction well (i.e. "errr your new tests are failing").
This combined with the "cult of the CEO" sets the stage for organisational dissonance where developer complaints can be dismissed as "not wanting to be replaced" and the benefits can be overstated. There will be ways of measuring this, to project it as huge net benefit (which the cult of the CEO will leap upon) and there will be ways of measuring this to project it as a net loss (rabble rousing developers). All because there is no industry standard measure accepted by both parts of the org that can be pointed at which yields the actual truth (whatever that may be).
If I might add absurd conjecture: We might see interesting knock-on effects like orgs demanding a lowering of review standards in order to get more AI PRs into the source.
Re: Watching AI drive Microsoft employees insane
#106Well, the coding agent is pretty much a junior dev at the moment. The seniors are teaching it. Give it a 100k PRs with senior developer feedback and it'll improve just like you'd anticipate a junior would. There is no way that FANG aren't using the comments by the seniors as training data for their next version. It's a long-term play to have pricey senior developers argue with an llm
These things don't learn after training. There is no teaching going on here, and the arguments probably don't make for good training data without more refinement. That's why junior devs are still better than LLMs IMO, they do learn.
This is a performative waste of time
Re: Watching AI drive Microsoft employees insane
#107Earlier quoted context omitted.
> I can't think of another feature so blatently pushed in your face Passkeys. As someone who doesn't see the value of it, every hype-driven company seems to be pushing me to replace OPT 2FA with something worse right now.
It's because OTP is trivially phishable: setup a fake login form that asks the user for their username and password, then forwards those on to the real system and triggers the OTP request, then requests THAT of the user and forwards their response. Passkeys fix that.
Except if you use a proper password manager that prevents you from using the autofill on domains/pages others than the hardcoded ones. In my case, it would immediately trigger my "sus filter" if the automatic prompt doesn't show up and I would have to manually find the entry.
Re: Watching AI drive Microsoft employees insane
#108It's mind blowing that a computer program can accomplish this much and yet absurd that it accomplishes so little.
Re: Watching AI drive Microsoft employees insane
#109Interesting that every comment has "Help improve Copilot by leaving feedback using the or buttons" suffix, yet none of the comments received any feedback, either positive or negative. > This seems like it's fixing the symptom rather than the underlying issue? This is also my experience when you haven't setup a proper system prompt to address this for everything an LLM does. Funniest PRs are the ones that "resolves" t…
> I can't imagine how the people who have to deal with this are feeling. It's like you have a junior developer except they don't even read what you're telling them, and have 0 agency to understand what they're actually doing.
That comparison is awful. I work with quite a few Junior developers and they can be competent. Certainly don't make the silly mistakes that LLMs do, don't need nearly as much handholding, and tend to learn pretty quickly so I don't have to keep repeating myself.
LLMs are decent code assistants when used with care, and can do a lot of heavy lifting, they certainly speed me up when I have a clear picture of what I want to do, and they are good to bounce off ideas when I am planning for something. That said, I really don't see how it could meaningfully replace an intern however, much less an actual developer.
Re: Watching AI drive Microsoft employees insane
#110Earlier quoted context omitted.
> Do we know for a fact there are Microsoft employees who were told they have to use CoPilot and review its change suggestions on projects? It wouldn't be out of character, Microsoft has decided that every project on GitHub must deal with Copilot-generated issues and PRs from now on whether they want them or not. There's deliberately no way to opt out. https://github.com/orgs/community/discussions/159749 Like Googles…
Holy sh*t I didn't know this was going on. It's like an AI tsunami unleashed by Microsoft that will bury the entire software industry... They are like Trump and his tariffs, but for the software economy. What this tells me is that software enterprises are so hellbent in firing their programmers and reducing their salary costs they they are willing to combust their existing businesses and reputation into the dumpster…
> What this tells me is that software enterprises are so hellbent in firing their programmers and reducing their salary costs they they are willing to combust their existing businesses and reputation into the dumpster fire they are making. I expected this blatant disregard for human society to come ten or twenty years into the future
Have you been sleeping under a rock for the last decade? This has been going on for a long long time. Outsourcing been the name of the game for so long people seem to forgot it's happening it all.