Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Watching AI drive Microsoft employees insane

old.reddit.com

Post reply on HN
Print view View original

11–20 of 580 posts

Re: Watching AI drive Microsoft employees insane

#11

Post by lovehashbrowns »

mtmail wrote: post #7
cebert wrote: post #2

Do we know for a fact there are Microsoft employees who were told they have to use CoPilot and review its change suggestions on projects? We have the option to use GitHub CoPilot on code reviews and it’s comically bad and unhelpful. There isn’t a single member of my team who find it useful for anything other than identifying typos.

Depends on team but seems management is pushing it from https://news.ycombinator.com/item?id=44031432 "From talking to colleagues at Microsoft it's a very management-driven push, not developer-driven. Friend on an Azure team had a team member who was nearly put on a PIP because they refused to install the internal AI coding assistant. Every manager has "number of developers using AI" as an OKR, but anecdotally most d…

All of that is working, at least, because the very small company I work for with a limited budget is working on getting an extremely expensive copilot license. Oh no, I might have to deal with this soon..

Re: Watching AI drive Microsoft employees insane

#12

Post by api »

rvz wrote: post #4

After all of that, every PR that Copilot opened still has failing tests and it failed to fix the issue (because it fundamentally cannot reason). No surprises here. It always struggles on non-web projects or on software where it really matters that correctness is first and foremost above everything, such as the dotnet runtime. Either way, a complete disastrous start and what a mess that Copilot has caused.

Part of why it works better on web projects is the sheer volume of training data. There is probably more JS written than any other language by orders of magnitude. Its quality is pretty dubious though.

I have so far only found LlMs useful as a way of researching, an alternative to web search, and doing very basic rote tasks like implementing unit tests or doing a first pass explanation of some code. Tried actually writing code and it’s not usable.

Re: Watching AI drive Microsoft employees insane

#14

Post by bayindirh »

ankitml wrote: post #13

GitHub is not the place to write code. IDE is the place. Along with pre CI checks, some tests, coverage etc. they should get some PM before making decisions..

This is the future envisioned by Microsoft. Vibe coding all the way down, social network style.

They are putting this in front of the developers as take it or leave it deal. I left the platform, doing my coding old way, hosting it somewhere else.

Discoverability? I don't care. I'm coding it for myself and hosting in the open. If somebody finds it, nice. Otherwise, mneh.

Re: Watching AI drive Microsoft employees insane

#15

Post by XorNot »

jsheard wrote: post #3
cebert wrote: post #2

Do we know for a fact there are Microsoft employees who were told they have to use CoPilot and review its change suggestions on projects? We have the option to use GitHub CoPilot on code reviews and it’s comically bad and unhelpful. There isn’t a single member of my team who find it useful for anything other than identifying typos.

> Do we know for a fact there are Microsoft employees who were told they have to use CoPilot and review its change suggestions on projects? It wouldn't be out of character, Microsoft has decided that every project on GitHub must deal with Copilot-generated issues and PRs from now on whether they want them or not. There's deliberately no way to opt out. https://github.com/orgs/community/discussions/159749 Like Googles…

Which almost feels unique to AI. I can't think of another feature so blatently pushed in your face, other then perhaps when everyone lost their minds and decided to cram mobile interfaces onto every other platform.

Re: Watching AI drive Microsoft employees insane

#16

Post by jsheard »

api wrote: post #12
rvz wrote: post #4

After all of that, every PR that Copilot opened still has failing tests and it failed to fix the issue (because it fundamentally cannot reason). No surprises here. It always struggles on non-web projects or on software where it really matters that correctness is first and foremost above everything, such as the dotnet runtime. Either way, a complete disastrous start and what a mess that Copilot has caused.

Part of why it works better on web projects is the sheer volume of training data. There is probably more JS written than any other language by orders of magnitude. Its quality is pretty dubious though. I have so far only found LlMs useful as a way of researching, an alternative to web search, and doing very basic rote tasks like implementing unit tests or doing a first pass explanation of some code. Tried actually wr…

> Part of why it works better on web projects is the sheer volume of training data.

OTOH webdev is known for rapid framework/library churn, so before too long there will be a crossroads where the pre-AI training data is too old and the fresh training data is contaminated by the firehose of vibe coded slop.

Re: Watching AI drive Microsoft employees insane

#17

Post by worldsayshi »

diggan wrote: post #5

Interesting that every comment has "Help improve Copilot by leaving feedback using the or buttons" suffix, yet none of the comments received any feedback, either positive or negative. > This seems like it's fixing the symptom rather than the underlying issue? This is also my experience when you haven't setup a proper system prompt to address this for everything an LLM does. Funniest PRs are the ones that "resolves" t…

> How are people reviewing that?

I agree that not auto-collapsing repeated annotations is an annoying bug in the github interface.

But just pointing out that annotations can be hidden in the ... menu to the right (which I just learned).

Re: Watching AI drive Microsoft employees insane

#18

Post by eterevsky »

margorczynski wrote: post #8

With how stochastic the process is it makes it basically unusable for any large scale task. What's the plan? To roll the dice until the answer pops up? That would be maybe viable if there was a way to automatically evaluate it 100% but with a human in the loop required it becomes untenable.

The plan is to improve AI agents from their current ~intern level to a level of a good engineer.

Re: Watching AI drive Microsoft employees insane

#19

Post by xnorswap »

diggan wrote: post #5

Interesting that every comment has "Help improve Copilot by leaving feedback using the or buttons" suffix, yet none of the comments received any feedback, either positive or negative. > This seems like it's fixing the symptom rather than the underlying issue? This is also my experience when you haven't setup a proper system prompt to address this for everything an LLM does. Funniest PRs are the ones that "resolves" t…

> How are people reviewing that? 90% of the page height is taken up by "Check failure",

Typically, you wouldn't bother manually reviewing something until the automated checks have passed.

Re: Watching AI drive Microsoft employees insane

#20

Post by worldsayshi »

bayindirh wrote: post #14
ankitml wrote: post #13

GitHub is not the place to write code. IDE is the place. Along with pre CI checks, some tests, coverage etc. they should get some PM before making decisions..

This is the future envisioned by Microsoft. Vibe coding all the way down, social network style. They are putting this in front of the developers as take it or leave it deal. I left the platform, doing my coding old way, hosting it somewhere else. Discoverability? I don't care. I'm coding it for myself and hosting in the open. If somebody finds it, nice. Otherwise, mneh.

As long as the resulting PR is less than 100 lines and the AI is a bit more self sufficient (like actually making sure tests pass before "pushing") it would be ok I think. I think this process is intended for fixing papercuts rather than building anything involved. It just isn't good enough yet.
Post reply on HN