Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we rooted Copilot

research.eye.security

Post reply on HN
Print view View original

71–80 of 147 posts

Re: How we rooted Copilot

#71

Post by blastonico »

blastonico wrote: post #70

How does he know that the response isn't just hallucinations? I'm telling it because I work there and I don't recognize any of those processes. In fact I found one script named keepAliveJupyterSvc.sh in a public repo: https://github.com/shivamkm07/code-interpreter/blob/load-tes...

Oh boy, this really seems to be hallucination.

Guys, chatbots are mostly token generators, they don't run programs e give you responses...it's not a simple shell program, it computes things in GPU and return tokens, in which are translated back to English.

Re: How we rooted Copilot

#72

Post by whazor »

bramhaag wrote: post #44
n2d4 wrote: post #10

Earlier quoted context omitted.

The important part: > Now what have we gained with root access to the container? > Absolutely nothing! > We can now use this access to explore parts of the container that were previously inaccessible to us. We explored the filesystem, but there were no files in /root, no interesting logging to find, and a container breakout looked out of the question as every possible known breakout had been patched. I'm sure there a…

> a container breakout looked out of the question as every possible known breakout had been patched This is the part that concerns me. It only encourages an attacker to sit on an exploit like this until a new container breakout is discovered.

I expect that they run their containers more isolated as virtual machines. So they have bigger problems of there is a breakout possible.

Re: How we rooted Copilot

#73

Post by bgwalter »

eastbound wrote: post #65
wkat4242 wrote: post #62

Earlier quoted context omitted.

Meh it depends whether you use those things of course. There's other IDEs, other languages. And Microsoft isn't doing this out of charity. A lot of the really useful plugins are not working on the open source version, so people that use them provide telemetry which is probably valuable. Or they use it as a gateway to their services like GitHub Copilot. If a mega corporation gives you something for free it's always mo…

So, no OSS contribution is valid unless you are using this very library? Did Microsoft contribute more to the OSS world, or did the OSS world contribute more to Microsoft? I pardon Microsoft because they have donated Typescript, which is a true civilizational progress. You could say the OSS world has contributed to Microsoft because they’ve given them a real OS, which they didn’t have inner expertise to develop. We’r…

The open source ecosystem was perfect before Microsoft tried to meddle, assimilate and destroy.

Microsoft has destroyed several open source projects by infiltrating them with mediocre MSFT employees.

Microsoft bought the GitHub monopoly in order to control open source further. Microsoft then stole and violated the copyright by training "AI" on the GitHub open source.

Microsoft finances influential open source organizations like OSI in order to make them more compliant and business friendly.

The useful projects are tiny compared to the entire open source stack. Paying for NPM repositories is a goodwill gesture and another power grab.

Re: How we rooted Copilot

#74

Post by Wilder7977 »

CharlesW wrote: post #41
blendergeek wrote: post #39

Earlier quoted context omitted.

Free software and open source are two ideologies for the same thing. Free Software is the ideology of developing the software for the benefit of mankind (it's sometimes termed a "political" stance but I see it as an ethical stance). Open source is the ideology of saving money at a corporation by not paying the developers. Sure open source can benefit mankind but will only develop corporate software for money. When de…

I applaud your choice! I just can't think of any free software examples that don't also benefit corporations.

Lichess, the gazillion of self-hosting software. There are many examples of free software that are exclusively (or let's say predominantly) used in noncommercial environments.

In any case, I agree with the commenter, and I think that developing a software which is also used by companies is different from looking for vulnerabilities in the context and scope of a bug bounty program for a specific company. Yes, you could argue that users of said company are going to be more secure, but it's evidence t like even in this case the company is the direct beneficiary.

Re: How we rooted Copilot

#75

Post by qbit42 »

MattGaiser wrote: post #33
citizenpaul wrote: post #21

Earlier quoted context omitted.

I'll never understand why people do free dev work for multinational trillion dollar conglomerates.

It mostly pays in career benefits. Same reason why plenty intern for free.

Who is interning for free as a software engineer?

Re: How we rooted Copilot

#76

Post by bgwalter »

tptacek wrote: post #63
bramhaag wrote: post #44

Earlier quoted context omitted.

> a container breakout looked out of the question as every possible known breakout had been patched This is the part that concerns me. It only encourages an attacker to sit on an exploit like this until a new container breakout is discovered.

Are you not concerned about all the other platforms that rely on containers as security boundaries between tenants? There are a lot of them.

It is hard to answer that since the stack is so convoluted. Some parts are forced on the user. Copilot is built into Microsoft Office workplace applications.

If you break out of a container, do you have access to the same system that serves these applications? Who knows, it looks like a gigantic mess.

Re: How we rooted Copilot

#77

Post by Barbing »

simonw wrote: post #9
varispeed wrote: post #2

Earlier LLMs used to be a goldmine for company secrets (when it learned documents that shouldn't be on public internet). Most of it seem to be scrubbed now.

Do you have any concrete examples of this? I have not seen any myself.

I looked for an alleged case of an LLM apparently reproducing email signatures—but couldn’t find it exactly, and of course many email signatures have been published over the years, especially on newsgroups. (Maybe it was conspiratorial kind of thinking from web commenters assuming ChatGPT was training on emails users were feeding it, which as mentioned certainly doesn’t need to be the case.)

Something like the top screenshot here, though:

https://www.zdnet.com/article/chatgpt-can-leak-source-data-v...

(not parent commenter but) tl;dr no

Re: How we rooted Copilot

#78

Post by 0xbadcafebee »

simonw wrote: post #8

OK, I think I understand what this is about: the vulnerability that they reported (and Microsoft fixed) is that there was a trick you could use to run your own code with root privileges inside the container - when the system was designed to have you only execute code as a non-root user. It turned out not to really matter, because the container itself was still secured - you couldn't make network requests from it and…

I have to give Microsoft props here. Most companies don't bother to lock things down well enough, but they were thorough.

Re: How we rooted Copilot

#79

Post by 0xbadcafebee »

bramhaag wrote: post #4

> We reported the vulnerability to Microsoft in April and they have since fixed it as a moderate severity vulnerability. As only important and critical vulnerabilities qualify for a bounty award, we did not receive anything, except for an acknowledgement on the Security Researcher Acknowledgments for Microsoft Online Services webpage. I guess it makes sense that a poor little indie company like Microsoft can't pay bu…

M$: If you're not going to send any money, send some swag. Make it cool and hackers will wear it, and now you have them advertising for you and possibly even want to work for you. Culture is a tool, and hackers have culture, so learn how to use it.

Re: How we rooted Copilot

#80

Post by eddythompson80 »

blastonico wrote: post #70

How does he know that the response isn't just hallucinations? I'm telling it because I work there and I don't recognize any of those processes. In fact I found one script named keepAliveJupyterSvc.sh in a public repo: https://github.com/shivamkm07/code-interpreter/blob/load-tes...

That repo, and its contributors are MS/Azure employees working on the service for running python code in a container. I don't know why it's under a personal account. Though it says it's a fork from an Office repo that I can't find.
Post reply on HN