Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we rooted Copilot

research.eye.security

Post reply on HN
Print view View original

131–140 of 147 posts

Re: How we rooted Copilot

#131

Post by silverliver »

selfmodruntime wrote: post #130
bravesoul2 wrote: post #123

Earlier quoted context omitted.

I bet the container was in an isolated VM too.

Every infra I ever worked in used this pattern to a degree. Many proxmox vm's in a kubernetes cluster.

I've seen people manually create a separate unprivileged user on the host for each VM they run, so for them the pattern becomes:

1. VM running on hypervisor as unprivileged host user

2. Container running in VM as unprivileged vm user

3. Payload running in container as unprivileged container user.

Not sure whether layered isolation is worth the increased attack surface. For normal users (not targets of state actors), it probably is.

Re: How we rooted Copilot

#132

Post by dudeinjapan »

chrz wrote: post #121
dudeinjapan wrote: post #115

Earlier quoted context omitted.

We should give all the credit to the Product Manager because he told the engineers to make it secure.

Lets send a thank you letter to Bill Gates

I presume you mean Bill Gates Sr. because he fathered Bill Gates.

Re: How we rooted Copilot

#133

Post by stogot »

kenjackson wrote: post #95
stogot wrote: post #91

Earlier quoted context omitted.

I would give the one engineer the credit for doing things better, not Microsoft. Microsoft overall culture of security is terrible. Look at the CISA report.

What CISA report?

This one: https://www.cisa.gov/sites/default/files/2025-03/CSRBReviewO...

Re: How we rooted Copilot

#134

Post by baxtr »

bravesoul2 wrote: post #125
baxtr wrote: post #6

Earlier quoted context omitted.

In my humble experience company secrets are mostly useless for other companies.

Except when they aren't. Defence in depth and zero trust and short expiry makes them way less useful for sure. Startups are probably most vulnerable as they are likely to use more "pet" techniques for infra, like SSH open to any IP to make changes.

Can you provide an example of a revealed secret that had a significant financial impact on a company?

Re: How we rooted Copilot

#135

Post by blincoln »

blastonico wrote: post #94
MattGaiser wrote: post #84

Earlier quoted context omitted.

This is very out of date. They now often trigger tooling and return the outputs of the tooling.

Not really. You're referring to agents, but the model doesn't always require agents, and the public chatbot is not connected to a shell freely evaluating arbitrary commands.

> and the public chatbot is not connected to a shell freely evaluating arbitrary commands.

[ citation needed ]

Re: How we rooted Copilot

#136

Post by username135 »

afro88 wrote: post #81

It's crazy to me that someone can write a post called "How We Rooted Copilot" when in reality they got root in an ephemeral python sandbox container that was locked down so much that they couldn't do anything. I read "rooted copilot" and I think they got root on a vm that is core to copilot itself. A much more accurate title would be "How We Rooted the Copilot Python Sandbox"

Agreed. It feels like Im seeing more of this lately

Re: How we rooted Copilot

#137

Post by immibis »

stevage wrote: post #104
wizzwizz4 wrote: post #101

Earlier quoted context omitted.

They're not. It's better to think of Copilot as a collaborative storytelling session with a text autocomplete system, which some other program is rudely hijacking to insert the result of running certain commands. Sometimes the (completion randomly selected from the outputs of the) predictive text model goes "yes, and". Other times, it goes "no, because". As observed in the article, if it's autocompleting the result o…

funny how it sounds kind of the opposite of how people might work. Get enough 'no's from someone and they might finally cave in. get enough 'yes'es and they might get sick of doing everything you ask.

It's narrowing down the space of all possible conversations. One with a lot of nos is probably a conversation with someone who says no a lot. An early LLM result was that you got higher-quality translations if you demarcated the answer with "the expert French translator says:" instead of just "French translation:"

Re: How we rooted Copilot

#138

Post by brookst »

bravesoul2 wrote: post #124
stingraycharles wrote: post #111

Earlier quoted context omitted.

“how we escalated privileges from a regular user to a root user in a completely locked sandbox” pretty much sums it up. it’s a nothing burger, which actually goes to show just how effective sandboxing is for defense in depth.

Also shows you how shit an LLM is for defence, as it actively helps you look for exploits. LLM is like an insane quadruple agent and you dont know whose side it is on (if any at all)

LLMs are on “sides” the same way books are: not at all. Tools don’t have agency.

Re: How we rooted Copilot

#139

Post by brookst »

blastonico wrote: post #94
MattGaiser wrote: post #84

Earlier quoted context omitted.

This is very out of date. They now often trigger tooling and return the outputs of the tooling.

Not really. You're referring to agents, but the model doesn't always require agents, and the public chatbot is not connected to a shell freely evaluating arbitrary commands.

This is false.

Modern chatbots use tool orchestration where the LLM is given a list of tools it can use. Tools include things like searching the web and, yes, executing code. The LLM can generate arbitrary code (typically Python) and call a tool to execute it and return results. The tool takes the Python input, launches it (in a container in this example) and returns the output, which the LLM can further process.

Re: How we rooted Copilot

#140

Post by brookst »

baxtr wrote: post #134
bravesoul2 wrote: post #125

Earlier quoted context omitted.

Except when they aren't. Defence in depth and zero trust and short expiry makes them way less useful for sure. Startups are probably most vulnerable as they are likely to use more "pet" techniques for infra, like SSH open to any IP to make changes.

Can you provide an example of a revealed secret that had a significant financial impact on a company?

The Panama Papers killed one company and cost others hundreds of millions. The Uber hack. Cambridge Analytica.
Post reply on HN