Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How we hacked McKinsey's AI platform

codewall.ai

Post reply on HN
Print view View original

31–40 of 213 posts

Re: How we hacked McKinsey's AI platform

#31

Post by jfkimmes »

simonw wrote: post #16
joenot443 wrote: post #15

> One of those unprotected endpoints wrote user search queries to the database. The values were safely parameterised, but the JSON keys — the field names — were concatenated directly into SQL. I was expecting prompt injection, but in this case it was just good ol' fashioned SQL injection, possible only due to the naivety of the LLM which wrote McKinsey's AI platform.

Yeah, gotta admit I'm a bit disappointed here. This was a run-of-the-mill SQL injection, albeit one discovered by a vulnerability scanning LLM agent. I thought we might finally have a high profile prompt injection attack against a name-brand company we could point people to.

Not the same league as McKinsey, but I like to point to this presentation to show the effects of a (vibe coded) prompt injection vulnerability:

https://media.ccc.de/v/39c3-skynet-starter-kit-from-embodied...

> [...] we also exploit the embodied AI agent in the robots, performing prompt injection and achieve root-level remote code execution.

Re: How we hacked McKinsey's AI platform

#33

Post by frankfrank13 »

Some insider knowledge: Lilli was, at least a year ago, internal only. VPN access, SSO, all the bells and whistles, required. Not sure when that changed.

McKinsey requires hiring an external pen-testing company to launch even to a small group of coworkers.

I can forgive this kind of mistake on the part of the Lilli devs. A lot of things have to fail for an "agentic" security company to even find a public endpoint, much less start exploiting it.

That being said, the mistakes in here are brutal. Seems like close to 0 authz. Based on very outdated knowledge, my guess is a Sr. Partner pulled some strings to get Lilli to be publicly available. By that time, much/most/all of the original Lilli team had "rolled off" (gone to client projects) as McKinsey HEAVILY punishes working on internal projects.

So Lilli likely was staffed by people who couldn't get staffed elsewhere, didn't know the code, and didn't care. Internal work, for better or worse, is basically a half day.

This is a failure of McKinsey's culture around technology.

Re: How we hacked McKinsey's AI platform

#34

Post by TheDong »

simonw wrote: post #16
joenot443 wrote: post #15

> One of those unprotected endpoints wrote user search queries to the database. The values were safely parameterised, but the JSON keys — the field names — were concatenated directly into SQL. I was expecting prompt injection, but in this case it was just good ol' fashioned SQL injection, possible only due to the naivety of the LLM which wrote McKinsey's AI platform.

Yeah, gotta admit I'm a bit disappointed here. This was a run-of-the-mill SQL injection, albeit one discovered by a vulnerability scanning LLM agent. I thought we might finally have a high profile prompt injection attack against a name-brand company we could point people to.

Github actions has had a bunch of high-profile prompt injection attacks at this point, most recently the cline one: https://adnanthekhan.com/posts/clinejection/

I guess you could argue that github wasn't vulnerable in this case, but rather the author of the action, but it seems like it at least rhymes with what you're looking for.

Re: How we hacked McKinsey's AI platform

#35

Post by rzmmm »

sigmar wrote: post #21

I've got no idea who codewall is. Is there acknowledgment from McKinsey that they actually patched the issue referenced? I don't see any reference to "codewall ai" in any news article before yesterday and there's no names on the site. https://www.google.com/search?q=codewall+ai

Yeah can't find much information either. I would like to see at least some proof. Either via Mckinsey or from the security team.

Re: How we hacked McKinsey's AI platform

#36

Post by vanillameow »

codechicago277 wrote: post #18
aerhardt wrote: post #9

Earlier quoted context omitted.

The LLM that wrote this simply couldn’t help itself.

Picked up a vibe, but couldn’t confirm it until the last paragraph, but yeah clearly drafted with at least major AI help.

Can we stop softening the blow? This isn't "drafted with at least major AI help", it's just straight up AI slop writing. Let's call a spade a spade. I have yet to meet anyone claiming they "write with AI help but thoughts are my own" that had anything interesting to say. I don't particularly agree with a lot of Simon Willison's posts but his proofreading prompt should pretty much be the line on what constitutes acceptable AI use for writing.

https://simonwillison.net/guides/agentic-engineering-pattern...

Grammar check, typo check, calls you out on factual mistakes and missing links and that's it. I've used this prompt once or twice for my own blog posts and it does just what you expect. You just don't end up with writing like this post by having AI "assistance" - you end up with this type of post by asking Claude, probably the same Claude that found the vulnerability to begin with, to make the whole ass blog post. No human thought went into this. If it did, I strongly urge the authors to change their writing style asap.

"So we decided to point our autonomous offensive agent at it. No credentials. No insider knowledge. And no human-in-the-loop. Just a domain name and a dream."

Give me a fucking break

Re: How we hacked McKinsey's AI platform

#37

Post by danenania »

simonw wrote: post #16
joenot443 wrote: post #15

> One of those unprotected endpoints wrote user search queries to the database. The values were safely parameterised, but the JSON keys — the field names — were concatenated directly into SQL. I was expecting prompt injection, but in this case it was just good ol' fashioned SQL injection, possible only due to the naivety of the LLM which wrote McKinsey's AI platform.

Yeah, gotta admit I'm a bit disappointed here. This was a run-of-the-mill SQL injection, albeit one discovered by a vulnerability scanning LLM agent. I thought we might finally have a high profile prompt injection attack against a name-brand company we could point people to.

> I thought we might finally have a high profile prompt injection attack against a name-brand company we could point people to.

These folks have found a bunch: https://www.promptarmor.com/resources

But I guess you mean one that has been exploited in the wild?

Re: How we hacked McKinsey's AI platform

#38

Post by jacquesm »

causal wrote: post #29
bee_rider wrote: post #19

I don’t love the title here. Maybe this is a “me” problem, but when I see “AI agent does X,” the idea that it might be one of those molt-y agents with obfuscated ownership pops into my head. In this case, a group of pentesters used an AI agent to select McKinsey and then used the AI agent to do the pentesting. While it is conventional to attribute actions to inanimate objects (car hits pedestrians), IMO we should be…

Yah it's just an ad, and "Pentesting agents finds low-hanging vulnerability" isn't gonna drive clicks.

It's not an ad for McKinsey though.
Post reply on HN