Live data from Hacker News

How we hacked McKinsey's AI platform

codewall.ai

111–120 of 213 posts

Re: How we hacked McKinsey's AI platform

#111
post #95

Earlier quoted context omitted.

No, they don't have world class technology teams, they hire contractors to do all the tech stuff, their expertise is in management, yes that's world class.

Is it though? Managing teams to not torpedo your company with stupid stuff like this is kinda core to “good management.” The evidence would indicate they’re not very good at that either.

They were good. Not so good now.

Re: How we hacked McKinsey's AI platform

#112

Earlier quoted context omitted.

Can we stop softening the blow? This isn't "drafted with at least major AI help", it's just straight up AI slop writing. Let's call a spade a spade. I have yet to meet anyone claiming they "write with AI help but thoughts are my own" that had anything interesting to say. I don't particularly agree with a lot of Simon Willison's posts but his proofreading prompt should pretty much be the line on what constitutes accep…

Sorry but seems like most people don't care or even like AI writing more: https://x.com/kevinroose/status/2031397522590282212

That's the problem with AI writing in a nutshell. In a blind, relatively short comparison (similarly used for RLHF), AI writing has a florid, punchy quality that intuitively feels like high quality writing.

But then after you read the exact same structure a dozen times a day on the web, it becomes like nails on the chalkboard. It's a combination of "too much of a good thing" with little variation throughout a long piece of prose, and basic pattern recognition of AI output from a model coalescing to a consistent style that can be spotted as if 1-3 human ghost writers wrote 1/4 of the content on the web.

Re: How we hacked McKinsey's AI platform

#113

> One of those unprotected endpoints wrote user search queries to the database. The values were safely parameterised, but the JSON keys — the field names — were concatenated directly into SQL. I was expecting prompt injection, but in this case it was just good ol' fashioned SQL injection, possible only due to the naivety of the LLM which wrote McKinsey's AI platform.

I just wonder how much professional grade code written by LLMs, "reviewed" by devs, and commited that made similar or worse mistakes. A funny consequence of the AI boom, especially in coding, is the eventual rise in need for security researchers.

Re: How we hacked McKinsey's AI platform

#115

Earlier quoted context omitted.

> McKinsey is trying to do software like they do their other engagements. It doesn't work. I mean, it doesn't work for their consulting gigs either. There's a reason McKinsey has such a bad reputation.

But it does work for them? They make tons of money.

As an ex-consultant: consulting at that level is kind of a grift. They over-promise and under-deliver as SOP. It's ripe for AI disruption, whatever that looks like.

Re: How we hacked McKinsey's AI platform

#117

Some insider knowledge: Lilli was, at least a year ago, internal only. VPN access, SSO, all the bells and whistles, required. Not sure when that changed. McKinsey requires hiring an external pen-testing company to launch even to a small group of coworkers. I can forgive this kind of mistake on the part of the Lilli devs. A lot of things have to fail for an "agentic" security company to even find a public endpoint, mu…

Couple of things to add: McKinsey has a weird structure where there are too many cooks in the kitchen. Everybody there is reviewed on client impact, meaning it ends up being an everybody-for-themselves situation. So as a developer you have little guidance (in fact, you're still being reviewed on client impact, even if you have 0 client exposure). Then a (Senior) Partner comes in with this idea (that will get them a g…

Why would anyone work there, then, unless that's the only place they could get hired as a dev?

And if the latter is the case, then that sort of stamps the case closed from the get-go...

Re: How we hacked McKinsey's AI platform

#118
post #26

Earlier quoted context omitted.

Those short "punchy sentence" paragraphs are my new trigger: > No credentials. No insider knowledge. And no human-in-the-loop. Just a domain name and a dream. It just sounds so stupid.

It's LinkedIn speech. Two word sentences, each one on a new line.

Ah. That might be why I find it especially triggering.

Re: How we hacked McKinsey's AI platform

#119

I don’t love the title here. Maybe this is a “me” problem, but when I see “AI agent does X,” the idea that it might be one of those molt-y agents with obfuscated ownership pops into my head. In this case, a group of pentesters used an AI agent to select McKinsey and then used the AI agent to do the pentesting. While it is conventional to attribute actions to inanimate objects (car hits pedestrians), IMO we should be…

Ok, we've reverted the title (submitted title was "AI Agent Hacks McKinsey")

Re: How we hacked McKinsey's AI platform

#120
post #30

Earlier quoted context omitted.

> now that unfortunately some folks attribute agency to these agentic systems. You're doing that by calling them "agentic systems".

Unfortunately that’s what they are called. I was hoping the phrasing would highlight the problem rather than propagate it.

Eh, if you tell me that I need to do X, then I can make choices on how to accomplish X, that I am no longer an agent as a human?

You're trying to redefine long standing definitions for God knows what reason.

Post reply on HN