Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Trusted access for the next era of cyber defense

openai.com

Post reply on HN
Print view View original

71–79 of 79 posts

Re: Trusted access for the next era of cyber defense

#71

Post by spacebacon »

spacebacon wrote: post #68
ACCount37 wrote: post #67

Earlier quoted context omitted.

Bold fucking claims for a "paper" that: makes an LLM with an awkward architectural tumor, and proves that it doesn't completely die on a purely synthetic task. Further than most "AI psychosis" papers go, but still not in any way far. And "makes these treasured black boxes irrelevant"? With wild claims like this, either demo a generational improvement on a live model or GTFO.

I’ve been here over a decade longer than you sport. No need to bully people out when you are only 8 months in. I will be updating here when the model is live. Expect no further engagement.

[dead]

Re: Trusted access for the next era of cyber defense

#72

Post by Kye »

gavinray wrote: post #31
hoss1474489 wrote: post #30

Earlier quoted context omitted.

I see a Security button in the what’s new box in the Codex section of the ChatGPT website. It appears to allow me to run vulnerability scans against my connected GitHub repositories. Direct link: https://chatgpt.com/codex/cloud/security

I also have access to this but can't be certain if it was there before or not. Anyone else who hasn't verified able to access?

404 for me

Re: Trusted access for the next era of cyber defense

#73

Post by TeMPOraL »

Melatonic wrote: post #66
TeMPOraL wrote: post #62

Earlier quoted context omitted.

"Solve things" or actually do something useful, pick one. If anything, maybe the security community can finally be arsed to consider ad-hoc delegation of authority as a core concept and a basic use case, because that's arguably the primary source of persistent user-level security issues in computing. In real life, it's absolutely normal to ask random people on the fly to do something in your name, with your credentia…

Are those really things people do all the time ? Not sure I would trust any kid with my credit card

I didn't say any kid, but rather one's own kid - whether they can or cannot be trusted is mostly a parenting issue, so different people have different experiences.

But yes, me and my siblings would often do grocery runs for our mom, with her card in hands, and I also think nothing of handing my own card to my wife (who already knows the PIN), or mine or her siblings, or even some acquaintances, because I trust them.

The larger point (even larger than my previous comment) is that, contrary to what cybersecurity (and especially cryptocurrency aficionados) community believes, the real world runs on trust. Trust is not a bug, it's a feature - an optimization that makes societies and civilizations scale. Trust has its own limits and structural complexities, it has its ebbs and flows, but it's absolutely vital and systems that do not support it (or try to eliminate it) simply gets worked around. Not out of spite, but out of necessity - otherwise nothing would ever get done.

Re: Trusted access for the next era of cyber defense

#76

Post by mikewarot »

TeMPOraL wrote: post #62
mikewarot wrote: post #55

It's important to keep perspective, the holes that everyone (including LLMs now) keep finding in pretty much everything are mostly the fault of running things with ambient authority, instead of using systems based on default deny, and capabilities. I used to think we were 20 years away from a shift to Capabilities based Operating Systems, which were ----> this Unfortunately, I think we're about to repeat history, and…

"Solve things" or actually do something useful, pick one. If anything, maybe the security community can finally be arsed to consider ad-hoc delegation of authority as a core concept and a basic use case, because that's arguably the primary source of persistent user-level security issues in computing. In real life, it's absolutely normal to ask random people on the fly to do something in your name, with your credentia…

In the 1970s my Mom would give me a $20 capability token to exchange in a store for cigarettes, in her name, while she waited withy sisters in the car. I was about 15 at the time, and it was a normal part of life. She never gave me a signed blank check.

Giving $20 to an AI is far safer than giving it your credit card. The effects are limited to $20 of loss.

Re: Trusted access for the next era of cyber defense

#77

Post by TeMPOraL »

mikewarot wrote: post #76
TeMPOraL wrote: post #62

Earlier quoted context omitted.

"Solve things" or actually do something useful, pick one. If anything, maybe the security community can finally be arsed to consider ad-hoc delegation of authority as a core concept and a basic use case, because that's arguably the primary source of persistent user-level security issues in computing. In real life, it's absolutely normal to ask random people on the fly to do something in your name, with your credentia…

In the 1970s my Mom would give me a $20 capability token to exchange in a store for cigarettes, in her name, while she waited withy sisters in the car. I was about 15 at the time, and it was a normal part of life. She never gave me a signed blank check. Giving $20 to an AI is far safer than giving it your credit card. The effects are limited to $20 of loss.

Right. But there's almost no software that supports the equivalent of limited capability tokens, much less casually handing them around. In contrast, in real life, it's a common use case, and we don't usually even bother with capability limits, because it's too much hassle - we rely on trust (part of which is persistent relationship that continues beyond current interaction) + spatial proximity and temporal limits.

I.e. even if your mom handed you her credit card, she was still there in a car nearby (spatial proximity), and was waiting for you there (temporal limit), and she was your mom (persistent trust-based relationship), which is sufficient protection from the risk of you running away and spending her money on hookers.

(How you managed to buy cigarettes as a 15yo is beyond me - or maybe there were no age checks in 1970s yet?)

Coming back: in real life, we don't bother with restricting the access tool, everyone is transiently giving much more access than they need to random things, and expect them to not abuse it. Meanwhile, cybersecurity is mostly stuck in the mindset of passwords being your identity, and being like underwear (change frequently, don't share), and the concept of delegation of authority doesn't exist beyond some enterprise systems. Which is why, in real world, everyone says "fuck it" and just shares passwords as needed.

Re: Trusted access for the next era of cyber defense

#78

Post by mikewarot »

NoahZuniga wrote: post #64
mikewarot wrote: post #55

It's important to keep perspective, the holes that everyone (including LLMs now) keep finding in pretty much everything are mostly the fault of running things with ambient authority, instead of using systems based on default deny, and capabilities. I used to think we were 20 years away from a shift to Capabilities based Operating Systems, which were ----> this Unfortunately, I think we're about to repeat history, and…

Many (maybe even most bugs) the ais are finding are memory safety errors, which is pretty clearly not "the fault of running things with ambient authority". The data is treated as untrusted, but due to a mistake can still do something it shouldn't.

The thing is, if you're only allowed to access the actual files the user selected (at runtime) those are the only things that could possibly be corrupted. A memory error in any given app doesn't set up for a system compromise in a capabilities based system.
Post reply on HN