Earlier quoted context omitted.
you re the hacker or the google?
The google
Can a Mod please change thread title to I'm The Google. AMA.
We hacked Gemini's Python sandbox and leaked its source code (at least some)
111–120 of 151 posts
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#112Their "LLM bugSWAT" events, held in vibrant locales like Las Vegas, are a testament to their commitment to proactive security red teaming. I don't understand why security conferences are attracted to Vegas. In my opinion its a pretty gross place to conduct any conference.
Excluding uptight scolds is a feature not a bug. There's a lot of overlap between people who find Vegas objectionable and people who find red teaming objectionable (because why would any decent person know attacking/exploiting techniques).
The irony is that Vegas takes a dim view of those that take advantage of their gaming venues. The institutions that run it are quite aggressive when it comes to being attacked.
Anyways, security conferences such as BSides run all over the world in various cities where red teaming type activities is embraced. IMO it'd be nice to diversify from Vegas, preferably places with more scenery/greenery like Boulder or something.
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#113That's the system I work on! Please feel free to ask any questions. All opinions are my own and do not represent those of my employer.
I imagine you need to make and destroy sandboxed environments quite often. How fast does your code create a sandboxed environment? Do you make the environments on demand or do you make them preemptively so that one is ready to go the moment that it is needed? If you make them on demand, have you tested ZFS snapshots to see if it can be done even faster using zfs clone?
I use ZFS, but isn't the situation the sandbox is in totally different? Why would it be optimal?
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#114That's the system I work on! Please feel free to ask any questions. All opinions are my own and do not represent those of my employer.
This is why hacker news is so cool
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#115Earlier quoted context omitted.
The google
Question: how does it feel inside google in terms of losing their lunch to OpenAi? Losing here is very loose, I don’t think OpenAI won yet but seems to have made a leap ahead of google in terms of marker share and we know google was sitting on tons of breakthroughs and research. Any panicking or internal discontent at google’s product policies? No need to answer if you’re uncomforable that your employer may hold you…
They just released a SOTA model (Gemini 2.5 Pro) that beats all models on most benchmarks, it's a great comeback from the model side but IMO they are less strong on the product side, they pioneered the sticky ecosystem of web app products model, though kinda like the Microsoft Office suite that (originally) had to be downloaded, ironically building on XML HTTP request support the IE5 introduced for Outlook.
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#116Probably best text I've seen in AI train ride recently:
""""" As companies rush to deploy AI assistants, classifiers, and a myriad of other LLM-powered tools, a critical question remains: are we building securely ? As we highlighted last year, the rapid adoption sometimes feels like we forgot the fundamental security principles, opening the door to novel and familiar vulnerabilities alike. """"
There this case and there many other cases. I worry for copy & paste dev.
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#117That's the system I work on! Please feel free to ask any questions. All opinions are my own and do not represent those of my employer.
I imagine you need to make and destroy sandboxed environments quite often. How fast does your code create a sandboxed environment? Do you make the environments on demand or do you make them preemptively so that one is ready to go the moment that it is needed? If you make them on demand, have you tested ZFS snapshots to see if it can be done even faster using zfs clone?
Seconding this. Also curious if this is done with microkernels (I put Unikraft high on the list of tech I'd use for this kind of problem, or possibly the still-in-beta CodeSandbox SDK – and maybe E2B or Fly but didn't have as good experiences with those).
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#118Earlier quoted context omitted.
I submitted this HN link with a title that exactly matches the one on the article, but I didn't write the title on the article. AFAIK HN posts should match the title of the article they link to.
Actually the rule is designed to let you correct misleading titles: " Please use the original title, unless it is misleading or linkbait; don't editorialize. " - https://news.ycombinator.com/newsguidelines.html I've done that now ( https://news.ycombinator.com/item?id=43509103 ). I appreciate your scruples though! Because even though you would have been on the right side of HN's rules to correct a misleading (and/or…
Can you run the country too?
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#119I've been using a similar trick to scrape the visible internal source code of ChatGPT Code Interpreter into a GitHub repository for a while now: https://github.com/simonw/scrape-openai-code-interpreter It's mostly useful for tracking what Python packages are available (and what versions): https://github.com/simonw/scrape-openai-code-interpreter/blo...
Meanwhile they could just decide to publish this list in a document somewhere and keep it automatically up to date with their infra. But not, secrecy for the sake of secrecy.
What would the benefit of doing this be?
Re: We hacked Gemini's Python sandbox and leaked its source code (at least some)
#120> However, the build pipeline for compiling the sandbox binary included an automated step that adds security proto files to a binary whenever it detects that the binary might need them to enforce internal rules. In this particular case, that step wasn’t necessary, resulting in the unintended inclusion of highly confidential internal protos in the wild ! Protobufs aren't really these super secret hyper-proprietary thi…
The protos in question are related to internal authn/z so it's conceivable that having access to that structure would be valuable information to an attacker.
The protos were already available. See above.
A valuable information would be able to run those RPC calls as Principal (their root user)