Live data from Hacker News

Gitlab servers are being exploited in DDoS attacks

therecord.media

51–60 of 177 posts

Re: Gitlab servers are being exploited in DDoS attacks

#51

Seems like one solution would be to require users uploading images to remove metadata themselves before uploading. The website could reject images that contain matadata. Was Gitlab was extracting the metadata and using it for some purpose. If not, what is the reason to accept images with metadata. Perhaps they assume their customers prefer less "security" and more "convenience", instead of vice versa (less "convenien…

Theoretically the pure-perl implementation will be immune to most categories of exploits that sloppy binary parsing runs into-- buffer overflows, use after frees, and so on.

That better security goes out the window once you start using eval(), of course.

Re: Gitlab servers are being exploited in DDoS attacks

#52

Earlier quoted context omitted.

If you have to process it at all, do it in a WebAssembly sandbox on the server. Or, alternatively, in a seccomp-secured sandbox that isn't allowed to make any system calls whatsoever, just read data from one file descriptor and write processed data to another.

I've seen companies use Headless Chrome and then WebAssembly to process files. You then lock down the Headless Chrome process. You're then "triple covered"; WebAssembly's limited context, JavaScript engine's limited context, and the Chrome process boundary itself. This is obviously "expensive" though. Doesn't scale very well.

> This is obviously "expensive" though. Doesn't scale very well.

Unlike this issue then, going by the 1Tbps attack it's reportedly causing...

Re: Gitlab servers are being exploited in DDoS attacks

#53

Earlier quoted context omitted.

Article said GitLab patched back in April. Safe to say you didn’t deploy these patches? No judgment. I’m paid to make things, not apply patches. This is however why I don’t use self-hosted, pros and cons, etc.

Not him, but how are you supposed to know about the update? Do you need to check some page every day if there's an update? Why can't security updates just autoupdate like apps on phones or at least email the admin saying there is an important update.

Depending on how you have it installed you could have your package management system automatically install the updates. They aren't always perfect though, I've had at least one Gitlab update that required manually running migrations commands since the ones in the update script failed for some reason. I wouldn't trust doing it automatically.

And I'm not sure about gitlab, but their are often mailing lists for security updates for major software packages.

Re: Gitlab servers are being exploited in DDoS attacks

#54
post #41

It's somewhat refreshing that the underlying bug isn't from some C or C++ utility, but instead a Perl program using eval: https://github.com/exiftool/exiftool/blob/11.70/lib/Image/Ex... Another instance of "avoid eval as much as possible" for languages that have it.

My personal career favorite use of eval was for an import system that "unrolled" the loop that went through the columns for each row, using eval. It was much faster, but obviously a huge security risk.

Today with modern JIT compilers its probably not much faster...

Re: Gitlab servers are being exploited in DDoS attacks

#55
post #36

> GitLab Workhorse could check if the file is a valid TIFF of JPEG before passing it to ExifTool This approach doesn't work in general. An attacker could craft a polyglot file - and in that case it's a matter of which format is tried first. Valid tiff's could potentially be processed as something entirely different.

Yup. PoC||GTFO article (one of many, IIRC) on crafting a polyglot PDF / JPEG file: https://github.com/angea/pocorgtfo/blob/master/contents/arti...

[deleted]

Re: Gitlab servers are being exploited in DDoS attacks

#56
post #9
post #7

Earlier quoted context omitted.

Is that because people use hackjob dependencies to handle it more often than not?

ExifTool is hackjob? I think not. But also, file uploads should be handled in a jail or box of some type - and never let their analysis make network calls.

It's a perl program that evaluates untrusted strings it finds in user files. What exactly is your standard for "is hackjob"? It appears to be a complete piece of shit.

Re: Gitlab servers are being exploited in DDoS attacks

#57
post #54
post #41

It's somewhat refreshing that the underlying bug isn't from some C or C++ utility, but instead a Perl program using eval: https://github.com/exiftool/exiftool/blob/11.70/lib/Image/Ex... Another instance of "avoid eval as much as possible" for languages that have it.

My personal career favorite use of eval was for an import system that "unrolled" the loop that went through the columns for each row, using eval. It was much faster, but obviously a huge security risk. Today with modern JIT compilers its probably not much faster...

I saw a very similar thing with eval. There was an evaluation of a nested JSON object

    x["a"]["b"]["c"] 
And the developer decided that this was best evaluated by eval. During the code review phase I talked to them and asked why they were using eval, and they didn't know it could be evaluated directly as they were a little unclear if javascript supported that syntax.

Re: Gitlab servers are being exploited in DDoS attacks

#58

Earlier quoted context omitted.

Article said GitLab patched back in April. Safe to say you didn’t deploy these patches? No judgment. I’m paid to make things, not apply patches. This is however why I don’t use self-hosted, pros and cons, etc.

Not him, but how are you supposed to know about the update? Do you need to check some page every day if there's an update? Why can't security updates just autoupdate like apps on phones or at least email the admin saying there is an important update.

GitLab team member here.

The way to keep up-to-date on critical security updates for GitLab is to sign up for our Security Alerts mailing list: https://about.gitlab.com/company/preference-center/

Re: Gitlab servers are being exploited in DDoS attacks

#59

How can you protect yourself from file upload threats? It's basically the worst possible threat model -- executing complex user input that conforms to a spec that was written 20 years ago by some proprietary company with no security. Executing everything on an isolated container with no permissions? Audit trial etc/good logging? If someone comes up with an RCE you're basically done for, you can only mitigate it but n…

The first step is always "don't do it at all". Here is the original commit:

https://gitlab.com/gitlab-org/gitlab-workhorse/-/commit/8656...

It's hard to find a linked detailed requirement for this. I would certainly prefer if GitLab didn't silently mangle uploaded images (not least if I'm working on an EXIF library..).

Bonus points for a commit that includes the words "perl" and "exec" not also having a detailed security review attached.

Re: Gitlab servers are being exploited in DDoS attacks

#60
post #3

"..Bowling said he discovered a way to abuse how ExifTool handles uploads for DjVu file format used for scanned documents to gain control over the entire underlying GitLab web server" Ah, the good old "File upload vulnerability". File uploads remain one of the hardest problems to solve when it comes to security.

> uploads for DjVu file format used for scanned documents to gain control over the entire underlying GitLab web server A usecase for WASM's nanoprocesses (capability-based security) perhaps? Of course, until such a time someone exploits the WASM runtime itself.

RLBox is a toolkit for doing just this: https://plsyssec.github.io/rlbox_sandboxing_api/sphinx/

Used in Firefox to sandbox some libraries, including image handling IIRC.

Post reply on HN