Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Gitlab servers are being exploited in DDoS attacks

therecord.media

Post reply on HN
Print view View original

131–140 of 177 posts

Re: Gitlab servers are being exploited in DDoS attacks

#131

Post by webignition »

lemoncucumber wrote: post #127
beefield wrote: post #125

Earlier quoted context omitted.

Okay, I'll bite. I have known for a long time that eval is evil. Then, last year I actually needed to evaluate a string (from a file). As the case was safe enough (input 100% controlled by me), I did not worry too much and just used eval. But what would be a safe way to evaluate things if you needed to do that in unsafe environment? Say, you would like to make a safe website that allows user type a python code snippe…

Sure, that's basically what services like AWS Lambda do. As a starting point, you'd want to run the code in a short-lived VM with little to no network access which is dedicated to just running untrusted code.

Yes, this is what I'm currently doing with a cloud-based website automated-testing system.

In my case, code supplied by the end user is compiled into a different language such that I think I can prevent intentionally-malicious activity.

Nonetheless, spinning up a VM to create an environment in which potentially untrustworthy code is executed before then destroying the VM seems the safest option.

Re: Gitlab servers are being exploited in DDoS attacks

#132

Post by morelisp »

KronisLV wrote: post #130

I feel like this article could have been far more useful with the following points being explicitly mentioned, or at least summarized: - the problem appeared in GitLab 11.9.0 - the problem seems to have been fixed in GitLab 13.8.8 - the vulnerability uses ExifTool, so to exploit it, a user needs to be able to upload images - if an update is not (yet) possible, DjVu format file uploads can be blocked to avert this vul…

> the vulnerability uses ExifTool, so to exploit it, a user needs to be able to upload images... this vulnerability isn't relevant for GitLab instances that just have 1 user, or are not publicly accessible on the Internet

A lot of private GitLabs contain mirrors of public repositories or vendored copies of public libraries. Our GitLab is private but practically speaking there's probably several hundred people, most of whom we couldn't identify, that could "upload an image" to it.

Re: Gitlab servers are being exploited in DDoS attacks

#133

Post by jorams »

smashed wrote: post #106
Gigachad wrote: post #63

Earlier quoted context omitted.

The line existed in the 2014 commit which migrated the repo to git. It wasn't designed in the current era of mass automated abuse and internet connected everything.

That line is in a Perl metadata cleaning library. Not in Gitlab itself. Gitlab, who prides themselves and sell the gospel of improving security, willfully chose to use that library which was obviously never designed for their use case. I looked at the front-page of that library. It says it cleans metadata from a huge number of file format. Frankly it looks more like something you would use on your own, known safe, fi…

> Why does Gitlab need to clean metadata from DjVu files?

It doesn't. It needs to clean metadata from JPEG and TIFF files. They didn't properly check if the files were actually of those types, and Exiftool performed its own content type detection to end up in its DjVu code.

> Wtf are DjVu files?!

DjVu is basically an alternative to PDF.[1]

[1]: https://en.wikipedia.org/wiki/DjVu

Re: Gitlab servers are being exploited in DDoS attacks

#134

Post by lemoncucumber »

nl wrote: post #129
lemoncucumber wrote: post #127

Earlier quoted context omitted.

Sure, that's basically what services like AWS Lambda do. As a starting point, you'd want to run the code in a short-lived VM with little to no network access which is dedicated to just running untrusted code.

This is a bad solution. Lambda allows arbitraty network access and may allow access to your AWS resources. If you have to do this, the best approach is to containerise it, use capabilities to enforce restrictions and run in a virtual machine as isolated as possible. It's still not great though. Some languages (eg Java) have additional features that help with this though.

Yes, I was simplifying — the important part is keeping the untrusted code off of a trusted network. If you want to do the legwork of carefully segregating things then of course network access can work.

I didn’t mention containers since they don’t provide strong isolation and some people misuse them as though they do. There’s no harm in using them as another layer of defense, but hardware virtualization provides much better security.

Re: Gitlab servers are being exploited in DDoS attacks

#135

Post by tentacleuno »

mxrlkn wrote: post #109
zmmmmm wrote: post #100

Earlier quoted context omitted.

it was literally a pseudo "standard": https://en.wikipedia.org/wiki/JSONP

Right, but for it to be JSONP, the response should be injected into a script tag, I believe.

And it should be wrapped in a function call (most times, you can choose which function is called by a query parameter).

Re: Gitlab servers are being exploited in DDoS attacks

#136

Post by leejo »

majou wrote: post #69
Gigachad wrote: post #63

Earlier quoted context omitted.

The line existed in the 2014 commit which migrated the repo to git. It wasn't designed in the current era of mass automated abuse and internet connected everything.

2014 was very much that era, I'd accept this excuse maybe for 2007

The comment in the code slightly above says:

    this doesn't work in perl 5.6.2! grrrr
So this code is at least 20 years old, and probably pre-2000.

Re: Gitlab servers are being exploited in DDoS attacks

#137

Post by [deleted] »

dylan604 wrote: post #78
jdc wrote: post #73

Earlier quoted context omitted.

How much longer did said dev continue working there?

Fire the dev, or educate the dev. Sure, eval is just a sure sign of a lot of lack of understanding, but I'd hope for something less vile as eval would be a little more understanding that everyone learns something sometime

[deleted]

Re: Gitlab servers are being exploited in DDoS attacks

#138

Post by rndgermandude »

jeffbee wrote: post #56
djbusby wrote: post #9

Earlier quoted context omitted.

ExifTool is hackjob? I think not. But also, file uploads should be handled in a jail or box of some type - and never let their analysis make network calls.

It's a perl program that evaluates untrusted strings it finds in user files. What exactly is your standard for "is hackjob"? It appears to be a complete piece of shit.

ExifTool is the tool to extract and write and fix metadata, bar none. I haven't seen anything that even comes remotely close at how good it is at handling just jpeg metadata (exif, XMP, IPTC, MarkerNotes, and all the fine bugs every vendor has when creating these), let alone other formats too.

ExifTool is essentially for (image) metadata what ffmpeg is for video.

It isn't a hack job, but it started out as one, like so very many other things. Version 1.00 was released end of 2003, while the problematic code was added in 2008. Mind you, the problematic code does not just eval whatever it sees, it tries to make sure the input isn't dangerous first. That check failed spectacularly, defeated by a newline combined with how '$' in perl regex works without any special flags[0]. Using eval was a bad choice to begin with (but I am told something you would commonly see in perl software of the time, yes, even in 2008 still), it was the lazy choice of re-using perl to unescape C-strings instead of rolling your own unescaping code.

So what do you suggest? Use libexif[1]? Exiv2[2]? Where would I run it? Can you suggest any operating system that never had a "stupid" RCE?

>It appears to be a complete piece of shit.

Let's see your code then. All the code you ever wrote that is possibly still in use somewhere. So if you never ever fucked up or got lazy, feel free to cast the first stone, otherwise I would suggest you dial down your rhetoric when it comes to taking massive steaming piles on other people's work.

Yes, Phil Harvey had a "WTF?!"-class security bug here[4], shit happens, "goto fail", let me deRail your yaml and the Debian random number of the day is: 6.

He patched it promptly compared to other vendors and projects (public release on April 13th, while April 7th was the initial bug report, to Gitlab not ExifTool, which Gitlab then passed along[3]).

You can blame him for the bug, you can blame Gitlab for not running exiftool in some sandbox. But that half the gitlab instances remain unpatched some 7 months after patches became available, that you'll have to put on the people running these instances.

[0] https://github.com/exiftool/exiftool/commit/cf0f4e7dcd024ca9...

[1] http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=libexif

[2] http://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=exiv2

[3] https://hackerone.com/reports/1154542

[4] I cannot be sure if he wrote it, or if somebody else contributed it, but at the very least he didn't catch it during review. Looks like he wrote it, tho.

Re: Gitlab servers are being exploited in DDoS attacks

#139

Post by mjochim »

KronisLV wrote: post #130

I feel like this article could have been far more useful with the following points being explicitly mentioned, or at least summarized: - the problem appeared in GitLab 11.9.0 - the problem seems to have been fixed in GitLab 13.8.8 - the vulnerability uses ExifTool, so to exploit it, a user needs to be able to upload images - if an update is not (yet) possible, DjVu format file uploads can be blocked to avert this vul…

Caution, CVS-2021-22205 has later been found to be exploitable without authentication. No need to be "able to upload an image," unfortunately. Also no need to take the detour through a mirrored repo as sibling suggests; so long as the GitLab instance is accessible from the internet.

Re: Gitlab servers are being exploited in DDoS attacks

#140

Post by tester34 »

beefield wrote: post #125
Jach wrote: post #41

It's somewhat refreshing that the underlying bug isn't from some C or C++ utility, but instead a Perl program using eval: https://github.com/exiftool/exiftool/blob/11.70/lib/Image/Ex... Another instance of "avoid eval as much as possible" for languages that have it.

Okay, I'll bite. I have known for a long time that eval is evil. Then, last year I actually needed to evaluate a string (from a file). As the case was safe enough (input 100% controlled by me), I did not worry too much and just used eval. But what would be a safe way to evaluate things if you needed to do that in unsafe environment? Say, you would like to make a safe website that allows user type a python code snippe…

Of course, we are Google. /s

Have you tried writing your own interpreter?

Post reply on HN