Live data from Hacker News

Zulip Cloud security vulnerability with reusable invitation links

blog.zulip.com

21–30 of 38 posts

Re: Zulip Cloud security vulnerability with reusable invitation links

#22

Is anyone using Zulip and has it made a difference to Slack for example?

I liked it a lot, but I'm not an Apple user, and apparently the iOS app was (is?) not great. For me, it's like a vastly better version of NNTP, while slack is more of a slightly improved IRC.

Re: Zulip Cloud security vulnerability with reusable invitation links

#23
post #20

Earlier quoted context omitted.

35 gigabytes?! How many people are using that thing?

Only about ~40 users, so it's definitely an issue. Haven't investigated yet but this reminded me that we should. (edit: actually it's now down to 17 gigs)

Please report this in https://zulip.com/development-community/; this certainly sounds like a memory leak. We aren't aware of any other reports of memory leaks in Zulip in the last couple years, so we'd be very happy to help track down what's happening on your server.

https://zulip.readthedocs.io/en/latest/production/requiremen... is a useful reference for folks interested in memory requirements for a self-hosted Zulip server.

Re: Zulip Cloud security vulnerability with reusable invitation links

#24

Is anyone using Zulip and has it made a difference to Slack for example?

We used Zulip at $job[-1] and it was awfully painful. I’ve used nearly every chat client invented since the BBS days, and Zulip was one of my least favorite. We’re using Slack now at $job and I vastly prefer it (even though I still call it “IRC”). The biggest selling points of Zulip, from the POV of the people who put it in place and truly loved it, were that it’s free/OS software, it has explicit threads as a first…

Hi! I lead the Zulip project.

I'm sorry to hear you had a bad experience with Zulip. If you can share details on the UX problems you had, perhaps in https://zulip.com/development-community/ where we can have an extended conversation, I'd be very appreciative. I personally review every issue reported in zulip, and some of the specific problems you cite are not unfamiliar and surprising to me. Specifically:

* I'm not aware of a plausible explanation for Zulip frequently doing full reloads of the web view. Zulip's core design is to live-update everything [1] and there's only a handful of code paths that can trigger a full reload (the most common being when the server version was updated, and in that code path, the client has an algorithm where it waits up to 30 minutes for a moment when your window is idle and not focused to reload to get the updated version, to minimize the chance of disruption). * I'm also not sure what what state might be forgotten between web client restarts; almost all of our client UI updates happen via the client asking the server to change something, and the client updating via the same server-client push mechanism that updates other clients (the main exception to this is sending messages, where local echo is important for UX reasons). And at least the server-initiated reloads are designed to preserve your precise scroll position, compose box state, etc., and that's been true since ~2013. (Though we did fix a couple bugs where compose box state was not properly preserved in the last year).

I know you're not using Zulip currently, so your memory may be imprecise, but any additional detail that could help us reproduce what you experienced would be awesome!

[1] https://zulip.readthedocs.io/en/latest/subsystems/events-sys...

Re: Zulip Cloud security vulnerability with reusable invitation links

#25
post #9

Zulip in Docker¶ Zulip has an officially supported, experimental docker image. Please note that Zulip’s normal installer has been extremely reliable for years, whereas the Docker image is new and has rough edges, so we recommend the normal installer unless you have a specific reason to prefer Docker. [0] A server application without Helm charts? Stateful apps without Kubernetes Operator? For someone living with k8s a…

[deleted]

Re: Zulip Cloud security vulnerability with reusable invitation links

#26
No team is capable of writing complex software without eventually introducing security bugs, and we (Zulip) are no exception. I'm disappointed that we introduced this bug, and would like to apologize to our users.

As a bit of commentary, my experience is that most SaaS vendors fix security bugs quietly. Others will do an brief internal investigation and only consider doing a public disclosure if they believe data privacy regulations require them to do so, because they discovered that the vulnerability had been in fact exploited. Usually public disclosure decisions are made by business executives with PR considerations in mind, so that shouldn't be surprising. See, for example, https://www.zdnet.com/article/slack-resets-passwords-for-1-o....

Zulip's core values include transparency, and our policy is to publicly disclose all security bugs that we discover with a formal CVE number, and this blog post is part of that practice. As noted in the blog post, we did an extensive audit and emailed every customer who our audit logs could not prove was not affected with enough detail to let them audit the situation. We did that despite our audit finding no evidence that this vulnerability was ever intentionally exploited, because we think it's the right thing to do.

Some technical detail I can add for those interested is that our postmortem plans following this incident include further investment in the internal audit log system used for the investigation (`RealmAuditLog`, if any readers feel like grepping our codebase). Improvements we made in mid-2020 were very helpful in investigating whether this bug had in fact been exploited, and we take this as a sign that we should increase investment in that system.

Re: Zulip Cloud security vulnerability with reusable invitation links

#27
post #26

No team is capable of writing complex software without eventually introducing security bugs, and we (Zulip) are no exception. I'm disappointed that we introduced this bug, and would like to apologize to our users. As a bit of commentary, my experience is that most SaaS vendors fix security bugs quietly. Others will do an brief internal investigation and only consider doing a public disclosure if they believe data pri…

As a side note, I'm surprised that this specific security disclosure ended up on the Hacker News homepage, while our security release about an RCE issue affecting self-hosted Zulip servers last month did not (https://blog.zulip.com/2022/01/25/zulip-server-4-9-security-...).

The core bug there an insecure secret generation algorithm in upstream Erlang/RabbitMQ, which seems more broadly notable to me!

Despite our spending many hours composing detailed reports and lobbying security teams to get this problem fixed for everyone, not just Zulip, `apt install rabbitmq-server` continues to make a Debian/Ubuntu system vulnerable to RCE, and this fact has apparently been public for years.

We did manage to get `rabbitmq-server` fixed in Debian testing, and it is scheduled to be included in the next Debian stable release, I would really have thought an RCE issue like this would result in an immediate security advisory.

As a final remark, if you self-host Zulip, please subscribe to our announcements mailing list (https://groups.google.com/g/zulip-announce) and make sure your installation is up to date, so that you benefit from the hard work our security team does to keep Zulip users safe.

Re: Zulip Cloud security vulnerability with reusable invitation links

#28
post #9

Zulip in Docker¶ Zulip has an officially supported, experimental docker image. Please note that Zulip’s normal installer has been extremely reliable for years, whereas the Docker image is new and has rough edges, so we recommend the normal installer unless you have a specific reason to prefer Docker. [0] A server application without Helm charts? Stateful apps without Kubernetes Operator? For someone living with k8s a…

I couldn't agree more. I'm unsure why it appears that docker is a such an afterthought.

Re: Zulip Cloud security vulnerability with reusable invitation links

#29

Is anyone using Zulip and has it made a difference to Slack for example?

Sort of a weird context to ask for Zulip recommendations, but sure. I find Slack to be a miserably unproductive experience that makes it impossible to keep up with a team (or teams) of any significant size. Slack is good for recreation, not productivity. If the primary purpose of your organization is to be a social group, then by all means use Slack. Meanwhile, Zulip does what I need and does it well. It scales excellently to large organizations discussing broad arrays of topics. It gets out of my way and I don't find myself thinking about it. In that way it's the perfect tool, because it lets me spend energy focusing on engaging with the conversations rather than wrestling with the interface. I guess I'd describe it as natural and unobtrusive, the sort of thing that does its job so well that you only notice that it's there when something goes wrong and you realize what you've been taking for granted.

Note that my experience is almost exclusively with the web client. My small amount of experience with the mobile app is that it's much less polished.

Re: Zulip Cloud security vulnerability with reusable invitation links

#30
post #27
post #26

No team is capable of writing complex software without eventually introducing security bugs, and we (Zulip) are no exception. I'm disappointed that we introduced this bug, and would like to apologize to our users. As a bit of commentary, my experience is that most SaaS vendors fix security bugs quietly. Others will do an brief internal investigation and only consider doing a public disclosure if they believe data pri…

As a side note, I'm surprised that this specific security disclosure ended up on the Hacker News homepage, while our security release about an RCE issue affecting self-hosted Zulip servers last month did not ( https://blog.zulip.com/2022/01/25/zulip-server-4-9-security-... ). The core bug there an insecure secret generation algorithm in upstream Erlang/RabbitMQ, which seems more broadly notable to me! Despite our spe…

The blog post tries to paint the RabbitMQ cookie generation as a security issue, but the Erlang distribution cookie is NOT a security feature. Indeed, the Erlang documentation states:

> "Security" here does not mean cryptographically secure, but rather security against accidental misuse, such as preventing a node from connecting to a cluster with which it is not intended to communicate.

The purpose of the cookie is to guard against _accidental misuse_, such as a user connecting to the wrong node (prod vs testing?). The cookie is not meant to secure a system against unauthorised use. Any system relying on the cookie for security should already be considered broken. Thus I can see why there has not been such hurry to "fix" the issue.

Post reply on HN