Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Scaling to users requires Synapse Pro

element.io

Post reply on HN
Print view View original

11–17 of 17 posts

Re: Scaling to users requires Synapse Pro

#11

Post by Arathorn »

jauntywundrkind wrote: post #10

InfluxDB 3 just went alpha, and they similarly have very severe limitations on what their open-core product will do, very strongly drive folks to upgrade to enterprise. https://www.influxdata.com/blog/influxdb3-open-source-public... https://news.ycombinator.com/item?id=42684524 https://news.ycombinator.com/item?id=42703113 While I admit that I don't think losing open source developers is actually that big a harm to m…

woah there - hang on a sec

> No one's going to be happy running a 500x slower python project knowing there's the real deal running elsewhere, with a hip new runtime they can't get.

What if the current python project got 500x faster in general? As optimisation work for Synapse is not being paywalled - it’s just the worker scalability, which is not a bottleneck for normal sized servers anyway.

The reason Matrix servers are typically slow today is that state resolution and storage is algorithmically slow; federation is fullmesh and doesn’t support “thin server” approaches for participating in busy rooms, and joining big rooms still blocks on loads of state being synchronised before you can see other members & history.

Fixing this (and more) is very much on the menu for FOSS Synapse - and won’t be helped by faster workers, given workers are just for scalability, not for core performance. Conversely, $ from Synapse Pro will hopefully fund that work, which otherwise has been stuck for years now thanks to lack of $.

(Also: if you did decide Element had gone mad and don’t want anything to do with Synapse, you can try a different homeserver like one of the Conduit forks; don’t throw Matrix under the bus with Element :)

Re: Scaling to users requires Synapse Pro

#13

Post by jdenning »

Arathorn wrote: post #9
traspler wrote: post #5

So there is the Synapse version that is required to get the big money contracts and the Synapse that is pretty inconsequential to the business except as a testing playground. - Makes it difficult to see how "Element is fully committed to community Synapse" can be true in the long run. Does this not spread the development effort & focus even thinner across these projects? And would reduced memory & cpu requirements no…

It looks like i've done a bad job at explaining this, so i'll try to clarify: Making worker processes go fast has most benefit to enormous deployments, as they won't run out of headroom when running lots of single-core python workers. However, ALL deployment sizes benefit from algorithmic improvements to the protocol and its implementation - which are the cause of smaller servers being slower today. Specifically: * M…

But it seems like this move incentivizes you to not improve the open source server, at least such that the open source server is always inferior to the pro version.

If someone makes a new, more performant, open-source server, and it touches your bottom line then you're strongly motivated to "embrace, extend, extinguish".

The thing is, we've all heard this before, and it always ends up the same. I hope you prove me wrong, but I wouldn't bet on it.

Re: Scaling to users requires Synapse Pro

#15

Post by Arathorn »

jdenning wrote: post #13
Arathorn wrote: post #9

Earlier quoted context omitted.

It looks like i've done a bad job at explaining this, so i'll try to clarify: Making worker processes go fast has most benefit to enormous deployments, as they won't run out of headroom when running lots of single-core python workers. However, ALL deployment sizes benefit from algorithmic improvements to the protocol and its implementation - which are the cause of smaller servers being slower today. Specifically: * M…

But it seems like this move incentivizes you to not improve the open source server, at least such that the open source server is always inferior to the pro version. If someone makes a new, more performant, open-source server, and it touches your bottom line then you're strongly motivated to "embrace, extend, extinguish". The thing is, we've all heard this before, and it always ends up the same. I hope you prove me wr…

> But it seems like this move incentivizes you to not improve the open source server, at least such that the open source server is always inferior to the pro version.

The idea is that we absolutely improve FOSS synapse in all ways - other than supporting enormous deployments. For instance we continue to land perf improvements to FOSS synapse and make average sized servers as snappy as conceivably possible. And all features land in FOSS synapse, etc. If we don’t it would harm the public Matrix network and we obviously don’t want that.

> If someone makes a new, more performant, open-source server, and it touches your bottom line then you're strongly motivated to "embrace, extend, extinguish".

Rather than EEE, I’d expect us to simply compete with that server - adding more features, better perf, better commercial support, etc. For Matrix’s sake, I hope that we end up in that situation tbh.

> The thing is, we've all heard this before, and it always ends up the same. I hope you prove me wrong, but I wouldn't bet on it.

I think the difference is that typically folks doing this are being greedy to grow a profitable (or could-be-profitable) company as aggressively as possible. Whereas here the motive is simply to pay for our FOSS dev and get to breakeven and be able to sustainably grow Matrix for the benefit of the whole network. If in the end a bit of proprietary software is the necessary evil to get there, sobeit.

Of course this could change in future, eg if mgt changed, but that’s true of anything. But the intention is categorically not to EEE (and on the Matrix Foundation side, the governance and spec process is set up to stop Element from being able to EEE even if it wanted to).

Re: Scaling to users requires Synapse Pro

#16

Post by pauldix »

jauntywundrkind wrote: post #10

InfluxDB 3 just went alpha, and they similarly have very severe limitations on what their open-core product will do, very strongly drive folks to upgrade to enterprise. https://www.influxdata.com/blog/influxdb3-open-source-public... https://news.ycombinator.com/item?id=42684524 https://news.ycombinator.com/item?id=42703113 While I admit that I don't think losing open source developers is actually that big a harm to m…

Our intention with InfluxDB Core is that it's useful to large audience. Just not the group of people seeking a historical TSDB. It's a collector, processor, and recent data TSDB. If you're familiar with the TICK stack from our 1.x line, it's like Telegraf (the data collector), Kapacitor (the processor and monitoring agent), and an InfluxDB that is better on the most recent data.

The InfluxDB part of it is more narrowly scoped than previous versions, but the Telegraf and Kapacitor parts are much more feature rich than those previous products.

Re: Scaling to users requires Synapse Pro

#17

Post by traspler »

Arathorn wrote: post #9
traspler wrote: post #5

So there is the Synapse version that is required to get the big money contracts and the Synapse that is pretty inconsequential to the business except as a testing playground. - Makes it difficult to see how "Element is fully committed to community Synapse" can be true in the long run. Does this not spread the development effort & focus even thinner across these projects? And would reduced memory & cpu requirements no…

It looks like i've done a bad job at explaining this, so i'll try to clarify: Making worker processes go fast has most benefit to enormous deployments, as they won't run out of headroom when running lots of single-core python workers. However, ALL deployment sizes benefit from algorithmic improvements to the protocol and its implementation - which are the cause of smaller servers being slower today. Specifically: * M…

After re-reading and reading many other comments in various posts I think I have misunderstood it partly. So the workers part has been overhauled in Rust for Synapse Pro. Is this a complete re-implementation of Synapse so the big deployments with Synapse Pro will not run anything from the normal Synapse or is this only part of the whole stack and the Pro deployments will also run most of the normal Synapse?
Post reply on HN