Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The end of a myth: Distributed transactions can scale

muratbuffalo.blogspot.com

Post reply on HN
Print view View original

51–60 of 95 posts

Re: The end of a myth: Distributed transactions can scale

#51

Post by rezonant »

mlichvar wrote: post #50
rezonant wrote: post #39

Earlier quoted context omitted.

That would be PTP then

No, NTP and PTP are two different protocols. They can both use hardware timestamps and reach single-digit nanosecond accuracy in ideal conditions. The main difference is in existing support in switches and routers, which is needed to avoid the impact of asymmetric delay between ports (typically tens of nanoseconds per switch). PTP has good support in higher-end switches and routers, but it's difficult to secure and m…

Yes, PTP and NTP are indeed very different protocols.

There's no networking hardware timestamp support for NTP because NTP has nothing to do with hardware timestamps.

PTP can be done without hardware timestamps, but it was designed with hardware support in mind.

I don't know where you got it that NTP does anything even orders of magnitude close to nanoseconds:

> NTP can usually maintain time to within tens of milliseconds over the public Internet, and can achieve better than one millisecond accuracy in local area networks under ideal conditions

https://en.m.wikipedia.org/wiki/Network_Time_Protocol

> The Precision Time Protocol (PTP) is a protocol used to synchronize clocks throughout a computer network. On a local area network, it achieves clock accuracy in the sub-microsecond range, making it suitable for measurement and control systems.

https://en.m.wikipedia.org/wiki/Precision_Time_Protocol

Literally neither solution comes anywhere near nanosecond accuracy.

For reference, there are 1,000,000 nanoseconds in a millisecond

Re: The end of a myth: Distributed transactions can scale

#52

Post by dilyevsky »

bjornsing wrote: post #13
withinboredom wrote: post #4

Earlier quoted context omitted.

Not GCP related, but I found some local (to my server) telcos and reached out to their IT department and got access to use their internal NTP servers. So my servers are now stratum 1, connected to GPS and atomic clocks, doing skew via chrony. I’m not paying a dime. I merely had to agree to keep my access low and my servers peer with each other to keep their clocks within a few nanoseconds of each other. It’s amazing…

Time synchronization over a network with unknown latency characteristics is fundamentally impossible.

Drop the “over a network with unknown latency characteristics” - it’s cleaner

Re: The end of a myth: Distributed transactions can scale

#55

Post by _448 »

jamesblonde wrote: post #17

This is impressive, but won't have huge impact, IMO. Way back in 2015, MySQL Cluster (NDB Cluster engine) benchmarked 200m transactions/second on commodity hardware [1]. It was read-committed transactions, not snapshot isolation, but still impressive. NDB (or RonDB, the new DB by its author) uses a non-blocking 2-phase commit protocol (failed transaction coordinators are failed over) and is even open-source. Still, i…

Why not many are talking about RonDB? This is like Erlang of the database world! People at Ericsson were really very smart and ahead of their time.

FoundationDB, TiKV, CitusData's Postgres extension are very well known, but RonDB looks like a hidden gem.

What are the practical issues with RonDB that it is not widely known(or used?)?

Re: The end of a myth: Distributed transactions can scale

#56

Post by mlichvar »

rezonant wrote: post #51
mlichvar wrote: post #50

Earlier quoted context omitted.

No, NTP and PTP are two different protocols. They can both use hardware timestamps and reach single-digit nanosecond accuracy in ideal conditions. The main difference is in existing support in switches and routers, which is needed to avoid the impact of asymmetric delay between ports (typically tens of nanoseconds per switch). PTP has good support in higher-end switches and routers, but it's difficult to secure and m…

Yes, PTP and NTP are indeed very different protocols. There's no networking hardware timestamp support for NTP because NTP has nothing to do with hardware timestamps. PTP can be done without hardware timestamps, but it was designed with hardware support in mind. I don't know where you got it that NTP does anything even orders of magnitude close to nanoseconds: > NTP can usually maintain time to within tens of millise…

> There's no networking hardware timestamp support for NTP because NTP has nothing to do with hardware timestamps.

Both NTP and PTP don't care (as protocols) where the timestamps are coming from. That's an implementation detail.

> NTP can usually maintain time to within tens of milliseconds over the public Internet, and can achieve better than one millisecond accuracy in local area networks under ideal conditions

That was maybe 20-30 years ago, but not today. The wikipedia article needs an update. If you don't hit a routing asymmetry, in my experience it's usually milliseconds over Internet and tens of microseconds in local network if using SW timestamping. Please note that NTP clients by default use long polling intervals to avoid excessive load on public servers on Internet, so they need to be specifically configured for better performance in local networks.

You can find some measurements with HW timestamping here: https://chrony.tuxfamily.org/examples.html

Note that this is for the system clock, which has to be synchronized over PCIe to the hardware clock of the NIC. That adds hundreds of nanoseconds of uncertainty. It doesn't matter if the hardware clock is synchronized by PTP or NTP.

If you care only about the hardware clocks, it's easy to show how accurate is the synchronization by comparing their PPS signals on a scope. NTP between two directly connected NICs, or a with a hub, can get to single-digit nanosecond accuracy. I have seen that in my testing. It's just timestamps, it doesn't matter how they are exchanged.

Re: The end of a myth: Distributed transactions can scale

#57

Post by giovannibonetti »

_448 wrote: post #55
jamesblonde wrote: post #17

This is impressive, but won't have huge impact, IMO. Way back in 2015, MySQL Cluster (NDB Cluster engine) benchmarked 200m transactions/second on commodity hardware [1]. It was read-committed transactions, not snapshot isolation, but still impressive. NDB (or RonDB, the new DB by its author) uses a non-blocking 2-phase commit protocol (failed transaction coordinators are failed over) and is even open-source. Still, i…

Why not many are talking about RonDB? This is like Erlang of the database world! People at Ericsson were really very smart and ahead of their time. FoundationDB, TiKV, CitusData's Postgres extension are very well known, but RonDB looks like a hidden gem. What are the practical issues with RonDB that it is not widely known(or used?)?

Because RonDB is not a SQL database, it is just a KV store.

Re: The end of a myth: Distributed transactions can scale

#58

Post by exabrial »

amitport wrote: post #35

IBM IMS DB Everything is direct memory access. SNA protocol, No TCP. This is used since the 70s by many banks to process high volume transactions

Everything old is new again. The MongoDB people are still convinced they "invented" NoSQL databases, but really just poorly implemented an idea that https://en.wikipedia.org/wiki/ADABAS had perfected while the MongoDB hipsters were in diapers.

Re: The end of a myth: Distributed transactions can scale

#60

Post by _448 »

giovannibonetti wrote: post #57
_448 wrote: post #55

Earlier quoted context omitted.

Why not many are talking about RonDB? This is like Erlang of the database world! People at Ericsson were really very smart and ahead of their time. FoundationDB, TiKV, CitusData's Postgres extension are very well known, but RonDB looks like a hidden gem. What are the practical issues with RonDB that it is not widely known(or used?)?

Because RonDB is not a SQL database, it is just a KV store.

Yes, but as per the document you can put a MySQL server along with the Data server node and use SQL.
Post reply on HN