Building a Scalable Postgres Metrics Backend Using the Citus Extension
1–10 of 12 posts
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#2For anyone with questions about how to properly shard a table, how to fit a problem to a distribution strategy, or general questions about deployment, Citus has a Slack channel which is active and extremely welcoming. You can request an invite here: https://slack.citusdata.com/
Disclaimer: don't work there, but on the Slack and the Citus team is nothing short of awesome.
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#3We[0] just recently moved from Hbase to Postgres w/ Citus. Still not fully settled in, but got incredible support from the Citus team (seriously amazing guys) and things are looking much brighter than before.
On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries).
If you have a lot of data and know Postgres, give it a chance.
[0] https://pexe.so
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#4If you haven't already seen it, Dan from Heap Analytics has a great talk about how they use Postgres and Citus.
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#5We[0] just recently moved from Hbase to Postgres w/ Citus. Still not fully settled in, but got incredible support from the Citus team (seriously amazing guys) and things are looking much brighter than before. On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries). If you have a lot of data a…
Interesting product - good luck :)
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#6We[0] just recently moved from Hbase to Postgres w/ Citus. Still not fully settled in, but got incredible support from the Citus team (seriously amazing guys) and things are looking much brighter than before. On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries). If you have a lot of data a…
This is a super cool product. Good job.
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#7If you're looking for a metrics backend built on traditional Postgres, check out Tgres.
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#8We[0] just recently moved from Hbase to Postgres w/ Citus. Still not fully settled in, but got incredible support from the Citus team (seriously amazing guys) and things are looking much brighter than before. On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries). If you have a lot of data a…
> On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries).
Is that 20K separate transactions or are some of the 20K inserts grouped together?
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#9If you're looking for a metrics backend built on traditional Postgres, check out Tgres. http://grisha.org/blog/2016/07/29/state-of-tgres-2016/
+1 for tgres
https://github.com/tgres/tgres
On an aside, I found his posts on holt-winters very informative.
Re: Building a Scalable Postgres Metrics Backend Using the Citus Extension
#10We[0] just recently moved from Hbase to Postgres w/ Citus. Still not fully settled in, but got incredible support from the Citus team (seriously amazing guys) and things are looking much brighter than before. On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries). If you have a lot of data a…
> On a 8 worker nodes (16 cores, 80GB ram, 3TB SSD), we're at around 20k inserts/second, while most of the selects are running at around 100ms (besides some counts and complex queries). Is that 20K separate transactions or are some of the 20K inserts grouped together?
Separate (going through the master).