Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

You can now directly sync Postgres with Redis

github.com

Post reply on HN
Print view View original

11–20 of 21 posts

Re: You can now directly sync Postgres with Redis

#11

Post by foxyv »

hangonhn wrote: post #9

Did you really mean to leave your AES key in the code in RedflyEncryptionKeys? > public const string AesKey = ... Also, the way AES is used in the code is not good practice. It seems to be using plain AES ( https://learn.microsoft.com/en-us/dotnet/api/system.security... ), which isn't meant to be used bare like that. It needs to be coupled with a digest algorithm to protect the ciphertext. Maybe use AesGcm instead?

The reason for this is that AES is not an authenticated algorithm so there is no way to determine if the ciphertext has been modified since encryption. The ciphertext could be modified/corrupted and you wouldn't know.

Also, AES is deterministic and will encrypt the same data the same way every time. This means if you are encrypting a lot of fields you will be able to do statistical attacks. Using an initialization vector with AES GCM is similar to salting a hash. This way there is no statistical method to determine the contents of the ciphertext.

Re: You can now directly sync Postgres with Redis

#12

Post by porridgeraisin »

henning wrote: post #7

It'd be nice to see documentation of how it works. Is it using a replication channel for Postgres? The write-ahead log? It looks like it does schema introspection and then works off that. How does it handle schema changes? How does this work in scenarios where you have multi-master replication? What's the compatibility with popular Postgres extensions like PostGIS? What happens if the database is rebooted or crashes?…

> Yes, I realize the source is there

It isn't actually. The codein the repo just seems to be some scaffolding.

Re: You can now directly sync Postgres with Redis

#13

Post by [deleted] »

bradleyjkemp wrote: post #8
jasonlotito wrote: post #6

Earlier quoted context omitted.

Assuming the link to the repo is correct: it's open source. https://github.com/redfly-ai-org/redfly.ai/blob/main/LICENSE

I believe this is another case of abusing GitHub for visibility, not actually doing anything meaningful open source The code in the repo just seems to be connection code ("provides a way for anybody to test our Redis synchronization service on demand") rather than the sync service itself

[deleted]

Re: You can now directly sync Postgres with Redis

#14

Post by solatic »

Can someone ELI5 where this is going to be valuable?

If you set up Postgres read replicas, make sure your queries are covered by appropriate indexes, and your hot queries are handled by Postgres from RAM anyway - do you really need a separate Redis cache?

I feel like anytime I'm going to pull in Redis for a cache, it's going to be for data that I'm not interested in having guaranteed persistency for, like session tokens, and so it'll be for data that I wouldn't be storing in Postgres anyway?

Re: You can now directly sync Postgres with Redis

#15

Post by lotharcable »

solatic wrote: post #14

Can someone ELI5 where this is going to be valuable? If you set up Postgres read replicas, make sure your queries are covered by appropriate indexes, and your hot queries are handled by Postgres from RAM anyway - do you really need a separate Redis cache? I feel like anytime I'm going to pull in Redis for a cache, it's going to be for data that I'm not interested in having guaranteed persistency for, like session tok…

Probably useful for sharing stuff with services that already have access to redis, but you don't want to add SQL client dependencies for and/or don't want to give access to your database.

But you want to be able to store/have single source of authority and update them using a SQL database.

This would then eliminate the need to maintain a separate lambda or cron job or something like that to keep them in sync.

Re: You can now directly sync Postgres with Redis

#16

Post by vijaymohan1979 »

_1tem wrote: post #5
maxmcd wrote: post #4

Since db cache consistency is a complex research space with things like https://readyset.io/ and https://www.usenix.org/system/files/conference/nsdi13/nsdi13... it would be great to hear more specifics about what this is giving up in order to sidestep all that complexity.

There’s also Polyscale as a service.

https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A

Re: You can now directly sync Postgres with Redis

#17

Post by vijaymohan1979 »

solatic wrote: post #14

Can someone ELI5 where this is going to be valuable? If you set up Postgres read replicas, make sure your queries are covered by appropriate indexes, and your hot queries are handled by Postgres from RAM anyway - do you really need a separate Redis cache? I feel like anytime I'm going to pull in Redis for a cache, it's going to be for data that I'm not interested in having guaranteed persistency for, like session tok…

https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A

Re: You can now directly sync Postgres with Redis

#19

Post by vijaymohan1979 »

henning wrote: post #7

It'd be nice to see documentation of how it works. Is it using a replication channel for Postgres? The write-ahead log? It looks like it does schema introspection and then works off that. How does it handle schema changes? How does this work in scenarios where you have multi-master replication? What's the compatibility with popular Postgres extensions like PostGIS? What happens if the database is rebooted or crashes?…

https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A

Re: You can now directly sync Postgres with Redis

#20

Post by vijaymohan1979 »

hangonhn wrote: post #9

Did you really mean to leave your AES key in the code in RedflyEncryptionKeys? > public const string AesKey = ... Also, the way AES is used in the code is not good practice. It seems to be using plain AES ( https://learn.microsoft.com/en-us/dotnet/api/system.security... ), which isn't meant to be used bare like that. It needs to be coupled with a digest algorithm to protect the ciphertext. Maybe use AesGcm instead?

https://github.com/redfly-ai-org/redfly.ai/wiki/Q-&-A
Post reply on HN