21–28 of 28 posts
Earlier quoted context omitted.
I only just discovered foreign data wrappers today but it seems like you can write these in python though right? Might be easier than using Go to interface with C.
Messagebird uses Go on the backend; I suspect that's why they wrote the FDW in Go.
Yes, your hypothesis is correct. We chose Go, mostly because almost all of us were quite familiar with it and there weren't many C experts :)
So, naive question, and I know the concept of a ‘database’ is a bit different between MySQL and Postgres, does Postgres require a FDW to communicate between multiple databases on the same server?
I wonder if you've benchmarked your Go implementation against the Percona Lab's FDW for Clickhouse? https://github.com/Percona-Lab/clickhousedb_fdw
There's now also a more recent CH FDW written in C by the team at adjust, https://github.com/adjust/clickhouse_fdw
I wonder if you've benchmarked your Go implementation against the Percona Lab's FDW for Clickhouse? https://github.com/Percona-Lab/clickhousedb_fdw
Hi, Author here. I have to admit, I haven't done any benchmarking against the existing FDWs for Clickhouse. But I actually wrote the Go FDW 2 years ago(around Dec 2018). ;) There weren't any Clickhouse FDW available at that time and I probably would've tried them as well. Now I just got around to write the blog post and convincing the team to release the code. Though I have a suspicion that the percona FDW might win…
Earlier quoted context omitted.
Hi, Author here. I have to admit, I haven't done any benchmarking against the existing FDWs for Clickhouse. But I actually wrote the Go FDW 2 years ago(around Dec 2018). ;) There weren't any Clickhouse FDW available at that time and I probably would've tried them as well. Now I just got around to write the blog post and convincing the team to release the code. Though I have a suspicion that the percona FDW might win…
Thanks for the reply. I really appreciate your post and the additional FDW. If you ever decide to give it a go, a benchmark would be interesting, if for no other reason than to see how much penalty cgo actually incurs vs possible efficiency gains in your implementation strategy.
I sometimes fondly remember my time as an office hero, using MSAccess to attach to just about any data source imaginable, copy to a temptable and run whatever cleaning was required before loading into the server database. Good times; almost 25 years ago now. Sometimes I wonder if we're stuck.
I had actually considered doing this recently with a new hire. I purchased them a license with Access included, but ai have not had time to play with it to talk to MySQL 5.6.
In MSAccess, just go to "External Data" -> "New Data Source" -> "From Other Sources" -> "ODBC Databases" (chose whether to link or import the data) and then create a new data source under "Machine Data Sources" using the MySQL driver.
It became like muscle memory after a while :-)
Earlier quoted context omitted.
There's now also a more recent CH FDW written in C by the team at adjust, https://github.com/adjust/clickhouse_fdw
Good find, looks like a more up to date fork of the Percona FDW.
So the "adjust" CH FDW is the only usable choice today, thankfully quite actively maintained, primarily by the Adjust devs. I'd presume it's used for their production load.
This is really cool. I had not heard of Foreign Data Wrappers for Postgres before! Are these used in production commonly or more of a toy thing?
There is one concern though. Both Google Cloud SQL and AWS RDS support only postgres_fdw, so one should manage their own storage, cluster and backups.