This is written in C. I wonder how common it is to write PG extensions in safer langugaes and what would be the most suitable. I'm somewhat wary of using nontrivial C extensions, having seen so many of them sometimes seg fault the backend (eg PostGIS). There seem to be PG backend crashes described in this projects issues as well.
Apache AGE, a PostgreSQL extension with graph database functionality
31–40 of 77 posts
Re: Apache AGE, a PostgreSQL extension with graph database functionality
#32This is written in C. I wonder how common it is to write PG extensions in safer langugaes and what would be the most suitable. I'm somewhat wary of using nontrivial C extensions, having seen so many of them sometimes seg fault the backend (eg PostGIS). There seem to be PG backend crashes described in this projects issues as well.
I wish something like Lua + LuaJIT could be used to write such extensions; at least it's memory-safe. OTOH mapping these C interfaces to Lua structures, and making them work with GC may happen to be non-trivial.
Re: Apache AGE, a PostgreSQL extension with graph database functionality
#33This is written in C. I wonder how common it is to write PG extensions in safer langugaes and what would be the most suitable. I'm somewhat wary of using nontrivial C extensions, having seen so many of them sometimes seg fault the backend (eg PostGIS). There seem to be PG backend crashes described in this projects issues as well.
Postgres itself is written in C. I suppose its every internal interface is in C. I wonder how many unsafe sections would an extension written in Rust have to have to use these interfaces. I wish something like Lua + LuaJIT could be used to write such extensions; at least it's memory-safe. OTOH mapping these C interfaces to Lua structures, and making them work with GC may happen to be non-trivial.
(Also Python, Javascript, and Java)
I don't know specifics about the API coverage. It seems this extension mostly just implements new SQL visible functions and data types, which should be doable from those languages as well. Composite types might have to be defined as PG records (or json) instead of C level new PG object types.
Re: Apache AGE, a PostgreSQL extension with graph database functionality
#34has anyone here worked on graph neural networks ? basically creating embeddings for node based on their edge connectivity (or reachability) and using that for neural networks ? how do you do this at scale ?its generally a NP hard problem, but wondering whether something like AGE helps. not sure how Google, etc or even someone on fraud detection does this at scale
That results in a huge text file, that you then embed as if it were a normal text. The result is a normal 'word embedding' where the words are in reality the node id's. Works like a charm. Highly scalable.
Re: Apache AGE, a PostgreSQL extension with graph database functionality
#35Re: Apache AGE, a PostgreSQL extension with graph database functionality
#36Earlier quoted context omitted.
Postgres itself is written in C. I suppose its every internal interface is in C. I wonder how many unsafe sections would an extension written in Rust have to have to use these interfaces. I wish something like Lua + LuaJIT could be used to write such extensions; at least it's memory-safe. OTOH mapping these C interfaces to Lua structures, and making them work with GC may happen to be non-trivial.
PG ships with Lua support: https://www.postgresql.org/docs/current/external-pl.html (Also Python, Javascript, and Java) I don't know specifics about the API coverage. It seems this extension mostly just implements new SQL visible functions and data types, which should be doable from those languages as well. Composite types might have to be defined as PG records (or json) instead of C level new PG object types.
You cannot though write a new storage engine, a new kind of index, or something else that takes an extension.
Re: Apache AGE, a PostgreSQL extension with graph database functionality
#37Well sorry, we have PostgreSQL 15 already.
Re: Apache AGE, a PostgreSQL extension with graph database functionality
#38"Apache AGE is currently being developed for the PostgreSQL 12 release" Well sorry, we have PostgreSQL 15 already.
"and will support PostgreSQL 13 and all the future releases of PostgreSQL."
Re: Apache AGE, a PostgreSQL extension with graph database functionality
#39This is written in C. I wonder how common it is to write PG extensions in safer langugaes and what would be the most suitable. I'm somewhat wary of using nontrivial C extensions, having seen so many of them sometimes seg fault the backend (eg PostGIS). There seem to be PG backend crashes described in this projects issues as well.
And in PG, there is a special method to create a process, creating threads is not possible because the logging system makes heavy use of setjmp().