Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Postgres.new: In-browser Postgres with an AI interface

supabase.com

Post reply on HN
Print view View original

101–110 of 116 posts

Re: Postgres.new: In-browser Postgres with an AI interface

#101

Post by sgarland »

astockwell wrote: post #59
jedberg wrote: post #24

Earlier quoted context omitted.

Are you sure? Most of the popular videos today do not have what one would call great cinematography, but it doesn't seem to matter. No one cares. Sure, movies still use cinematographers, but movie watching time is getting eaten up with Instagram/TikTok, where cinematography doesn't matter. I fear applications will suffer the same fate. "Good enough" will take over "well-architected".

I think you are 100% spot-on. Good enough has always been fine for the vast majority of people and the vast majority of use-cases. Couple this with decreasing costs of storage (and ideally compute), and it doesn't matter if the data model is garbage, people can still get something workable that's better than the awful Excel files they curate now. It will still make errors, but eventually fewer than their spreadsheets…

> it doesn't matter if the data model is garbage

There is no "good enough" for data modeling. There is correct, and there is "this works, but it has latent bugs that will eventually surface." You either have referential integrity, or you don't.

Re: Postgres.new: In-browser Postgres with an AI interface

#102

Post by TeeWEE »

It is a neat tech demo but it clearly shows the limits of AI:

- I got it to generate invalid SQL resulting in errors - it merely generates reasonable SQL, but in my case it generated to disjoint set of tables…. - In practice you have tot review all code - It can point you into the wrong direction. Novel systems often have something smart/abstract in there. This system creates mostly Straightforward simple systems. That’s not where the value is

All in all, it’s not worth it to me. Writing code myself is easier than having to review LLM code

Within our organization we have forbidden full LLM merge request because more often than not the code was suboptimal. And had sneaky bugs/mistakes.

I’m not saying these can’t be overcome. But not with current LLM design. They mostly generate stuff they have seen and are bad as truly new stuff.

Re: Postgres.new: In-browser Postgres with an AI interface

#103

Post by Kiro »

TechDebtDevin wrote: post #58
Kiro wrote: post #52

Earlier quoted context omitted.

I presume you've done your competitor analysis because it's a really crowded space. https://news.ycombinator.com/item?id=41227656

Well this is the second time Supabase has had a product announcement very similar to mine. My implentation is more focused on data analysis and visualization via a natural language interface.However, the straighforward database operations that postgres.new tries to tackle are included.

I was mainly referring to the plethora of other services doing exactly that: analysis and visualization via natural language. Supabase is different since they are doing it locally in the browser but just regular text-to-SQL is one of the most common applications of LLMs. How will you differentiate yourself from the rest of the "chat with your database" services?

Re: Postgres.new: In-browser Postgres with an AI interface

#104

Post by Kiro »

threecheese wrote: post #98
codingwagie wrote: post #66

Earlier quoted context omitted.

No its that it will code a days worth of work in a few minutes

10x LoC is going to require more automation, to manage the sheer mass of it, which is more tools/money/layers of abstraction. AI coders need AI testers and AI peer reviewers, and need to iterate over and over to compensate for incorrectness to produce a working feature. That sounds hellishly inefficient. (but all it has to be is cheaper i suppose)

You're speaking theoretically but we're already using it like this and it's not hellish or inefficient, or I wouldn't use it. Granted, it fits certain tasks better than others but when it does it's a massive relief and I can't imagine going back.

Re: Postgres.new: In-browser Postgres with an AI interface

#106

Post by davidgomes »

solarkraft wrote: post #92

It would be cool to have this without the AI stuff. Also, does the WASM build perhaps enable using Postgres as an embedded db, where typically SQlite would be used?

Check out https://github.com/electric-sql/pglite, which this is built with.

Re: Postgres.new: In-browser Postgres with an AI interface

#107

Post by whalesalad »

TeeWEE wrote: post #102

It is a neat tech demo but it clearly shows the limits of AI: - I got it to generate invalid SQL resulting in errors - it merely generates reasonable SQL, but in my case it generated to disjoint set of tables…. - In practice you have tot review all code - It can point you into the wrong direction. Novel systems often have something smart/abstract in there. This system creates mostly Straightforward simple systems. Th…

I have had tremdendous success with using LLM's to generate SQL. In my use, the majority of the time, ChatGPT gets things spot-on. Even for really sophisticated queries that are going to inspect and aggregate multiple tables into one single output.

I do agree it is not perfect - and things need to be reviewed - but I rarely get the sort of gobbledygook that "resembles" valid SQL and is in fact meaningless.

Re: Postgres.new: In-browser Postgres with an AI interface

#108

Post by kiwicopple »

dvasdekis wrote: post #96

This tool is amazing for us. There's so many pieces to this - the capability is a big step forward for architecting databases. Rudimentary compared to what you've done, but is it possible to take an existing database schema, developed either in the Supabase migrations style or another tool like Flyway, and draw the diagram? That alone is huge for us in an enterprise setting, and would allow us to communicate the stru…

> take an existing database schema, developed either in the Supabase migrations ... and draw the diagram?

It might not be so obvious, but this diagram tool is already built into the Supabase Dashboard (under "Database")

I can see some value in providing this as a generic service for any PG database - I'll look into it - but I know that Dbeaver already has something like this built in (and I think pgadmin too)

Re: Postgres.new: In-browser Postgres with an AI interface

#109

Post by kiwicopple »

ignoramous wrote: post #97

From the blog's section on semantic search : Under the hood, we store the embeddings in a meta.embeddings table then pass back to AI the resulting IDs for each embedding. We do this because embedding vectors are big, and sending these back and forth to the model is not only expensive, but also error prone. Instead, the language model is aware of the meta.embeddings table and simply subqueries to it when it needs acce…

1. You can see the relevant code block here[0]. To summarise, instead of storing the embeddings "next to" the data that you provide, we create a table that can be referenced. This is because often we need to send the LLM some data from the table (like creating a chart). If the LLM sees a reference to `meta.embeddings` then it knows it can "fetch" that data later if it's needed (for RAG etc)

2. We haven't fine-tuned, it's all just detailed prompts which you can find in the code. We might need to fine tune later for local-only models, but for now GPT-4o is solid.

[0] https://github.com/supabase-community/postgres-new/blob/4d6c...

Re: Postgres.new: In-browser Postgres with an AI interface

#110

Post by underlines »

kiwicopple wrote: post #12
AlexErrant wrote: post #7

Clicking "New database" doesn't do anything for me...? No changes in the UI, and no messages in the console. Admittedly I'm not signed in with Github, but isn't that only for the AI thing (that really I don't want to use). Edit: Okay reading kiwicopple's comment makes it clearer that chatGPT is not-optional. I'm... not enthused by this. Why would you take something that's local-first and gatekeep it with something as…

We tested several models and GPT-4o was the most accurate and makes the most sense for our initial launch That said: it's 100% our intention to add local models. This is just our v1.

Benchmarks suggest otherwise. Toqan's sql benchmark shows other models way up in the ranking compared to gpt-4o [1]

Open Weight models specifically fine-tuned on sql generation and modification also rank pretty well compared to SOTA proprietary models. If you want to eval alternative models, check out sqleval [2]

1 https://prollm.toqan.ai/leaderboard/stack-unseen?type=concep...

2 https://github.com/defog-ai/sql-eval

Post reply on HN