Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Supabase Storage now supports the S3 protocol

supabase.com

Post reply on HN
Print view View original

61–70 of 194 posts

Re: Supabase Storage now supports the S3 protocol

#61

Post by kiwicopple »

jarpineh wrote: post #59

Hi, a question, but first some background. I've been looking at solutions to store columnar data with versioning, essentially Parquet. But, I'd also like to store PDFs, CSVs, images, and such for our ML workflows. I wonder if now, that Supabase is getting better for data science DuckDB crowd, could Supabase be that one solution for all this?

> Parquet. But, I'd also like to store PDFs, CSVs, images

yes, you can store all of these in Supabase Storage and it will probably "just work" with the tools that you already use (since most tools are s3-compatible)

Here is an example of one of our Data Engineers querying parquet with DuckDB: https://www.youtube.com/watch?v=diL00ZZ-q50

We're very open to feedback here - if you find any rough edges let us know and we can work on it (github issues are easiest)

Re: Supabase Storage now supports the S3 protocol

#63

Post by kiwicopple »

jimmySixDOF wrote: post #53

Supabase also announced this week Oriole (the team not just the table storage plugin) is joining them so I guess this is part of the same story. Anyway it's nice timing I was thinking about a hookup to Cloudflare R2 for something and this may be the way.

Oriole are joining to work on the OrioleDB postgres extension. That's slightly different to this release:

- This: for managing large files in s3 (videos, images, etc).

- Oriole: a postgres extension that's a "drop-in replacement" for the default storage engine

We also hope that the team can help develop Pluggable Storage in Postgres with the rest of the community. From the blog post[0]:

> Pluggable Storage gives developers the ability to use different storage engines for different tables within the same database. This system is available in MySQL, which uses the InnoDB as the default storage engine since MySQL 5.5 (replacing MyISAM). Oriole aims to be a drop-in replacement for Postgres' default storage engine and supports similar use-cases with improved performance. Other storage engines, to name a few possibilities, could implement columnar storage for OLAP workloads, highly compressed timeseries storage for event data, or compressed storage for minimizing disk usage.

Tangentially: we have a working prototype for decoupled storage and compute using the Oriole extension (also in the blog post). This stores Postgres data in s3 and there could be some inter-play with this release in the future

[0] https://supabase.com/blog/supabase-aquires-oriole

Re: Supabase Storage now supports the S3 protocol

#64

Post by cqqxo4zV46cp »

paradite wrote: post #52
gherkinnn wrote: post #34

Earlier quoted context omitted.

This is my biggest reservation towards Supabase. Google bought Firebase in 2014. I've seen Vercel run Nextjs in to the ground and fuck up their pricing for some short-term gains. And Figma almost got bought by Adobe. I have a hard time trusting products with heavy VC backing.

I think Vercel and Next.js are built by the same group of people, the same people who made Now.sh, started the company (Zeit), then changed product name to Now.sh, then changed company and product name to Vercel.

Yes. That doesn’t mean that they haven’t ran it into the ground.

Re: Supabase Storage now supports the S3 protocol

#66

Post by jarpineh »

kiwicopple wrote: post #61
jarpineh wrote: post #59

Hi, a question, but first some background. I've been looking at solutions to store columnar data with versioning, essentially Parquet. But, I'd also like to store PDFs, CSVs, images, and such for our ML workflows. I wonder if now, that Supabase is getting better for data science DuckDB crowd, could Supabase be that one solution for all this?

> Parquet. But, I'd also like to store PDFs, CSVs, images yes, you can store all of these in Supabase Storage and it will probably "just work" with the tools that you already use (since most tools are s3-compatible) Here is an example of one of our Data Engineers querying parquet with DuckDB: https://www.youtube.com/watch?v=diL00ZZ-q50 We're very open to feedback here - if you find any rough edges let us know and we…

Well, this is great news. I'll take "just works" guarantee any day ;)

We have yet to make a commitment to any one product. Having Postgres there is a big plus for me. I'll have to see about doing a test or two.

Re: Supabase Storage now supports the S3 protocol

#67

Post by kiwicopple »

fenos wrote: post #51
sgt wrote: post #49

Can Supabase host static content yet (in a decent way)?

We don’t support static website hosting just yet - might happen in the future :)

just a small elaboration:

You can serve HTML content if you have a custom domain enabled. We don't plan to do anything beyond that - there are already some great platforms for website hosting, and we have enough problems to solve with Postgres

Re: Supabase Storage now supports the S3 protocol

#68

Post by mort96 »

kiwicopple wrote: post #7
jonplackett wrote: post #4

Dear supabase. Please don’t get bought out by anyone and ruined. I’ve built too many websites with a supabase backend now to go back.

we don't have any plans to get bought. we only have plans to keep pushing open standards/tools - hopefully we have enough of a track record here that it doesn't feel like lip service

Is it even up to you? Isn't it up to your Board of Directors (i.e investors) in the end?

Re: Supabase Storage now supports the S3 protocol

#69

Post by MatthiasPortzel »

gherkinnn wrote: post #34
jonplackett wrote: post #4

Dear supabase. Please don’t get bought out by anyone and ruined. I’ve built too many websites with a supabase backend now to go back.

This is my biggest reservation towards Supabase. Google bought Firebase in 2014. I've seen Vercel run Nextjs in to the ground and fuck up their pricing for some short-term gains. And Figma almost got bought by Adobe. I have a hard time trusting products with heavy VC backing.

I’m not defending Vercel or VC backed companies per se, but I don’t understand your comments towards Vercel. They still offer a generous hobby plan and Next.js is still actively maintained open source software that supports self-hosting.

Heroku might be a better example of a company that was acquired and the shut down their free plan.

Re: Supabase Storage now supports the S3 protocol

#70

Post by jonplackett »

kiwicopple wrote: post #35
jonplackett wrote: post #33

Earlier quoted context omitted.

Absolutely. I am so impressed with SB. It’s like you read my mind and then make what I’ll need before I realise. (This is not a paid promotion)

>like you read my mind we receive a lot of community feedback and ultimately there are only a few "primitives" developers need to solve 90% of their problems I think we're inching closer to the complete set of primitives which can be used to combine into second-order primitives (eg: queues/search/maps can all be "wrappers" around the primitives we already provide)

That's a neat way of thinking about it.

Thanks for an awesome product. Please also never get bought or make plans to in the future, or if you really, really, really have to then please not by google.

Post reply on HN