Congratz to neon team (i like what they built), but i don’t see the value or relation to databricks. I hope neon will continue as a standalone product, otherwise we lose a solid postgres provider from the market.
They claim they will in the FAQ… but we know how this usually goes
Databricks acquires Neon
181–190 of 234 posts
Re: Databricks acquires Neon
#182Hey everyone, I'm an engineer at Neon and I wanted to share this FAQ which covers a lot of the questions that are being brought up in the comments here: https://neon.tech/databricks-faq We're really excited about this, and will try to respond to some of the questions people have here later.
The FAQ, as meaningless as history as shown it is, is missing one key question: why?
That's the easiest to answer. Money.
Re: Databricks acquires Neon
#183Earlier quoted context omitted.
What's wrong with minio out of curiosity? Ceph an option?
Try expanding a cluster, or changing erasure coding configuration, or using anything that needs random access within a file (parquet), or any day 2 operation.
Even some basic s3 storage patterns weren’t considered when the core storage scheme was designed. Lacks an index and depends on filesystem to organize objects and then crumbles to lock contention when too many versions are stored or under walkdir calls when anything is listed. It also can’t even support writing to the same set of keys as S3 should allow since it implicitly depends on underlying filesystem paths.
They might have added an index by now but gatekept it to their enterprise AIStor offering since they’ve abandoned any investment in open source at this point or appearance that they care about that. Their initial inclination in response to this issue says everything - https://github.com/minio/minio/issues/20845#issuecomment-259...
Re: Databricks acquires Neon
#184Not too familiar with Neon other than the basics - its premise is that you use S3 as bottomless storage for Postgres and it’s otherwise the same as standard Postgres right? And this is all open source? Why are people paying? Can’t you use a cloud provider and have them host this for you?
Neon is only scale-to-zero from the point of view of the tenants; whoever runs the software, must keep nodes running, including triple-replicated WAL storage.
Re: Databricks acquires Neon
#185Earlier quoted context omitted.
Eh you don’t even need to go through all the trouble building a startup. imo Neon was interesting and filled a niche while open source solutions were really gaining maturity and adoption. Now they have, lots and lots of recommendations in this comment section, so my sense is that building a startup would be like reinventing the Neon wheel, just too late. Perhaps, depending on licensing, running OSS as a software is v…
Oh, my startup isn't about Postgres, but rather a GPU-accelerated Spark: https://news.ycombinator.com/item?id=43964505 What are some bad UX choices you generally dislike in data products?
If you come up with the answers to some of these questions, I'd definitely read those blog articles on how you came to those conclusions. Keep asking interesting questions! Cheers
Re: Databricks acquires Neon
#186Data warehousing is quickly becoming a commodity through open-source. I know a company who had 2PBs+ of data in Cloudera. But instead of moving to the cloud (and Databricks), they saved 5X costs by building their own analytics platform with Iceberg, Trino and Superset. The k8s operators are enterprise quality now. On-premises S3 is good, too. You can have great hardware (servers with 128 cpus and 1 TB) and networking…
Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant. We’d look for the “call us” pricing rather than hate it, which I normally do. We added databricks and it was considered one of my top three achievements, because they don’t have to think about data platforms again, just focus on using it.…
> Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant.
Hence IBM talking up Iceberg: https://www.ibm.com/think/topics/apache-iceberg
Re: Databricks acquires Neon
#187From the actual article >As Neon became GA last year, they noticed an interesting stat: 30% of the databases were created by AI agents, not humans. When they looked at their stats again recently, the number went from 30% to over 80%. That is, AI agents were creating 4 times more databases versus humans. For me this has alarm bells all over it. Databricks is trying to pump postgres as some sort of AI solution. We do l…
and how many of those DBs are still being actively used...
Re: Databricks acquires Neon
#188Data warehousing is quickly becoming a commodity through open-source. I know a company who had 2PBs+ of data in Cloudera. But instead of moving to the cloud (and Databricks), they saved 5X costs by building their own analytics platform with Iceberg, Trino and Superset. The k8s operators are enterprise quality now. On-premises S3 is good, too. You can have great hardware (servers with 128 cpus and 1 TB) and networking…
Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant. We’d look for the “call us” pricing rather than hate it, which I normally do. We added databricks and it was considered one of my top three achievements, because they don’t have to think about data platforms again, just focus on using it.…
> My last gig wouldn’t allow open source software or any company that might not be there in a decade
I bet they had VMware all over the place.
Re: Databricks acquires Neon
#189Not too familiar with Neon other than the basics - its premise is that you use S3 as bottomless storage for Postgres and it’s otherwise the same as standard Postgres right? And this is all open source? Why are people paying? Can’t you use a cloud provider and have them host this for you?
Neon is only scale-to-zero from the point of view of the tenants; whoever runs the software, must keep nodes running, including triple-replicated WAL storage.
[deleted]
Re: Databricks acquires Neon
#190Earlier quoted context omitted.
Oh, my startup isn't about Postgres, but rather a GPU-accelerated Spark: https://news.ycombinator.com/item?id=43964505 What are some bad UX choices you generally dislike in data products?
If you come up with the answers to some of these questions, I'd definitely read those blog articles on how you came to those conclusions. Keep asking interesting questions! Cheers
Which questions are you referring to specifically?