Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databricks acquires Neon

databricks.com

Post reply on HN
Print view View original

151–160 of 234 posts

Re: Databricks acquires Neon

#151

Post by dilyevsky »

Robdel12 wrote: post #128
swyx wrote: post #112

Earlier quoted context omitted.

what if you had joined at neon's previous valuation (whatever it was) and got a sudden payday (assuming you had juuuust enough vesting)

I was a very early employee at the other two start ups that were acquired and even with equity it was not worth it. After all the class A shares were paid out, the rest of us got little. I mean, hindsight 20/20 here, but I would have loved the theoretical money @ 1 billion. But those are so rare and my experience in the past 15 years hasn’t matched those unicorns. Basically I’ve come to the conclusion unless you have…

The 1b is very likely not all cash. Probably significant portion id db illiquid equity

Re: Databricks acquires Neon

#152

Post by SOLAR_FIELDS »

lars_francke wrote: post #93
jamesblonde wrote: post #74

Data warehousing is quickly becoming a commodity through open-source. I know a company who had 2PBs+ of data in Cloudera. But instead of moving to the cloud (and Databricks), they saved 5X costs by building their own analytics platform with Iceberg, Trino and Superset. The k8s operators are enterprise quality now. On-premises S3 is good, too. You can have great hardware (servers with 128 cpus and 1 TB) and networking…

Anyone looking for an open-source Cloudera alternative based on Kubernetes operators. We're building one (~5 years old now): https://stackable.tech/ & https://github.com/stackabletech/ On-premise open-source S3 is a problem though. MinIO is not something we're touching and other than that it looks a bit empty with enterprise ready solutions.

Don’t SeaweedFS and ceph/rook also offer this? Ceph/rook is definitely enterprise ready

Re: Databricks acquires Neon

#153

Post by swyx »

jamesblonde wrote: post #74

Data warehousing is quickly becoming a commodity through open-source. I know a company who had 2PBs+ of data in Cloudera. But instead of moving to the cloud (and Databricks), they saved 5X costs by building their own analytics platform with Iceberg, Trino and Superset. The k8s operators are enterprise quality now. On-premises S3 is good, too. You can have great hardware (servers with 128 cpus and 1 TB) and networking…

if Databricks just wanted a row DB they couldve done postgres themselves. paying this much for Neon i think is a sign that Neon has something special they want (which, knowing their marketing line, is "independently scalable storage and compute for postgres")

Re: Databricks acquires Neon

#154

Post by richardw »

jamesblonde wrote: post #74

Data warehousing is quickly becoming a commodity through open-source. I know a company who had 2PBs+ of data in Cloudera. But instead of moving to the cloud (and Databricks), they saved 5X costs by building their own analytics platform with Iceberg, Trino and Superset. The k8s operators are enterprise quality now. On-premises S3 is good, too. You can have great hardware (servers with 128 cpus and 1 TB) and networking…

Not commoditising for enterprise. My last gig wouldn’t allow open source software or any company that might not be there in a decade, or which kept data anywhere but our own tenant. We’d look for the “call us” pricing rather than hate it, which I normally do. We added databricks and it was considered one of my top three achievements, because they don’t have to think about data platforms again, just focus on using it. It’s SO expensive for an enterprise to rejig for a new platform that you can’t rely on (insert open source project here).

I managed to add one startup and so far it’s done very well, but it was an exceptional case and the global CEO wanted the functionality. But it used MongoDB and ops team didn’t have any skills, so rather than learn one tiny thing for an irrelevant data store they added cash to use Atlas with all the support and RBAC etc etc. They couldn’t use the default Azure firewall because they only know one firewall, so added one of those too. Also loaded with contracts. Kept hiring load down, one number to call, job done. Startups cost is $5-10k per year. Support BS about $40k. (I forget the exact numbers but it dwarfed the startup costs.)

Startups are from Venus, enterprise are from Jupiter.

Re: Databricks acquires Neon

#155

Post by winwang »

isignal wrote: post #102
mritchie712 wrote: post #19

Earlier quoted context omitted.

Databricks started in 2013 when Spark sucked (it still does) and they aimed to make it better / faster (which they do). The product is still centered Spark, but most companies don't want or need Spark and a combination of Iceberg and DuckDB will work for 95% of companies. It's cheaper, just as fast or faster and way easier to reason about. We're building a data platform around that premise at Definite[0]. It includes…

Aren't the alternatives you mentioned - icerberg and duckdb - both storage solutions while spark is a way to express distributed compute? I'm a bit out of touch with this space, is there a newer way to express distributed compute?

Not a new way like Ray, but a new way to express Spark super-efficiently (GPU-acceleration): https://news.ycombinator.com/item?id=43964505

Re: Databricks acquires Neon

#156

Post by winwang »

robertkoss wrote: post #13
flanked-evergl wrote: post #3

Maybe unrelated but Databricks is the most annoying garbage I have ever had to use. It fascinates me how anyone uses it by choice.

I used to be a big fan of the platform because back in 2020 / 2021 it really was the only reasonable choice compared to AWS / Azure / Snowflake for building data platforms. Today it suffers from feature creep and too many pivots & acquisitions. That they are insanely bad at naming features doesn't help either.

I'm building another Spark-based choice now with ParaQuery (GPU-accelerated Spark): https://news.ycombinator.com/item?id=43964505

Re: Databricks acquires Neon

#157

Post by winwang »

swalsh wrote: post #15
flanked-evergl wrote: post #3

Maybe unrelated but Databricks is the most annoying garbage I have ever had to use. It fascinates me how anyone uses it by choice.

Really hard disagree. Coming from hadoop, databricks is utopia. It's stable, fast, scales really well if you have massive datasets. The biggest gripe in have is how crazy expensive it is.

If cost (or perf) is the issue, we're building a super-efficient, GPU-accelerated, easy-to-use Spark: https://news.ycombinator.com/item?id=43964505

Re: Databricks acquires Neon

#158

Post by positisop »

dilyevsky wrote: post #150
positisop wrote: post #142

Earlier quoted context omitted.

Look under the hood, the limitations are based on the core, sticking a UI on it does not hide what needs to happen at scale.

Guessing you’re referring to minio not ceph? Have they still not figured out how to do day 2? I mainly avoid them because of their license and the way they interpret it

They are not efficient; they have a one-time static hash to create a cluster. After that, it is all duct tape and glue. Want to expand? Add another cluster (pool) and then look for the cluster that contains the object. They don't know which cluster has the object, and performance does not scale as well with additional clusters. Want to decommission a single node, drain the cluster. They refer to multiple pools as a single cluster, but it is essentially a set of static hashes that lack the intelligence to locate objects. Got the initial EC configuration not quite right.. sorry need to redo the entire cluster.

MinIO is a good fit if you want a small cluster that doesn't require day 2 operational complexity, as you only store a few TBs.

I have not looked into them recently, but I doubt the core has changed. Being VC-funded and looking for an exit makes them overinvest in marketing and story telling.

Re: Databricks acquires Neon

#159

Post by winwang »

MOARDONGZPLZ wrote: post #94
flanked-evergl wrote: post #3

Maybe unrelated but Databricks is the most annoying garbage I have ever had to use. It fascinates me how anyone uses it by choice.

Databricks is the Jira of dealing with data. No one wants to use it, it sucks, there are too many features to try to appease all possible users but none of them particularly good, and there are substantially better options now than there were not long ago. I would never, ever use it by choice.

What options do you use? I don't work for Databricks but I am building my own data infra startup, so I'd like to hear what "good" looks like!

Re: Databricks acquires Neon

#160

Post by dilyevsky »

positisop wrote: post #158
dilyevsky wrote: post #150

Earlier quoted context omitted.

Guessing you’re referring to minio not ceph? Have they still not figured out how to do day 2? I mainly avoid them because of their license and the way they interpret it

They are not efficient; they have a one-time static hash to create a cluster. After that, it is all duct tape and glue. Want to expand? Add another cluster (pool) and then look for the cluster that contains the object. They don't know which cluster has the object, and performance does not scale as well with additional clusters. Want to decommission a single node, drain the cluster. They refer to multiple pools as a s…

That tracks with my past analysis as well, thanks
Post reply on HN