Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Databricks in talks to acquire startup Neon for about $1B

upstartsmedia.com

Post reply on HN
Print view View original

61–70 of 177 posts

Re: Databricks in talks to acquire startup Neon for about $1B

#61

Post by jimbokun »

999900000999 wrote: post #36

Supabase just raised 200 million. What’s with all these Postgres hosting services being worth so much now? Someone at AWS probably thought about this, easy to provision serverless Postgres, and they just didn’t build it. I’m still looking for something that can generate types and spit it out in a solid sdk. It’s amazing this isn’t a solved problem. A long long time ago, I was apart of a team trying to sort this out.…

> Maybe a NoSql DB that “understands” the Postgres API?

I believe there are several of these already, like Cockroach DB.

Re: Databricks in talks to acquire startup Neon for about $1B

#62

Post by jimbokun »

999900000999 wrote: post #46
investa wrote: post #40

Earlier quoted context omitted.

Building backends is easy. It is sort of weird. In 2003 no one would bat an eyelid at building an entire app and chucking it on a server. I guess front-end complexity had made that a specialism so with all that dev energy drained they have no time for the backend. The backend is substantial easier though! These high value startups timed well to capture the vibe coding (was known as builidng an MVP before), front end…

It’s harder than signing up for Firebase. You have to understand a separate set of concerns. Spin something up on ec2, hook it into a db, configure https , figure out why it went down, etc. You’re right though, once I build a complex front end I want someone else to do the backend.

You need all that stuff when you need to scale. For an MVP you can get away with very little.

Re: Databricks in talks to acquire startup Neon for about $1B

#63

Post by jimbokun »

speakfreely wrote: post #45
taw1285 wrote: post #37

I am fairly new to all this data pipeline services (Databricks, Snowflakes etc). Say right now I have an e-commerce site with 20K MAU. All metrics are going to Amplitude and we can use that to see DAU, retention, and purchase volume. At what point in my startup lifecycle do we need to enlist the services?

A non-trivial portion of my consulting work over the past 10 years has been working on data pipelines at various big corporations that move absurdly small amounts of data around using big data tools like spark. I would not worry about purchasing services from Databricks, but I would definitely try to poach their sales people if you can.

Which is also a reason to not use Databricks, as they will cost your company money by selling gullible users things they don’t need.

Re: Databricks in talks to acquire startup Neon for about $1B

#64

Post by jimbokun »

User23 wrote: post #53

Meanwhile here I am wondering why everyone isn’t using SQLite.

If you can serve all your traffic by a single instance running Sqlite in same process as your application, have at it.

If you need to serve your dats across a network to many clients, managing that with SQLite is much trickier.

Re: Databricks in talks to acquire startup Neon for about $1B

#65

Post by emmelaich »

speakfreely wrote: post #45
taw1285 wrote: post #37

I am fairly new to all this data pipeline services (Databricks, Snowflakes etc). Say right now I have an e-commerce site with 20K MAU. All metrics are going to Amplitude and we can use that to see DAU, retention, and purchase volume. At what point in my startup lifecycle do we need to enlist the services?

A non-trivial portion of my consulting work over the past 10 years has been working on data pipelines at various big corporations that move absurdly small amounts of data around using big data tools like spark. I would not worry about purchasing services from Databricks, but I would definitely try to poach their sales people if you can.

I'm aware of a govt agency with a few hundred gb of data using Mongo, Databricks and were being pushed towards Snowflake as well. Boggles the mind.

Re: Databricks in talks to acquire startup Neon for about $1B

#66

Post by baggiponte »

jimbokun wrote: post #60
richardw wrote: post #50

Earlier quoted context omitted.

Capture enterprise AI enthusiasm by providing a 1-stop shop for data and AI, optionally hosted on your own cloud tenant. Keep deploying functionality so clients never need another supplier. Partner with SAP, OpenAI, anyone who holds market share. Buy anyone that either helps growth or might help a competitor grow. Enterprise view: delegate AI environment to Databricks unless you’re a real player. Market is too chaoti…

Can someone translate this to non-CEO speak?

You basically pay databricks a “fee” to choose the more appropriate and modern stack for you to build on, and keep it up to date. Never used it, but it handles with lots of the administrative bs (compliance, SLAs, idk) for you so you can just ship.

Re: Databricks in talks to acquire startup Neon for about $1B

#67

Post by janderson215 »

jimbokun wrote: post #60
richardw wrote: post #50

Earlier quoted context omitted.

Capture enterprise AI enthusiasm by providing a 1-stop shop for data and AI, optionally hosted on your own cloud tenant. Keep deploying functionality so clients never need another supplier. Partner with SAP, OpenAI, anyone who holds market share. Buy anyone that either helps growth or might help a competitor grow. Enterprise view: delegate AI environment to Databricks unless you’re a real player. Market is too chaoti…

Can someone translate this to non-CEO speak?

[flagged]

Re: Databricks in talks to acquire startup Neon for about $1B

#68

Post by lizard »

speakfreely wrote: post #45
taw1285 wrote: post #37

I am fairly new to all this data pipeline services (Databricks, Snowflakes etc). Say right now I have an e-commerce site with 20K MAU. All metrics are going to Amplitude and we can use that to see DAU, retention, and purchase volume. At what point in my startup lifecycle do we need to enlist the services?

A non-trivial portion of my consulting work over the past 10 years has been working on data pipelines at various big corporations that move absurdly small amounts of data around using big data tools like spark. I would not worry about purchasing services from Databricks, but I would definitely try to poach their sales people if you can.

Just curious, what would you consider, "absurdly small amounts of data around using big data tools like spark" and what do you recommend instead?

I recently worked on some data pipelines with Databricks notebooks ala Azure Fabric. I'm currently using ~30% of our capacity and starting to get pushback to run things less frequently to reduce the load.

I'm not convinced I actually need Fabric here, but the value for me has been its the first time the company has been able to provision a platform that can handle the data at all. I have a small portion of it running into a datbase as well which has been constant complaints about volume.

At this point I can't tell if we just have unrealistic expectations about the costs of having this data that everyone wants, or if our data engineers are just completely out of touch with the current state of the industry, so Fabric is just the cost we have to pay to keep up.

Re: Databricks in talks to acquire startup Neon for about $1B

#69

Post by viccis »

datadrivenangel wrote: post #2

Databricks is trying hard to get into serverless, but it seems like they refuse to allow it to actually be cheaper, which defeats the purpose of serverless.

There are so many gotchas. I'm getting so tired of working around it, but my company is all in on serverless so the pain will continue. A lot of it is tied up with Unity Catalog shortcomings, but Serverless and UC are basically joined at the hip.

A few just off the top of my head:

* You can't .persist() DataFrames in serverless. Some of my work involves long pipelines that wind up with relatively small DFs at the end of them, but need to do several things with that DF. Nowhere near as easy as just caching it. * Handling object storage mounted to Unity Catalog can be a nightmare. If you want to support multiple types of Databricks platforms (AWS, Azure, Google, etc.), then you will have to deal with the fact that you can't mount one type's object storage with another. If you're on Azure Databricks, you can't access S3 via Unity Catalog. * There's no API to get metrics like how much memory or CPU was consumed for a given job. If you want to handle monitoring and alerting on it yourself, you're out of luck. * For some types of Serverless compute, startup times from cold can be 1 minute or more.

They're getting better, but Databricks is an endless progression of unpleasant surprises and being told "oh no you can't do it that way", especially compared to Snowflake, whose business Databricks has been working to chew away at for a while. Their Variant type is a great example. It's so much more limited than Snowflake's that I'm still learning new and arbitrary ways in which it's incompatible with Snowflake's implementation.

Post reply on HN