111–117 of 117 posts
Earlier quoted context omitted.
For when you need more executors without rewriting your logic.
Using a Python solution like Dask might actually be better, because you can work with all of the Python data frameworks and tools, but you can also easily scale it if you need it without having to step into the Spark world.
And you can still use Python data frameworks with Spark so not sure what you're getting.
Earlier quoted context omitted.
Around 20 years ago I was working for a database company. During that time, I attended SIGMOD, which is the top conference for databases. The keynote speaker for the conference Stonebraker, who started Postgres, among other things. He talked about the history of relational databases. At that time, XML databases were all the rage -- now nobody remembers them. Stonebraker explained that there is nothing new in the hier…
MongoDB is a $2b/year revenue company growing at 20% y/y. JSON stores are not going anywhere and it's an essential tool for dealing in data where you have no control over the schema or you want to do it in the application layer. And the only "battle" is one you've invented in your head. People who deal in data for a living just pick the right data store for the right data schema.
I don't think MongoDB is going anywhere on the medium term, and there is always going to be some customers. Just like the network databases in the 70s, or XML databases in the 90s.
Bad ideas never die, they just resurface in another form, which people label as "new".
https://db.cs.cmu.edu/papers/2024/whatgoesaround-sigmodrec20...
Me, circa 2007, when lovely hadoop guy trying to tell me how good it was for 10m rows of data... It was much slower than Oracle free version as I recall.
Earlier quoted context omitted.
> and in a space that wasn't easy for guys from other parts of tech to join easily and call it out (e.g., SRE, Backend, Design, Front-end, Systems Engineering, etc.). As an SRE/SysEng/Devops/SysAdmin (depending on the company that hires me): most people in the same job as me could easily call it out. You don't have to be such a big nerds to know that you can fit 6TB of memory in a single (physical) server. That's bee…
> The unspoken secret is that this kind of BS wasn't/isn't only going on in the data fields (in my opinion). Yes, once you see it in one area you notice if everywhere. A lot of IT spend is CEOs chasing something they half heard/misunderstanding a competitor doing, or a CTO taking Gartner a little too seriously, or engineering leads doing resume driven architecture. My last shop did a lot of this kind of this stuff "w…
Exactly. It cannot be unseen. It’s almost a curse.
Earlier quoted context omitted.
So is Phillip Morris - feel free to smoke all you want.
What an idiotic and childish take. Yes using JSON to store your data will kill millions of people.
So, if your evidence is "well MongoDB makes money" then that doesn't mean much.
Earlier quoted context omitted.
At companies I've worked at "Big Data" was often used to mean "too big to open in Excel" or in the extreme case "too big to fit in RAM on my laptop"
Annoyingly medium data is my term for this. Around 0.5 to 50 GB is such an annoying area, because Excel starts falling over on the lower end and even nicer computers will start seriously struggling on the larger end if you're not being extremely efficient.