Billions wasted on Hadoop startups, the same will eventually be true of Docker
1–10 of 216 posts
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#2> ...but standard VMs allow the use of standard operating systems that solved all the hard problems decades ago, whereas Docker is struggling to solve those problems today.
What are these supposed "hard problems" the author speaks of?
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#3I have come across quite a few articles that mention that for 99% of 'big' data problems, Hadoop and the like are an overkill. Simple tools, with a beefy machine is just as sufficient for the task.
Is that the reality of today?
Personally I too feel that distributed computing is an overkill for most 'big' data problems.
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#4> ...but standard VMs allow the use of standard operating systems that solved all the hard problems decades ago, whereas Docker is struggling to solve those problems today. What are these supposed "hard problems" the author speaks of?
Sandboxed, consistent environments to run code in?
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#5> The same will eventually be said of Docker. I’ve yet to hear a single benefit attributed to Docker that isn’t also true of other VMs,
I bought a Raspberry Pi and using a few commands, installed pre-configured Docker ARM images for 8-9 different media applications that would've taken me days to setup and manage individually. I didn't have to worry about dependencies or compilations. It just worked.
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#6I have come across quite a few articles that mention that for 99% of 'big' data problems, Hadoop and the like are an overkill. Simple tools, with a beefy machine is just as sufficient for the task. Is that the reality of today? Personally I too feel that distributed computing is an overkill for most 'big' data problems.
I've heard people say that a huge portion of the "big" data is garbage, throwing more garbage at something won't make it better, so you'd get much more out of it by cutting it down to a more manageable amount of good data, which you can then easily process in postgres or whatever.
But even if you "collect everything and sort it out later", in my own personal experience and in what I've read here on HN, you can go a long long way before you need to reach for the power tools. What most companies call "big data" is typically not that much (in quantity and in velocity). Most companies don't have tens or hundreds of terabytes of data. For example, I'm currently processing timeseries data in postgres using the timescaledb extension, which makes it perform very well. Still too early to state numbers, but its looking promising so far and if their claims are true, then I won't need anything else. We will see :)
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#7I have come across quite a few articles that mention that for 99% of 'big' data problems, Hadoop and the like are an overkill. Simple tools, with a beefy machine is just as sufficient for the task. Is that the reality of today? Personally I too feel that distributed computing is an overkill for most 'big' data problems.
It really depends on how "big" your data is.
If you have data that is actually big (= doesn't fit onto a machine), then Hadoop is a reasonable candidate. Otherwise you are fine with a lot simpler tools.
This has probably been true for quite some time, and companies are just slowly realizing now that their relevant data isn't actually that big. On the other hand computing power and storage has still grown in the last years, and less resource intensive ETL has become more acessible, so the bar for "big" data has been raised quite a bit.
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#8> The same will eventually be said of Docker. I’ve yet to hear a single benefit attributed to Docker that isn’t also true of other VMs, I bought a Raspberry Pi and using a few commands, installed pre-configured Docker ARM images for 8-9 different media applications that would've taken me days to setup and manage individually. I didn't have to worry about dependencies or compilations. It just worked.
[deleted]
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#9> The same will eventually be said of Docker. I’ve yet to hear a single benefit attributed to Docker that isn’t also true of other VMs, I bought a Raspberry Pi and using a few commands, installed pre-configured Docker ARM images for 8-9 different media applications that would've taken me days to setup and manage individually. I didn't have to worry about dependencies or compilations. It just worked.
Exactly.
"other VMs"? The whole point of Docker is that it's not a VM...
Re: Billions wasted on Hadoop startups, the same will eventually be true of Docker
#10> ...but standard VMs allow the use of standard operating systems that solved all the hard problems decades ago, whereas Docker is struggling to solve those problems today. What are these supposed "hard problems" the author speaks of?
Sandboxed, consistent environments to run code in?
and environments as code