Storm - the Hadoop of realtime processing
tech.backtype.com
Storm - the Hadoop of realtime processing
1–10 of 47 posts
Re: Storm - the Hadoop of realtime processing
#2Storm sounds great, but this post probably should have waited until it was actually open-sourced. As it is, it just comes across as naked self-promotion based on a technology that could for all we know be vaporware.
Re: Storm - the Hadoop of realtime processing
#3Storm sounds great, but this post probably should have waited until it was actually open-sourced. As it is, it just comes across as naked self-promotion based on a technology that could for all we know be vaporware.
Precisely. I read the whole post looking for a link to the source on github or something, and then the last sentence was just a huge letdown.
Re: Storm - the Hadoop of realtime processing
#4Storm sounds great, but this post probably should have waited until it was actually open-sourced. As it is, it just comes across as naked self-promotion based on a technology that could for all we know be vaporware.
Also, the lack of any scalability charts or diagrams of architecture is suspicious.
If you can't make it opensource, at least write a serious paper to support the claims. Like Google did for Big-Table.
A lot of people think their systems are scalable and fault-tolerant. Most are not. And from the information provided, we can't tell.
Re: Storm - the Hadoop of realtime processing
#5This sounds like something that's been painfully over-engineered.
One of the main problems they solve is "distributed RPC", from TFA: "There are a lot of queries that are both hard to precompute and too intense to compute on the fly on a single machine."
That's generally a sign that you've made a mistake somewhere in your application design. Pain is a response that tells you "stop doing that".
Re: Storm - the Hadoop of realtime processing
#6Storm sounds great, but this post probably should have waited until it was actually open-sourced. As it is, it just comes across as naked self-promotion based on a technology that could for all we know be vaporware.
Also, the lack of any scalability charts or diagrams of architecture is suspicious. If you can't make it opensource, at least write a serious paper to support the claims. Like Google did for Big-Table. A lot of people think their systems are scalable and fault-tolerant. Most are not. And from the information provided, we can't tell.
We've released open source projects (most notably ElephantDB and Cascalog) in the past that are successfully used in production by us as well as other companies. You should check them out if you're interested in a measure of quality, though I understand your concern.
We're a startup — we're not going to write an academic paper supporting the claims in the post. Nevertheless, Storm's an exciting project many people are curious to learn more about; that's why we've written something about it now.
We have a demo coming soon, and Storm itself will be open sourced soon enough.
Re: Storm - the Hadoop of realtime processing
#7This sounds like something that's been painfully over-engineered. One of the main problems they solve is "distributed RPC", from TFA: "There are a lot of queries that are both hard to precompute and too intense to compute on the fly on a single machine." That's generally a sign that you've made a mistake somewhere in your application design. Pain is a response that tells you "stop doing that".
So there are no complex distributed problems?
Re: Storm - the Hadoop of realtime processing
#8" To compute reach, you need to get all the people who tweeted the URL, get all the followers of all those people, unique that set of followers, and then count the number of uniques. It's an intense computation that potentially involves thousands of database calls and tens of millions of follower records."
Or you could use a Graph DB to solve a Graph problem.
URL -> tweeted_by -> users -> followed_by -> users
Try that on Neo4j.
Re: Storm - the Hadoop of realtime processing
#9This sounds like something that's been painfully over-engineered. One of the main problems they solve is "distributed RPC", from TFA: "There are a lot of queries that are both hard to precompute and too intense to compute on the fly on a single machine." That's generally a sign that you've made a mistake somewhere in your application design. Pain is a response that tells you "stop doing that".
So there are no complex distributed problems?
Didn't you know, everything is a website that can be written single-threaded?
Re: Storm - the Hadoop of realtime processing
#10Earlier quoted context omitted.
Also, the lack of any scalability charts or diagrams of architecture is suspicious. If you can't make it opensource, at least write a serious paper to support the claims. Like Google did for Big-Table. A lot of people think their systems are scalable and fault-tolerant. Most are not. And from the information provided, we can't tell.
We've released open source projects (most notably ElephantDB and Cascalog) in the past that are successfully used in production by us as well as other companies. You should check them out if you're interested in a measure of quality, though I understand your concern. We're a startup — we're not going to write an academic paper supporting the claims in the post. Nevertheless, Storm's an exciting project many people ar…
I absolutely understand the issue of being resource constrained.
It seems like this is buzz-worthy, (like http://mailchimp.com/omnivore/), but this pitch is nerd-focused, not potential-customer focused. If you pitch to nerds, you want a github link. If you pitch to potential customers, highlight the benefits that are now possible due to this innovation.
At least in our batch, we got drilled this repeatedly: Don't talk features. Talk benefits.