Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache Flink

github.com

Post reply on HN
Print view View original

11–20 of 65 posts

Re: Apache Flink

#11

Post by virgilp »

rakoo wrote: post #9
virgilp wrote: post #7

Earlier quoted context omitted.

I said "you went to far" to claim Kafka, Spark and Flink do basically the same thing. It's reasonable to use all 3 of them in the same team - so they clearly don't do the same thing. > I believe it drastically reduces adoption of these tools As does any competition. Tons of smartphone makers = less adoption for any one of them - and many will close down. Still not a bad thing.

> As does any competition. Those are open-source frameworks, which is antithetical to competition. The whole point of open-source is that the contributions of one entity can benefit everyone, while in this situation, having so many frameworks "dilutes" the efforts because the same problems need to be solved in each framework. Smartphone makers aren't here to share their technical advances with everyone, they're here…

I don't think that's how it works; "best" is sooo hard to define. Humans are very diverse, and have different needs/preferences. Also what is best today may not be best tomorrow (not because of the system itself, but because of changes in the environment), so it pays off to have "sub-par" systems evolve in parallel with different slightly different goals or implementations.

I don't see "open-source" as being antithetical to competition, at all. In fact, e.g. Databricks only exists because of Spark. There is competition in the open-source world too, and IMO that's a good thing.

Re: Apache Flink

#12

Post by powerapple »

meritt wrote: post #4

Apache Flink, Flume, Storm, Samza, Spark, Apex, and Kafka all do basically the same thing. I feel like this is a bit overboard. And this is before we talk about the non-Apache stream-processing frameworks out there. * Apache Flink is an open source stream processing framework * Apache Flume is a distributed, reliable, and available software for efficiently collecting, aggregating, and moving large amounts of log data…

... I am using Apache Pulsar to replace current Kafka setup * Apache Pulsar is an open-source distributed pub-sub messaging system

Re: Apache Flink

#14

Post by wpaladin »

_ZeD_ wrote: post #8

"Prerequisites for building Flink: Unix-like environment (we use Linux, Mac OS X, Cygwin) Java 8 (Java 9 and 10 are not yet supported)" Sigh is too much to ask for proper crossplatform support? And when the hell they will add support for recent versions of Java????

https://jira.apache.org/jira/browse/FLINK-8033

Re: Apache Flink

#15

Post by continuations »

powerapple wrote: post #12
meritt wrote: post #4

Apache Flink, Flume, Storm, Samza, Spark, Apex, and Kafka all do basically the same thing. I feel like this is a bit overboard. And this is before we talk about the non-Apache stream-processing frameworks out there. * Apache Flink is an open source stream processing framework * Apache Flume is a distributed, reliable, and available software for efficiently collecting, aggregating, and moving large amounts of log data…

... I am using Apache Pulsar to replace current Kafka setup * Apache Pulsar is an open-source distributed pub-sub messaging system

What compelled you to replace Kafka with Pulsar?

Re: Apache Flink

#16

Post by IshKebab »

skrebbel wrote: post #10
_ZeD_ wrote: post #8

"Prerequisites for building Flink: Unix-like environment (we use Linux, Mac OS X, Cygwin) Java 8 (Java 9 and 10 are not yet supported)" Sigh is too much to ask for proper crossplatform support? And when the hell they will add support for recent versions of Java????

Make a PR

I wish people would stop making this predictable and tedious comment that adds nothing to the conversation.

Re: Apache Flink

#17

Post by 11sdmb8m »

IshKebab wrote: post #16
skrebbel wrote: post #10

Earlier quoted context omitted.

Make a PR

I wish people would stop making this predictable and tedious comment that adds nothing to the conversation.

"Make a PR" actually means "This seems not to be relevant enough for the current maintainers to do it, and if it feels important to you, you should write it yourself and post a patch", which is a good answer.

Re: Apache Flink

#18

Post by PurpleRamen »

rakoo wrote: post #9
virgilp wrote: post #7

Earlier quoted context omitted.

I said "you went to far" to claim Kafka, Spark and Flink do basically the same thing. It's reasonable to use all 3 of them in the same team - so they clearly don't do the same thing. > I believe it drastically reduces adoption of these tools As does any competition. Tons of smartphone makers = less adoption for any one of them - and many will close down. Still not a bad thing.

> As does any competition. Those are open-source frameworks, which is antithetical to competition. The whole point of open-source is that the contributions of one entity can benefit everyone, while in this situation, having so many frameworks "dilutes" the efforts because the same problems need to be solved in each framework. Smartphone makers aren't here to share their technical advances with everyone, they're here…

Since when is there no competition and diversity in open source? Isn't too much choice usually a major complain with linux, programming languages, frameworks, etc?

Re: Apache Flink

#19

Post by m_mueller »

meritt wrote: post #4

Apache Flink, Flume, Storm, Samza, Spark, Apex, and Kafka all do basically the same thing. I feel like this is a bit overboard. And this is before we talk about the non-Apache stream-processing frameworks out there. * Apache Flink is an open source stream processing framework * Apache Flume is a distributed, reliable, and available software for efficiently collecting, aggregating, and moving large amounts of log data…

isn't this just a sign of a still young field (data science)? maybe we'll soon start to enter a consolidation phase.
Post reply on HN