Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Apache NiFi

nifi.apache.org

Post reply on HN
Print view View original

101–110 of 140 posts

Re: Apache NiFi

#101

Post by zok3102 »

coredog64 wrote: post #94
edmundsauto wrote: post #50

Earlier quoted context omitted.

Can you elaborate by what you mean on a TIBCO like approach? I haven't used their tools, but would like to know more about the issues you ran into. What were examples of the leaky abstraction>?

TIBCO is garbage. They had a halo for a long time from Rendezvous/EMS. But their money maker was this integration suite called BusinessWorks. It was this horrifyingly complex application that forced you into these ruts so that it could compile Java code. I kid you not, the developer environment for complex code was notepad.exe. They spent a bunch of money on M&A and eventually had to go private and buy out the founde…

IMO, there is no “TIBCO-like approach” to application integration c. 2020 any more than there is say an Oracle or AWS or Google approach to databases. It’s multi-paradigm, multi-usecase and polyglot. TIBCO as a vendor supports approaches and patterns ranging from event-driven functions to data streams and stateful orchestration to stateless mediation to choreography. The “runtimes” are built on anything from Golang, Python & Node to Java, Scala & .NET.

What you‘re referring to sounds like the legacy version of BusinessWorks 5.x that was launched back in 2001. The current generation of BusinessWorks 6.x provides Eclipse-based tooling just like closest alternatives like Talend, Mule, Fuse, etc. and deploys to 18+ PaaSes (k8s, swarm, GKE, AKS, etc.) or its own managed iPaaS aka TIBCO Cloud Integration. It’s aimed at Enterprise Integration specialists at a Global 2000 or F500.

If you‘re an app developer at a large bank/telco/retailer/airline building integration logic or stream processing or event-driven data pipelines, you‘re likely to use Project Flogo (flogo.io) It’s 3-clause BSD FLOSS and has commercial support and optionally commercial extensions available. Oh and you’re likely going to use Flogo apps with Apache Pulsar or Apache Kafka messaging. Both Pulsar and Kafka are available as commercially supported software from TIBCO (Rendezvous or EMS are our traditional proprietary messaging products). Flogo apps can deploy to TIBCO Cloud, dozen+ flavors of k8s, AWS Lambda, Google Cloud Run or as a binary on an edge device.

(Disclaimer: Product at TIBCO. Used to work on BW 6.0 back when the only PaaS was good ol’ Heroku)

Re: Apache NiFi

#102

Post by Mouse47 »

JimmyRuska wrote: post #88
mistrial9 wrote: post #72

Earlier quoted context omitted.

it is genuinely odd to see this prejuidice against graphical tools persist in 2020 .. text file dot-conf and graphical tools are different but.. is it controversial that a graphical tool can emit performant code ? I have no experience with NiFi itself.. just that graphical tools are not inherently "non-technical"

I don't think it's a prejudice. I've seen pentaho introduced as a way to enable business analysts to do work otherwise expensive developers would have to do. It simply didn't work, business analysts were not able to use it, devs had to take over and when they did they did not want to use pentaho. It's an easy trap for upper management to fall into thinking the selling point of these tools is the ability to allow the…

+1 to this. I've seen similar stories play out with Microsoft's Logic Apps, Power Apps, and to a lesser extent Dynamics.

Re: Apache NiFi

#103

Post by leoh »

cs02rm0 wrote: post #25

I've used it a fair bit, though not for a couple of years. Few points, some of which may be out of date: * I've seen customers fall into the trap of thinking they don't need expensive developers because you can drag and drop, just people who can use a mouse can crack on with NiFi. * It persisted its config to an XML file, including the positions of boxes on the UI. Trying to keep this config in source control with mu…

Curious if anyone from the NiFi team cares to comment / has thoughts on how to work around these issues (e.g. source control)

Re: Apache NiFi

#104

Post by chirau »

monstrado wrote: post #14

Incredible piece of software. I've used it in production at my last two jobs. You can build almost anything in NiFi once you get into the mindset of how it works. A good way to get started with NiFi is to use it as a highly available quartz-cron scheduler. For example, running "some process" every 5 seconds. Disclaimer: I'm an Apache NiFi committer. An article you might find interesting about it's ability to scale. h…

Quick question, what does the role of Data Engineer at Epic Games entail and what technologies are you working with?

Re: Apache NiFi

#106

Post by RhodesianHunter »

chii wrote: post #97
mistrial9 wrote: post #72

Earlier quoted context omitted.

it is genuinely odd to see this prejuidice against graphical tools persist in 2020 .. text file dot-conf and graphical tools are different but.. is it controversial that a graphical tool can emit performant code ? I have no experience with NiFi itself.. just that graphical tools are not inherently "non-technical"

There's a general level of elitism in the programming world. GUI tools are seen by some programmers as a "worse" tool, but without any real evidence, and it's likely due to the fact that GUI tools enable non-programmers to reach some level of output that used to be the sole domain of programmers. It's a similar train of thought that leads to a lot of game developers to look down on games made with GUI tools like game…

Can we claim that it's elitism when it's born from experience?

Programmers have seen these sorts of UIs fail to deliver countless times as described in the root of this thread. Where is the counter example?

Re: Apache NiFi

#107

Post by lmm »

mistrial9 wrote: post #72
zokier wrote: post #36

Earlier quoted context omitted.

> I've seen customers fall into the trap of thinking they don't need expensive developers because you can drag and drop, just people who can use a mouse can crack on with NiFi. That is sort of the key problem I see with NiFi (and equivalents). The heavy emphasis on graphical UI and visual paradigm sort of implies that its oriented towards non-developers, but problem is that it doesn't make non-developers suddenly exp…

it is genuinely odd to see this prejuidice against graphical tools persist in 2020 .. text file dot-conf and graphical tools are different but.. is it controversial that a graphical tool can emit performant code ? I have no experience with NiFi itself.. just that graphical tools are not inherently "non-technical"

> is it controversial that a graphical tool can emit performant code ?

No. But it's controversial to claim that a graphical tool can do maintainable code.

Re: Apache NiFi

#108

Post by wpietri »

chii wrote: post #97
mistrial9 wrote: post #72

Earlier quoted context omitted.

it is genuinely odd to see this prejuidice against graphical tools persist in 2020 .. text file dot-conf and graphical tools are different but.. is it controversial that a graphical tool can emit performant code ? I have no experience with NiFi itself.. just that graphical tools are not inherently "non-technical"

There's a general level of elitism in the programming world. GUI tools are seen by some programmers as a "worse" tool, but without any real evidence, and it's likely due to the fact that GUI tools enable non-programmers to reach some level of output that used to be the sole domain of programmers. It's a similar train of thought that leads to a lot of game developers to look down on games made with GUI tools like game…

GUI tools have a very long history of being about to revolutionize the industry but ending up as dead ends. If you want evidence, just look to the past.

I'm honestly all for tools that enable non-programmers to get things done. E.g., Excel, Google Forms, Zapier, and a lot of Salesforce tooling all deliver to some extent on that promise. But there's only so much they can do.

You're right that there's also an elitism problem. But if we're going to get past that, we have to start by understanding what GUI tools are good for. And being honest about their limits.

Re: Apache NiFi

#109

Post by fmakunbound »

Reminds me of the early 2000s when we were all into BPM, graphical or otherwise. The drawbacks are pretty obvious. I bet the engineers who built it had fun, tho.

Re: Apache NiFi

#110

Post by yenwel »

leoh wrote: post #103
cs02rm0 wrote: post #25

I've used it a fair bit, though not for a couple of years. Few points, some of which may be out of date: * I've seen customers fall into the trap of thinking they don't need expensive developers because you can drag and drop, just people who can use a mouse can crack on with NiFi. * It persisted its config to an XML file, including the positions of boxes on the UI. Trying to keep this config in source control with mu…

Curious if anyone from the NiFi team cares to comment / has thoughts on how to work around these issues (e.g. source control)

They already have a solution for that https://nifi.apache.org/registry.html
Post reply on HN