I have experience from multiple projects with NiFi and it was the main reason for me and others quitting the company.
Somehow management were convinced by some salesmen that this would be the golden bullet, however, all of their deliveries were delayed. We experienced issues debugging flows with performance problems, and even basic version control was problematic due to ids being replaced every time.
Apache NiFi
51–60 of 140 posts
Re: Apache NiFi
#52[deleted]
Re: Apache NiFi
#53We used NiFi...one of the worst experiences. It installs like an appliance and feels like you are grappling with a legacy tool weighed down by a classic view on architecture and maintenance. We had built a data pipeline and it was for very high-scale data. The theory of it was very much like a TIBCO type approach around data-pipelines. Sadly the reality was also like a TIBCO type approach around data-pipelines. One p…
> Sadly the reality was also like a TIBCO type approach around data-pipelines.
Thats exactly how it looks like, thanks for confirming. Will avoid.
Re: Apache NiFi
#54I am new to this site, why is there just a link to Apache NiFI on the front page? Is this somehow news? Sorry not trying to be rude just confuses me a bit since NiFi has been around for some time.
Someone found it interesting, I guess. :) Personally, I'm finding the discussion very interesting, since I haven't seen it here before.
Sometimes Wikipedia articles hit the front page. It's fine. Usually.
Re: Apache NiFi
#55We have a team using this at work. They had built a process and needed it to be put on a VM and run periodically. They said the requirements were a dual core machine and 8gb of ram. The “binary” was like 1.8gb. I’m sure it included a jre and a full nifi runtime, but god damn that is ridiculous. Had this process been built using go or crystal or something like that it probably would have been less than a megabyte and…
From the spec (8gb) it appears that NiFi was not needed. Were you co- locating zookeeper on the same nodes?
Also you say it needed to be run periodically? It’s supposed to be a long running service. If you need something that you can spin up and shutdown in a container or VM then it is probably not the solution.
Use it if you have high volumes of data you need to transport from A -> B. We run on a cluster with 256gb ram, 128 core per a node.
Re: Apache NiFi
#56I've used it a fair bit, though not for a couple of years. Few points, some of which may be out of date: * I've seen customers fall into the trap of thinking they don't need expensive developers because you can drag and drop, just people who can use a mouse can crack on with NiFi. * It persisted its config to an XML file, including the positions of boxes on the UI. Trying to keep this config in source control with mu…
> I've seen customers fall into the trap of thinking they don't need expensive developers because you can drag and drop, just people who can use a mouse can crack on with NiFi. That is sort of the key problem I see with NiFi (and equivalents). The heavy emphasis on graphical UI and visual paradigm sort of implies that its oriented towards non-developers, but problem is that it doesn't make non-developers suddenly exp…
Doesn't Microsoft Access have a similar interface? It's been a long time since I've used it.
Re: Apache NiFi
#57I've used it a fair bit, though not for a couple of years. Few points, some of which may be out of date: * I've seen customers fall into the trap of thinking they don't need expensive developers because you can drag and drop, just people who can use a mouse can crack on with NiFi. * It persisted its config to an XML file, including the positions of boxes on the UI. Trying to keep this config in source control with mu…
I have not found that argument persuasive to the managers who believe that coding is inherently wasteful if what you’re trying to do is technically possible in a workflow builder GUI.
Compare coding to managing and suddenly they're on board. In practical terms: a checklist for work can achieve the same thing as a manager verbalising and visualising that checklist. Every argument the manager has against that is the same as 'gui workflow' vs code.
Re: Apache NiFi
#58It reminds me of LONI Pipeline[1], which was created for the need of a neuroscience lab to process images of brain scans.
Re: Apache NiFi
#59Incredible piece of software. I've used it in production at my last two jobs. You can build almost anything in NiFi once you get into the mindset of how it works. A good way to get started with NiFi is to use it as a highly available quartz-cron scheduler. For example, running "some process" every 5 seconds. Disclaimer: I'm an Apache NiFi committer. An article you might find interesting about it's ability to scale. h…
This page is classic Apache project in that I read it and have no idea what it does. Can you high level explain what this thing is really for?
Re: Apache NiFi
#60So Apache has at least a handful of software packages that do about the same thing, but with different interfaces and connectivity?