Ask HN: Is there an open source equivalent to Palantir?

I would start looking at gephi: https://gephi.org and sigma.js for visualization. To run influence or social network analysis I would look at R's igraph: http://igraph.org/r/ or spark graphx: http://spark.apache.org/docs/latest/graphx-programming-guide... . For the graph database. neo4j , elasticsearch or the open source graph database created by the uk intelligence service: https://github.com/gchq/Gaffer . Another interesting project is mazerunner that integrates spark graphx and neo4j together : https://neo4j.com/developer/apache-spark/#mazerunner

> Logos cannot be given yet and we'll likely never say who they are specifically

Why can't you say who they are?

The problem with Neo4j is that the end results are great, but the ingestion pipeline (especially for unstructured data) is very hard to make general purpose.

The ICIJ used a combination of Apache Tika, Nuix, Tesseract and a bunch of other components when loading data into Neo4j before interrogating it within Linkurious.

It's also worth noting, that Panama data-set is riddled with data quality issues (even if this is understandable given the size of the team compared to the scale of the problem).

Given their pricing (and the pricing of nearest competitors), combined with the relatively solved technical problem areas they operate in (connect to a data source, do some human driven NLP, draw a graph, show a map, map everything to some ontology), what they do is very ripe for some serious open source disruption.

The easy part is really the interface and information displays, the harder part (and where they make the lions share of their money) is in data connection services and software customization.

Building an Open Source Palantir tool wouldn't be all that difficult, in fact a great many organizations just build some subset of that tools using readily available open source components and with tighter coupling to their business needs. But these efforts are fractured and disorganized and there isn't a great centralized open source tool that really replicates their system.

Should there be? I think the general problem of pulling together lots of information into a common pool, then being able to annotate that data and map it to a semantic model is useful, and it generalizes well. But at the same time, many many sources of data are already available in nice semantically organized ways, with simpler interfaces (think IMDB, Pouet, Wikipedia, etc.) it's not quite clear that their approach offers enough payoff over these easier methods.

You might try Lumify from Altamira -- http://lumify.io/.