Live data from Hacker News

Palantir's secret weapon isn't AI – it's Ontology. An open-source deep dive

github.com

11–20 of 64 posts

Re: Palantir's secret weapon isn't AI – it's Ontology. An open-source deep dive

#13
post #7
post #3

It's just view, materialized view, udf, stored procedure in fancy corp speak.

Do you have any resources or books to learn all the details of all these? Also, what is UDF?

Pretty much any SQL book will cover it those, and there's a bunch of online SQL tutorials. UDF means user defined function, so if there's some function you want to perform in SQL but that function doesn't come out of the box, you can just write your own. And those can be defined in non-SQL syntax, such as UDF's written in python or C++, which can be pretty handy.

Re: Palantir's secret weapon isn't AI – it's Ontology. An open-source deep dive

#14
post #9

> 2-1. Modeling the Data World with "Nouns" and "Verbs" > Link type: The relationships between object types, supporting 1-to-1, 1-to-many, and many-to-many relationships. Seems incredibly naive in terms of symbolic representation of knowledge. Maybe I spent too much time with OWL.

I find myself distinctly unimpressed by the idea that slapping a nice UI and some TS/SCI controls on top of a graph database — the latter being something that NSA did, with considerably more sophistication, years prior in a Neo4J fork — is some kind of brilliant conceptual moat. Graph DBs are useful for certain kinds of problems, which happen to map well to counterterror social mapping strategies, this is nothing particularly new or noteworthy.

Re: Palantir's secret weapon isn't AI – it's Ontology. An open-source deep dive

#15
palantir doesn't do revolutionary things in terms of back-ends. matter of fact, their apps are at best mid. I'd rate them 3/10 compared to alternatives that can do similar things. Their front end is the real differentiator.

Their bread-and-butter is a few things

1) Willing to do dirty/harmful things no one else will touch

2) Making data and data analysis accessible to cops, dhs, anyone that is especially tech-averse (many police departments disqualify based on IQ test results measuring too high). You can type in a license plate, a name, an address, scan a face and it will show you every relevant information, but also contextualizes it and enriches it with any other data. You could to this in excel, postgresql, bigquery, etc.. but palantir gives these people simple text boxes, buttons, and links.

3) Their forward deployed engineers are great at what they do. They station their guys wherever Palantir is being used, and they'll work very closely to get things done. to make sure all problems are solved asap, and its users are very well educated on the usage of the platform.

This post looks like it's written by AI, but assuming it is in earnest, it isn't really ontology, at least no more than object oriented programming is ontology. Excel is all about numbers, palantir is all about people (or people-documents). It is simpler than excel and has BigQuery level analytical power behind it, and the human touch to make that interaction go over really well.

I said it's mid because you could do a lot more with just the dataset and queries. You could even possibly do more with command line tools and hoards of data files (minus the OCR and document scanning they do, as well as LLM/NLP). but that isn't accessible and takes a lot more time. Not to mention normalizing, extracting and structuring wildy unstructured data isn't easy. But with BigQ for example, it is done plenty, you just hire a team to do that for you typically.

Their ecosystem is basically google search (including image, reverse image,video,etc..) but much more targeted and oriented towards displaying collated data from hoards of structured and unstructured data (including pdfs, docx,etc..). I would prefer grep, bigquery,splunk myself. but for end users, palantir is unmatched in my experience.

But I'm not selling them here, I'm trying to communicate the power at the disposal of those who use palantir's platforms. Google could have crushed them any time, except even for Google the type of work required was too ghoulish and reputationally risky.

Even with MS copilot(lol), chatgpt, gemini,etc.. running as agents, they're not as simply as palantir's stuff is for searching your data. and you don't have specialists integrating all your data onsite either.

Ultimately, the bigger problem is that even in crowds like HN's, no one seems to have a good idea of what should be done about governments abusing datascience so efficiently. Every answer comes back to red-tapes and regulations, possibly criminal consequence. Are you willing to give up the liberties tech has enjoyed so that future generations can be well, and have shot at peace and prosperity? (ours is too far gone in my opinion)?

China is doing this too, but much more efficiently, much better and at a greater scale. but their society has accepted this, and traded certain liberties for social stability and economic prosperity. The west hasn't done that. lawmakers and the public at large need to be informed by those in tech about these things so informed decisions could be made.

Re: Palantir's secret weapon isn't AI – it's Ontology. An open-source deep dive

#16
post #7

Earlier quoted context omitted.

Do you have any resources or books to learn all the details of all these? Also, what is UDF?

Pretty much any SQL book will cover it those, and there's a bunch of online SQL tutorials. UDF means user defined function, so if there's some function you want to perform in SQL but that function doesn't come out of the box, you can just write your own. And those can be defined in non-SQL syntax, such as UDF's written in python or C++, which can be pretty handy.

> python or C++

Also sometimes Lua, which is kinda a nice middleground between c++ efficiency and python ease of writing

Re: Palantir's secret weapon isn't AI – it's Ontology. An open-source deep dive

#17
post #9

> 2-1. Modeling the Data World with "Nouns" and "Verbs" > Link type: The relationships between object types, supporting 1-to-1, 1-to-many, and many-to-many relationships. Seems incredibly naive in terms of symbolic representation of knowledge. Maybe I spent too much time with OWL.

Can you say more? What is state of the art?

I've learned this stuff as a hobby, so take it with a grain of salt. I'm not a specialist.

OWL 1, for example, has stuff like transitive properties (the classical example is A ancestorOf B, B ancestorOf C, therefore I can infer A ancestorOf C if I annotate ancestorOf as a transitive property).

Union, equivalence, inversion, symmetries, cardinality. Those are all possible to represent symbolic in OWL ontologies.

They're also neatly separated in different types (OWL Lite, OWL DL, OWL Full). OWL Lite and DL for example are proven to be decidable (you won't get some halt when doing inference, no matter what).

I know there are plenty of database engines to store triples and graphs, and plenty of reasoners out there.

I haven't studied OWL 2 yet or newer stuff like SHACL, but I know it's supposed to be even better.

Post reply on HN