What happened to the Semantic Web? (2017) [pdf]

I was a big believer in the semantic web for years, but there is a load of things wrong with it from conceptual problems to practical ones.

For starters the Semantic Web requires an enormous amount of labor to make things work at all. You need humans marking up stuff, often with no advantage other than the "greater good". In fact you do see semantic content where it makes sense today. Look at any successful websites header and you'll see a pretty large variety of semantic content, things that Google and social media platforms use the make the page more discoverable.

This problem is compounded by the fact that ML and NLP solved many of the practical problems that the semantic web was supposed to. Google basically works like a vast question answering system. If you want to find pictures of "frogs with hats on" you don't need semantic metadata.

A much larger problem is that the real vision of the semantic web wreaked of the classic "solution in search of a problem". The magic of semantic web wasn't the metadata; RDF was just the beginning.

RDF is literally a more verbose implementation of Prolog's predicates. The real goal was to build reasoning engines on top of RDF, essentially a prolog like reasoner that could answer queries. A big warning sign for me was that the majority of people doing "Semantic Web" work at the time didn't even know of the basics of how existing knowledge based representation and reasoning systems, like prolog, worked. They were inventing a Semantic future without any sense that this problem has been worked on in another form for decades.

OWL, which was the standard to be used for the reasoning part of the semantic web was computationally intractable in it's highest level description of the reasoning process. If you start with a computationally intractable abstraction as your formal specification, they you are starting very far from praxis.

For this reason it was hard to really do anything with the semantic web. Virtually nobody built weekend "semantic web demos" because there wasn't really anything you could do with it that you couldn't do easier with a simple database and some basic business logic... or just write in Prolog.

A few companies did use semantic, RDF databases but you quickly realize these offered no value over just building a traditional relational database, and today we have real graph data bases in abundance so any advantage you would get form processing boatloads of XML as a graph can be replicated without the markup overhead. And that's not even considering the work in graph representation coming out of deep learning.

Semantic web didn't work because it was half-pipe dream, and not even a very interesting one at that.

Many of the commenters here have pointed out that the semantic web did not become popular because the incentives are not supportive of the investment. I agree, and this leads me to wonder if there are domains in which the cost of investing in semantic markup are relevant.

I see scientific publishing as a venue that could clearly benefit from application of a semantic web-like domain-specific approach. There are a variety of possibilities [0], [1], but again many are implemented behind proprietary paywalls. There's value in search and knowledge representation, but the prohibitive cost of implementation is challenging.

[0] https://en.wikipedia.org/wiki/Semantic_Scholar [1] https://en.wikipedia.org/wiki/Web_of_Science

> Google actually knee-capped the web with the WHATWG's focus on less semantic documents. The platforms with their army of engineers are what did the technology in.

Nah. Those platforms had years or decades to make their overengineered crap work, and they never could. XHTML2.0 was never finished, all those RDF ontologies never got anywhere, and all the semantic search engines couldn't find anything.

Google made a search engine that worked better than any other precisely because it ignored all the semantic bollocks, and eventually, after the web had stagnated for years by focusing on said semantic nonsense, the WHATWG formed to standardise some HTML that websites and users actually wanted. The person you were replying to had the right of it. If the semantic web was actually any good, it would've produced a better search engine than Google.

Scalable rigid ontologies are hard to the point of being impossible. Even tasks well suited to ontologies such as Question Answering see better results from non-ontological approaches.

> light semantics of the hyperlink: the anchor tag is the semantic web, giving you information that a server-side image map or Java applet or Flash movie or JS-driven navigation couldn't reliably identify

This, 100x. Complex and precise ontologies don't survive a distributed environment because communication is competitive. Publishers don't seek to describe themselves "accurately" (as if everyone could agree what that means), they seek to attract a certain type of attention in maximum quantities.

The protocols that survive that policy framework are those that "carry their own weight" for both listeners (by compressing information that would be hard to get) and publishers (by allowing sufficient expression to pursue selfish ends without lying).

Hyperlinks nail this trade off (at least well enough that pagerank successfully trims the noise). But Semantic Web gave little incentive for publishers to represent themselves, accurately or at all.