I wonder what is the closest thing to Cyc we have in the open source realm right now. I know that we have some pretty large knowledge bases, like Wikidata, but what about expert system shells or inference engines?
OWL and SPARQL inference engines that use RDF and DSMs - there are LISPy variants like datadog still kicking around, but there are some great, high performance reasoner FOSS projects, like StarDog or Neo4j https://github.com/orgs/stardog-union/ Looks like Knowledge Graph and semantic reasoner are the search terms du'jour, I haven't tracked these things since OpenCyc stopped being active. Humans may not be able to eff…
Cyc: History's Forgotten AI Project
91–100 of 145 posts
Re: Cyc: History's Forgotten AI Project
#92Earlier quoted context omitted.
I'm very skeptical of Cyc and other symbolic approaches. However I think they have a good excuse for 'Why didn't it ever have the impact that LLMs are having now?': lack of data and lack of compute. And it's the same excuse that neural networks themselves have: back in those days, we just didn't have enough data, and we didn't have enough compute, even if we had the data. (Of course, we learned in the meantime that n…
Usually, LLM's output gets passed through beam search [1] which is as symbolic as one can get. [1] https://www.width.ai/post/what-is-beam-search It is possible to even have 3-gram model to output better text predictions if you combine it with the beam search.
Re: Cyc: History's Forgotten AI Project
#93Earlier quoted context omitted.
It's not "abandoned"; it's just that most money today goes into curve fitting; but there are features which can be better realized with production systems, e.g. things like explainability or causality.
>it's just that most money today goes into curve fitting It's pretty interesting to see comments like this like deep nets weren't the underdog for decades. You think they were first choice ? The creator of cyc spent decades on it, and he's dead. We use modern NNs today because they just work that much better. Gofai was abandoned in NLP long before the likes of GPT because non deep-net alternatives just sucked that mu…
Re: Cyc: History's Forgotten AI Project
#94Cyc is one of those bad ideas that won't die, and which keeps getting rediscovered on HN. Lenat wasted decades of his life on it. Knowledge graphs like Cyc are labor intensive to build and difficult to maintain. They are brittle in the face of change, and useless if they cannot represent the underlying changes of reality.
I'll tell you how I know /
I read it in the paper /
Fifteen years ago -
(John Prine)
Re: Cyc: History's Forgotten AI Project
#95I would love to see a Cyc 2.0 modeled in the age of LLMs. I think it could be very powerful, especially to help deal with hallucinations. I would love to see a causality engine built with LLMs and Cyc. I wrote some notes on it before ChatGPT came out: https://blog.jtoy.net/understanding-cyc-the-ai-database/
Re: Cyc: History's Forgotten AI Project
#96Earlier quoted context omitted.
Interesting that they're still using Allegro Common Lisp. I would be interested in knowing what technical issues (if any) prevented them from migrating to other implementations.
Out of curiosity, which implementation(s) did you have in mind and why would it be desirable to migrate a large project there?
I mostly wanted to know of any technical obstacles so SBCL could be improved. If I had to wildly guess, maybe GC performance? SBCL was behind ACL on that many years ago (on both speed and physical memory requirements) the last time I made a comparison.
Re: Cyc: History's Forgotten AI Project
#97Re: Cyc: History's Forgotten AI Project
#98Cyc is one of those bad ideas that won't die, and which keeps getting rediscovered on HN. Lenat wasted decades of his life on it. Knowledge graphs like Cyc are labor intensive to build and difficult to maintain. They are brittle in the face of change, and useless if they cannot represent the underlying changes of reality.
I think before 2022 it was still an open question whether it was a good approach. Now it's clear that knowledge graphs are far inferior to deep neural nets, but even still few people can explain the _root_ reason why. I don't think Lenat's bet was a waste. I think it was sensible based on the information at the time. The decision to research it largely in secret, closed source, I think was a mistake.
If that is so then symbolic AI does not easily scale because you cannot feed inconsistent information into it. Compare this to how humans and LLMs learn, they both have no problem with inconsistent information. Yet statistically speaking humans can easily produce "useful" information.
Re: Cyc: History's Forgotten AI Project
#99This is a pretty good article. I was one of the first hires on the Cyc project when it started at MCC and was at first responsible for the decision to abandon the Interlisp-D implementation and replace it with one I wrote on Symbolics machines. Yes, back then one person could write the code base, which has long since grown and been ported off those machines. The KB is what matters anyway. I built it so different peop…
> which was unusual in those days, even though cloud computing was ubiquitous at PARC I don't want to rob you of your literary freedom, but that threw me off. Mainframes were meant, yes?
No not at all. We’re talking early-mid 1980s so people in the research community (at least at the leading institutions) were by then pretty used to what’s called cloud computing these days. In fact the term “cloud” for independent resources you could call upon without knowing the underlying architecture came from the original Internet papers (talking originally about routing, and then the DNS) in the late 70s
So for example the mail or file or other services at PARC just lived in the network; you did the equivalent of an anycast to check your mail or look for a file. These had standardized APIs so it didn’t matter if you were running Smalltalk, Interlisp-D, or Cedar/Mesa you just had a local window into a general computing space, just as you do today.
Most was on the LAN, of course, as the ARPANET was pretty slow. But when we switched to TCP/IP the LAN/WAN boundaries became transparent and instead of manually bouncing through different machines I could casually check my mail at MIT from my desk at PARC.
Lispms were slightly less flexible in this regard back then, but then again Ethernet started at PARC. But even in the late 70s it wasn’t weird to have part of your computation run on a remote machine you weren’t logged into interactively.
The Unix guys at Berkeley eventually caught up with this (just look at the original sockets interface, very un-unixy) but they didn’t quite get it: I always laughed when I saw a sun machine running sendmail rather than trusting the network to do the right thing on its behalf. By the time Sun was founded that felt paleolithic to me.
Because I didn’t start computing until the late 70s I pretty much missed the whole removable media thing and was pretty much always network connected.
Re: Cyc: History's Forgotten AI Project
#100And there were description if EURISCO (with claims that it not only "win some game" but also that it "invented new structure of NAND-gate in silicon, used by industry now") and other expert systems.
One of the mentioned expert systems (without technical details) said was 2 times better in diagnose cancer than best human diagnostician of some university hospital.
And after that... Silence.
I always wonder, why did this expert system were not deployed in all USA hospitals, for example? If it is so good?
Now we have LLMs, but they are LANGUAGE models, not WORLD models. They predict distribution of possible next words. Same with images — pixels, not world concepts.
Looks like such systems are good for generating marketing texts, but can not be used as diagnosticians by definition.
Why did all these (slice of) world model approaches dead? Except Cyc, I think. Why we have good text generators and image generators but not diagnosticians 40 years later? What happens?..