Live data from Hacker News

Cyc: History's Forgotten AI Project

outsiderart.substack.com

91–100 of 145 posts

Re: Cyc: History's Forgotten AI Project

#91
post #3

I wonder what is the closest thing to Cyc we have in the open source realm right now. I know that we have some pretty large knowledge bases, like Wikidata, but what about expert system shells or inference engines?

OWL and SPARQL inference engines that use RDF and DSMs - there are LISPy variants like datadog still kicking around, but there are some great, high performance reasoner FOSS projects, like StarDog or Neo4j https://github.com/orgs/stardog-union/ Looks like Knowledge Graph and semantic reasoner are the search terms du'jour, I haven't tracked these things since OpenCyc stopped being active. Humans may not be able to eff…

Did you mean Datalog here?

Re: Cyc: History's Forgotten AI Project

#92
post #80
post #66

Earlier quoted context omitted.

I'm very skeptical of Cyc and other symbolic approaches. However I think they have a good excuse for 'Why didn't it ever have the impact that LLMs are having now?': lack of data and lack of compute. And it's the same excuse that neural networks themselves have: back in those days, we just didn't have enough data, and we didn't have enough compute, even if we had the data. (Of course, we learned in the meantime that n…

Usually, LLM's output gets passed through beam search [1] which is as symbolic as one can get. [1] https://www.width.ai/post/what-is-beam-search It is possible to even have 3-gram model to output better text predictions if you combine it with the beam search.

See https://news.ycombinator.com/item?id=40073039 for a discussion.

Re: Cyc: History's Forgotten AI Project

#93
post #55

Earlier quoted context omitted.

It's not "abandoned"; it's just that most money today goes into curve fitting; but there are features which can be better realized with production systems, e.g. things like explainability or causality.

>it's just that most money today goes into curve fitting It's pretty interesting to see comments like this like deep nets weren't the underdog for decades. You think they were first choice ? The creator of cyc spent decades on it, and he's dead. We use modern NNs today because they just work that much better. Gofai was abandoned in NLP long before the likes of GPT because non deep-net alternatives just sucked that mu…

Curve-fitting has demonstrated impressive results, but it's definitely not the end of science.

Re: Cyc: History's Forgotten AI Project

#94
post #22

Cyc is one of those bad ideas that won't die, and which keeps getting rediscovered on HN. Lenat wasted decades of his life on it. Knowledge graphs like Cyc are labor intensive to build and difficult to maintain. They are brittle in the face of change, and useless if they cannot represent the underlying changes of reality.

We are living in the future /

I'll tell you how I know /

I read it in the paper /

Fifteen years ago -

(John Prine)

Re: Cyc: History's Forgotten AI Project

#95
post #5

I would love to see a Cyc 2.0 modeled in the age of LLMs. I think it could be very powerful, especially to help deal with hallucinations. I would love to see a causality engine built with LLMs and Cyc. I wrote some notes on it before ChatGPT came out: https://blog.jtoy.net/understanding-cyc-the-ai-database/

I used to volunteer inputting data into Cyc back in the day. And I get massive déjà vu with current LLM's. I remember that the system ended up with an obsession with HVAC systems lol.

Re: Cyc: History's Forgotten AI Project

#96
post #23

Earlier quoted context omitted.

Interesting that they're still using Allegro Common Lisp. I would be interested in knowing what technical issues (if any) prevented them from migrating to other implementations.

Out of curiosity, which implementation(s) did you have in mind and why would it be desirable to migrate a large project there?

Steel Bank Common Lisp is the most performant (by speed of compiled code) and stable open source implementation, and arguably the most standard compliant and bug free of any Common Lisp implementation, free or otherwise. It also is available free of charge.

I mostly wanted to know of any technical obstacles so SBCL could be improved. If I had to wildly guess, maybe GC performance? SBCL was behind ACL on that many years ago (on both speed and physical memory requirements) the last time I made a comparison.

Re: Cyc: History's Forgotten AI Project

#98
post #38
post #22

Cyc is one of those bad ideas that won't die, and which keeps getting rediscovered on HN. Lenat wasted decades of his life on it. Knowledge graphs like Cyc are labor intensive to build and difficult to maintain. They are brittle in the face of change, and useless if they cannot represent the underlying changes of reality.

I think before 2022 it was still an open question whether it was a good approach. Now it's clear that knowledge graphs are far inferior to deep neural nets, but even still few people can explain the _root_ reason why. I don't think Lenat's bet was a waste. I think it was sensible based on the information at the time. The decision to research it largely in secret, closed source, I think was a mistake.

I assume the problem with symbolic inference is that from a single inconsistent premise logic can produce any statement possible.

If that is so then symbolic AI does not easily scale because you cannot feed inconsistent information into it. Compare this to how humans and LLMs learn, they both have no problem with inconsistent information. Yet statistically speaking humans can easily produce "useful" information.

Re: Cyc: History's Forgotten AI Project

#99
post #11

This is a pretty good article. I was one of the first hires on the Cyc project when it started at MCC and was at first responsible for the decision to abandon the Interlisp-D implementation and replace it with one I wrote on Symbolics machines. Yes, back then one person could write the code base, which has long since grown and been ported off those machines. The KB is what matters anyway. I built it so different peop…

> which was unusual in those days, even though cloud computing was ubiquitous at PARC I don't want to rob you of your literary freedom, but that threw me off. Mainframes were meant, yes?

> Mainframes were meant, yes?

No not at all. We’re talking early-mid 1980s so people in the research community (at least at the leading institutions) were by then pretty used to what’s called cloud computing these days. In fact the term “cloud” for independent resources you could call upon without knowing the underlying architecture came from the original Internet papers (talking originally about routing, and then the DNS) in the late 70s

So for example the mail or file or other services at PARC just lived in the network; you did the equivalent of an anycast to check your mail or look for a file. These had standardized APIs so it didn’t matter if you were running Smalltalk, Interlisp-D, or Cedar/Mesa you just had a local window into a general computing space, just as you do today.

Most was on the LAN, of course, as the ARPANET was pretty slow. But when we switched to TCP/IP the LAN/WAN boundaries became transparent and instead of manually bouncing through different machines I could casually check my mail at MIT from my desk at PARC.

Lispms were slightly less flexible in this regard back then, but then again Ethernet started at PARC. But even in the late 70s it wasn’t weird to have part of your computation run on a remote machine you weren’t logged into interactively.

The Unix guys at Berkeley eventually caught up with this (just look at the original sockets interface, very un-unixy) but they didn’t quite get it: I always laughed when I saw a sun machine running sendmail rather than trusting the network to do the right thing on its behalf. By the time Sun was founded that felt paleolithic to me.

Because I didn’t start computing until the late 70s I pretty much missed the whole removable media thing and was pretty much always network connected.

Re: Cyc: History's Forgotten AI Project

#100
I was born in late USSR and my father is software engineer. We had several books that were not available for "general public" (they were intended for libraries of science institutions). One of the book was, as I understand now, abridged translation of papers from some "Western" AI conference.

And there were description if EURISCO (with claims that it not only "win some game" but also that it "invented new structure of NAND-gate in silicon, used by industry now") and other expert systems.

One of the mentioned expert systems (without technical details) said was 2 times better in diagnose cancer than best human diagnostician of some university hospital.

And after that... Silence.

I always wonder, why did this expert system were not deployed in all USA hospitals, for example? If it is so good?

Now we have LLMs, but they are LANGUAGE models, not WORLD models. They predict distribution of possible next words. Same with images — pixels, not world concepts.

Looks like such systems are good for generating marketing texts, but can not be used as diagnosticians by definition.

Why did all these (slice of) world model approaches dead? Except Cyc, I think. Why we have good text generators and image generators but not diagnosticians 40 years later? What happens?..

Post reply on HN