Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Paradigms of Artificial Intelligence Programming (1992)

github.com

Post reply on HN
Print view View original

51–60 of 83 posts

Re: Paradigms of Artificial Intelligence Programming (1992)

#51

Post by mirker »

taeric wrote: post #48
pgorczak wrote: post #45

Earlier quoted context omitted.

Neural nets do use extremely long compositions of nonlinear functions whose gradients need to be computed over and over. I guess because chain rule (automatic differentiation) works so well, most of the libraries in use are only “a little” symbolic in that they can automatically figure out first derivatives for symbolic input and output variables.

I thought automatic differentiation is a little different? Will have to look again. I thought the norm for a while was to manually enter the gradient, if possible, or to use a calculated one at each step.

The norm is to use library functions like addition, multiply, power, etc., which have hardcoded derivatives. Then the derivative is computed “symbolically” by composing these base derivatives, which is essentially an exercise of implementing calculator logic. I use scare quotes because there are numerical problems with some functions that have to be hacked around with more manual derivatives, like the cross-entropy/softmax function. In those cases, people suggest to use a different library function special-cased out for the situation, which is not really “symbolic” in the typical mathematical sense.

Re: Paradigms of Artificial Intelligence Programming (1992)

#52

Post by Jtsummers »

eternalban wrote: post #35

Apparently Perlis devoted some time to making epigrams. I like #119 (and have amended it in line with #122 and recent developments in Canada, et al) : Programming is an unnatural act, but so far it is still mostly legal . https://web.archive.org/web/19990117034445/http://www-pu.inf... p.s. Actually applying Perlis's epigrams to epigrams, we inevitably reach the conclusion that epigrams stifle thought yet #125 still h…

Wrong thread?

Re: Paradigms of Artificial Intelligence Programming (1992)

#53

Post by zmgsabst »

mark_l_watson wrote: post #43

I wrote a small Common Lisp book for Springer Verlag about the same time that Peter wrote this fantastic book and I then met him shortly thereafter at a Lisp Users Vendors conference in San Diego. After 30 years of following his writing, Python notebooks, etc., I think that he just has a higher level view of software and algorithms. There is some talk on this thread about good old fashioned symbolic AI. I have mixed…

We’re building that paradigm right now, in the math community.

Homotopy type theory tells us that semantic information from a symbolic logic can also be represented by the topology of diagrams. But this is a two way relationship: the topological structure of diagrams also corresponds to some synthetic type theory.

The conjecture is that the topology of, eg, word2vec point clouds will correspond to some synthetic type theory describing the data — and this is bolstered by recent advancements in ML: Facebook translating by aligning embedding geometries, data covering models, etc.

I’m personally working on the problem of translating types into diagrams stored as matrices, in the hope that building one direction will give insights into the other. (Again, because equivalence relationships are symmetric.)

Re: Paradigms of Artificial Intelligence Programming (1992)

#54

Post by vasili111 »

MAXPOOL wrote: post #32

One of the top 5 programming books. Old AI is today's bleeding edge computer engineering. There is an enourmous amount of free lunches for computer engineers and software startups in the old school artificial intelligence. * modern SAT solver performance is impressive. They can solve huge problems. * Writing a complex systems configurator with Prolog or Datalog can be like magic. * Expert systems. There has never bee…

Does this book teaches how to write SAT-solvers?

Re: Paradigms of Artificial Intelligence Programming (1992)

#55

Post by ianbicking »

taeric wrote: post #42
MontyCarloHall wrote: post #27

It is interesting how almost none of these paradigms/principles are relevant to what we think of as AI today. Lisp, a language specifically formulated for AI applications, is now mostly relevant in the context of programming language theory (and essentially irrelevant to the statistical programming/linear algebra toolkits that underpin modern AI applications). Instead, Fortran and its descendants are actually what po…

I'm always surprised that the symbolic math paradigms aren't used more. I suppose most cost functions just aren't continuous? That said, seems most of what is important in today's code is the ability to drive a gpu. Don't know why that couldn't be done from a lisp.

I'm not sure if anything has or will happen with this, but this seemed like an interesting symbolic extension of neural nets: https://syncedreview.com/2020/11/19/facebook-building-automa...

At least my take on it is: what if you simulate a domain (physics being the most tractable) then can you train using differentials of the model itself and not just differentiating the internal neuron estimators?

Lots of open questions... like what's the driving force that asks for an effect that calls for differentiating the model? We kind of expect the training process to just recreate everything internally.

Now it has me thinking about "coprocessors"... a lot of the domain-specific math is given to the NN as input features, but what if the NN could select inputs to those algorithm? That would give the NN the ability to speculate. Of course you'd need to also differentiate those algorithms.

Now that I'm thinking about it, I guess this is just another activation function, just one that is domain-specific. Are there approaches that use eclectic activation functions? Like in each layer 1-10 is one activation function, 11-20 is another, and then you finish off with tanh or relu or some other genetic activation function.

Re: Paradigms of Artificial Intelligence Programming (1992)

#56

Post by Jach »

wodenokoto wrote: post #2

Can anyone recommend a source for setting up an environment in Linux to run the code in this book? Do I need to learn eMacs to get close to a “modern lisp” programming environment?

Emacs is popular. I like vim, but people are also using VS Code. https://lispcookbook.github.io/cl-cookbook/editor-support.ht... and other pages on the wiki may help you. https://github.com/CodyReichert/awesome-cl#community has a list of community spots if you want to seek out other opinions.

The SBCL implementation is very good, consider getting a binary directly from their site if your distro's version is out of date http://www.sbcl.org/

I disagree with a sibling comment that this book expects you to be comfortable with Lisp; the first chapter is literally an introduction, and the next two chapters cover most of the basics a working programmer should expect to cover quickly with chapter 3 being a handy reference to look back on for a lot of things. If you're new to programming or find the intro too fast, sure, look at other resources, but it's not too bad to just dive in. The main supplement is to figure out, with your editor of choice, how to send blocks of Lisp code to the Lisp prompt so that you can type and edit with an editor and not have to do everything directly on the prompt line.

Re: Paradigms of Artificial Intelligence Programming (1992)

#57

Post by kazinator »

math-dev wrote: post #6
andrejlamov wrote: post #5

Earlier quoted context omitted.

do you think elisp will work for this book? or would one need to implement unique common lisp features in elisp? or will elisp just be too slow

most of it should work in elisp, note that elisp and common lisp have different semantics for IF. you might find some variances later on. also elisp doesn’t have as nice a debugging environment as SLIME so I’d honestly recommend just setting up common lisp

As far as I can tell, any Elisp if-form that is also a valid Common Lisp if-form has the same semantics; but Elisp allows more than one else-form in its syntax, with additional semantics.

It's something like:

  (defmacro elisp-if (expr then &rest elses)
    `(cond (,expr ,then) (t ,@elses)))
Whereas the regular if is like:

  (defmacro cl-if (expr then &optional else)
    `(cond (,expr ,then) (t ,else)))
In a book that uses Common Lisp, you shouldn't see an if-form that isn't Elisp-compatible.

Re: Paradigms of Artificial Intelligence Programming (1992)

#58

Post by EddySchauHai »

larve wrote: post #47
davidhs wrote: post #40

Earlier quoted context omitted.

What are the other 4 top programming books?

Since I also put PAIP in my top 5 books, here are my 4 other ones: - The Pragmatic Programmer - This changed my life when I started programming 20 years ago. Most of the practices are now common, but it was almost radical back then. - Designing Data Intensive Applications This is so well written, so elegantly fundamental. It's an absolute pleasure, even if I don't really refer to it in practice. - Structure and Inter…

DDIA, SICP, and PAIP have been on my reading lists for years. Maybe I’ll finally get round to them after this glowing review :)

Re: Paradigms of Artificial Intelligence Programming (1992)

#60

Post by jaggirs »

MAXPOOL wrote: post #32

One of the top 5 programming books. Old AI is today's bleeding edge computer engineering. There is an enourmous amount of free lunches for computer engineers and software startups in the old school artificial intelligence. * modern SAT solver performance is impressive. They can solve huge problems. * Writing a complex systems configurator with Prolog or Datalog can be like magic. * Expert systems. There has never bee…

SAT for initializing nn parameters? Can you elaborate on this a bit, it seems interesting.
Post reply on HN