How to build a thinking AI • Hacker News Board

Re: How to build a thinking AI

#71

Post by pstorm » Sat, Jan 06, 2024, 6:58 AM UTC

dTal wrote: post #66

hathawsh wrote: post #41

Earlier quoted context omitted.

I like to think we could quickly create a next-level AI (maybe AGI?) if we simply model it on the Pixar movie "Inside Out". The little characters inside the girl's brain are different LLMs with different biases. They follow a kind of script that adapts to the current environment. They converse with each other and suggest to the girl what she should do or say. I'd try the idea myself, but I have a job. :-)

This sounds a lot like the mixture-of-experts architecture, and the current best-performing language models (GPT-4, mixtral-8x7b) already use this architecture. So congratulations, you win!

That's not really how MoEs work. They never directly interact with eachother. There is one manager type model that takes a prompt, directs token inference to 1 or more models, chooses the best response, and continues. The analogy would be closer to a "swarm of agents". (There are a handful of names for this approach, I think swarm is catching on the most)

Re: How to build a thinking AI

#72

Post by tudorw » Sat, Jan 06, 2024, 9:52 AM UTC

layer8 wrote: post #14

While I agree that autonomous iteration will be important to AGI, I somehow have trouble taking an author serious who presents tables as screenshot images containing red spell-checking squiggles.

You realise you meant to write 'seriously', I will only consider this comments validity once this error has been corrected, also that last part of your comment is not well written it should be 'presents tables containing red spell-checking squiggles as screenshot images'. You're welcome.

Re: How to build a thinking AI

#73

Post by TaupeRanger » Sat, Jan 06, 2024, 2:21 PM UTC

blackbear_ wrote: post #38

TaupeRanger wrote: post #11

Earlier quoted context omitted.

It's the same hyperbolic nonsense we've seen from hundreds of other confident "researchers" over the past 50 years. Eliasmith has a book called "How to Build a Brain", Hawkins built an entire company, Numenta, around a theory that hasn't created anything remotely useful or interesting in almost 2 decades and has pivoted to creating tools for current ML zeitgeist methods. This unknown researcher is exactly the same. W…

You really don't need to be so cynical, some things just need time. Light bulbs were patented only after 40 years of work by multiple researchers, and it took another half a century of work to achieve decent efficiency. Neural networks themselves have been in development for 50 years before taking off recently, and for most of that time people working in the field were considered nuts by their peers. But if you have…

I'm not cynical. I'm just tired of people making wild claims which inevitably don't pan out. Neural networks are the exact opposite of this. They became popular because people SHOWED that they were useful. The early papers on perceptrons and later feedforward NNs SHOWED that they are capable of solving classification problems at the very least. Those mentioned in my original reply have shown NOTHING for all their grandiose claims about human brains and "thought". If your theory works, then go ahead, IMPLEMENT HUMAN THOUGHT. Until then, stop writing about how you "solved" one of the deepest and most profound mysteries in human history.

Re: How to build a thinking AI

#74

Post by pfisch » Sat, Jan 06, 2024, 5:34 PM UTC

kelseyfrog wrote: post #62

pfisch wrote: post #3

At best we invite a dystopian future. At worst our own annihilation. It is crazy how powerless we are to stop it from happening.

State your priors, rationalist.

AI is going to replace most non physical jobs. Right now it is a helper, but soon it will be much more. I also think that the way it allows consolidation of power while hollowing out the working middle and upper class is going to lead to anti-democratic outcomes.