Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Smallest Brain You Can Build: A Perceptron in Python

ranpara.net

Post reply on HN
Print view View original

61–70 of 86 posts

Re: The Smallest Brain You Can Build: A Perceptron in Python

#61

Post by jupr »

tomtom1337 wrote: post #44
BatteryMountain wrote: post #35

Earlier quoted context omitted.

So, what if, we build a stack/set of transistors in same shape as a trained model? It would eliminate most of the software stack too and should run very fast. No memory/gpu required, the chip acts as both storage and processing device, purpose built to be physical model of a trained model.

This is literally what talaas has done with chatjimmy.ai. Try it, it's llama 3.1 8B at 16000 tokens per second. chatjimmy.ai https://taalas.com/the-path-to-ubiquitous-ai/

Wow that incredibly fast. I like this outcome more than centralized datacenters.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#62

Post by ninalanyon »

Is this something that could be scaled up and used, for instance, to recognize features in images?

Or to put it another way are there any local only tools that can be trained on my own set of images to automatically tag new images? Tools that do not already have built in classes of image.

I take a lot of photographs and it would be handy to reduce the drudgery of tagging them so to say broadly what the subject was so that they are easier to find later.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#63

Post by lioeters »

coldtea wrote: post #57
lioeters wrote: post #41

Earlier quoted context omitted.

This reminds me of a book, whose title I forget, about creating a compact set of knowledge that will enable us to "rebuild civilization from scratch" in case of a future/fictional post-collapse scenario. I sometimes wonder, given the immense complexity and global supply-chain dependencies of our computing stack, whether we could bootstrap from first principles something of equivalent power and expressivity, but order…

I think if civilization collapses they'll have other priorities, and very little benefit to get from the effort required for such simpler computing, for a good while...

I suppose people will have more important things to work on, like growing food and fighting off roving bands of bandits, than building primitive analog calculators/computers.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#64

Post by a1o »

romaniv wrote: post #59

I think it should be quite obvious that perceptrons are far from the smallest units that are capable of learning. They store many bytes of information, require a non-local update process, need numeric (i.e. symbolic) inputs and involve relatively complex computations. You can go much simpler. For example: https://medium.com/@VictorBanev/the-simplest-learning-machin... This is a description of a 5-line algorithm that…

That is a cool algorithm, indeed very interesting 5 lines. Also fun to see things in C#. :)

Re: The Smallest Brain You Can Build: A Perceptron in Python

#65

Post by lambertsimnel »

RetroTechie wrote: post #56
zkmon wrote: post #26

The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model. Going further backwards, the transistor (or a PN…

> Going further backwards, the transistor (or a PNP junction) is the hardware level enabler of the IF statement. The action (switching) (..) Back up a bit please! Analog computing is a thing. And it isn't even new - not by a long shot. There are good reasons why practically all computing today is the digital kind. But electronic 'equivalents' of neural nets is one area where analog might make sense. Adding inputs can…

> But electronic 'equivalents' of neural nets is one area where analog might make sense.

That's an interesting idea, but could the weights be transferred to different hardware and still work? If not, that would be a significant limitation, even if it were preferable in some cases.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#67

Post by mr_toad »

BatteryMountain wrote: post #35
zkmon wrote: post #26

The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model. Going further backwards, the transistor (or a PN…

So, what if, we build a stack/set of transistors in same shape as a trained model? It would eliminate most of the software stack too and should run very fast. No memory/gpu required, the chip acts as both storage and processing device, purpose built to be physical model of a trained model.

But it can only run that model, so it will be outdated in a few years at best.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#69

Post by utopiah »

One day I'll write about my 1-liner physics engine...

    let gravity = setInterval( _ => { if (projectile.object3D.position.y > 0) projectile.object3D.position.y \*= .99 }, 100)
Jokes aside I find that providing ridiculously short toy examples that provide the very limited foundation of a concept are extremely empowering in pedagogy. You "get" it right away because it "fits" in your mind, then you dare tinker with it and quickly see how limited it is, then get excited again. It's a powerful trick to learn more IMHO.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#70

Post by mr_toad »

zkmon wrote: post #26

The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model. Going further backwards, the transistor (or a PN…

An artificial neuron needs values to compare (the sum of weighted inputs). You can add values with transistors of course, but you need more than a dozen just to do simple addition. The activation function could be a simple binary comparison (e.g. between a weight and a threshold), but it’s usually more complicated.

Artificial neurons are significantly more complex that single transistors, and even a minimal hardwired circuit to implement just one neuron requires quite a number of transistors.

Post reply on HN