Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Smallest Brain You Can Build: A Perceptron in Python

ranpara.net

Post reply on HN
Print view View original

41–50 of 86 posts

Re: The Smallest Brain You Can Build: A Perceptron in Python

#41

Post by lioeters »

adrian_b wrote: post #36
zkmon wrote: post #26

The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model. Going further backwards, the transistor (or a PN…

The alternative IF expression or statement, in the form introduced by John McCarthy in 1958 (which he used in early 1959 to define his version of AND, OR and NOT), is one of the possible primitives for computation. There exist several equivalent sets of primitive operations. While the sets containing only NAND or only NOR, or both AND and NOT or both OR and NOT are more notorious, these logical operations are more ab…

This reminds me of a book, whose title I forget, about creating a compact set of knowledge that will enable us to "rebuild civilization from scratch" in case of a future/fictional post-collapse scenario. I sometimes wonder, given the immense complexity and global supply-chain dependencies of our computing stack, whether we could bootstrap from first principles something of equivalent power and expressivity, but orders of magnitude simpler.

There's a wide variety of computational primitives, including lambda calculus, combinators, cellular automata, rewriting systems. Perhaps some are more practical to implement in hardware, particularly the kind of DIY electronics or analog machines that can also be put together from scratch. It might look like a whole building of mechanical switches, powered by a water wheel ("watermill"), for example.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#44

Post by tomtom1337 »

BatteryMountain wrote: post #35
zkmon wrote: post #26

The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model. Going further backwards, the transistor (or a PN…

So, what if, we build a stack/set of transistors in same shape as a trained model? It would eliminate most of the software stack too and should run very fast. No memory/gpu required, the chip acts as both storage and processing device, purpose built to be physical model of a trained model.

This is literally what talaas has done with chatjimmy.ai.

Try it, it's llama 3.1 8B at 16000 tokens per second.

chatjimmy.ai https://taalas.com/the-path-to-ubiquitous-ai/

Re: The Smallest Brain You Can Build: A Perceptron in Python

#47

Post by Lerc »

kzrdude wrote: post #45

I think Karpathy's microgpt blogpost is the best in this genre in a long time, and it also includes a multi layer perceptron. It's a step up in the hierarchy, so reading both is helpful, of course. https://karpathy.github.io/2026/02/12/microgpt/

I'm not sure if I'd like to declare a best. There are so many different approaches and I think their ability to inform is cumulative,

I like the ability of this article to do the tiny training runs in browser. It makes the point of a bias clear. Too many tutorials get sucked into the proof of zero times anything is zero. Everyone knows that. What you should show is where that mstters in the problem at hand.

3blue1brown does one of the best depictions of why we need an activation function.

Karpathy's videos are a little tougher for a beginner to grasp, but excel at solving a complete problem. I knew all of the theory behind what it takes to make micrograd before I made my own by following the video, but what you get from doing it can't be understated.

It's hard to describe but it what you learn is more of a feel than pure knowledge. It gives you a better sense of knowing when the principles apply in other circumstances.

Perhaps it's the distinction of understanding how springs and gears work, then looking at a clock and understanding how the gears and springs move the hands. There's still more needed if you want to make a clock. And that stuff is what let's you also make a wind up toy.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#48

Post by Lerc »

zkmon wrote: post #26

The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model. Going further backwards, the transistor (or a PN…

Fundimentally, when you talk about a if statement, you are talking about the ability to do something different dependent upon some state.

It's the same thing as stimulus, response.

Unchanging in response to circumstances is static.

Changing in the absence of circumstances is randomness.

The conditional is all that remains. Changing in response to circumstances

(Arguably, unchanging in the absence of circumstances completes the truth table, but it's a whole lot of nothing)

Re: The Smallest Brain You Can Build: A Perceptron in Python

#49

Post by rusk »

BatteryMountain wrote: post #35
zkmon wrote: post #26

The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model. Going further backwards, the transistor (or a PN…

So, what if, we build a stack/set of transistors in same shape as a trained model? It would eliminate most of the software stack too and should run very fast. No memory/gpu required, the chip acts as both storage and processing device, purpose built to be physical model of a trained model.

There’s lots of things you can do in hardware that could be done in software but cost. FPGA should have solved this long ago, but apparently the guys who own the IP want to make it as hard as possible to use it …

Re: The Smallest Brain You Can Build: A Perceptron in Python

#50

Post by Lerc »

Bimos wrote: post #24

> A perceptron *is* the smallest brain you can build. > In 1958, a researcher named Frank Rosenblatt built a machine *he called* the perceptron. > It was *inspired* by a single brain cell, a neuron.

I did a lecture once which included a 5 minute whirlwind tour of neural net history.

I included a remark about how time travellers would find Rosenblatt a better target than Miles Dyson.

I was never quite sure on how close, or over, the line that was on appropriateness. It was definitely thought provoking.

Post reply on HN