Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Smallest Brain You Can Build: A Perceptron in Python

ranpara.net

Post reply on HN
Print view View original

71–80 of 86 posts

Re: The Smallest Brain You Can Build: A Perceptron in Python

#71

Post by WarmWash »

rahen wrote: post #29

In the early days of machine learning (before the first AI winter), networks like this were often implemented and trained in hardware: https://en.wikipedia.org/wiki/ADALINE That was the first thing that came to mind when I read "the smallest brain you can build ". Nowadays, that "small brain" would likely be built on a breadboard using op-amps instead.

The quasi-mythical memristor would be choice for bread boarding a brain. However I suppose you could train a model and then manually place fixed resistors to build the network

Re: The Smallest Brain You Can Build: A Perceptron in Python

#72

Post by WarmWash »

rahen wrote: post #60
Schlagbohrer wrote: post #52

Earlier quoted context omitted.

Amazing and anachronistic to see something like that from 1960. And then it makes me wonder why there wasn't more progress on neural nets being used for many things prior to the 21st century. (I haven't read the history of the AI winters but I have heard of them)

The first AI winter was largely triggered by Minsky in a book he published in 1969, which mathematically proved that single-layer perceptrons couldn't solve non-linear problems. Favorite quote: "Our intuitive judgment is that the extension [to multilayer systems] is sterile." Yet we had the computational power to run backpropagation in the 1960s and small Transformers in the 1970s (I'm the author of both): https://gi…

I wonder had we invented transformer architecture back in the 70's or 80's, if the pace of hardware innovation would have naturally slowed AI progression, and given humans decades to slowly adapt, rather than the current tidal wave (that seems to grow in size daily) bearing down on us.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#73

Post by DevarshRanpara »

romaniv wrote: post #59

I think it should be quite obvious that perceptrons are far from the smallest units that are capable of learning. They store many bytes of information, require a non-local update process, need numeric (i.e. symbolic) inputs and involve relatively complex computations. You can go much simpler. For example: https://medium.com/@VictorBanev/the-simplest-learning-machin... This is a description of a 5-line algorithm that…

True, there can be simpler versions compared to perceptron, just like you made. I have learned something new from that, Thanks for sharing.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#74

Post by DevarshRanpara »

Lerc wrote: post #47
kzrdude wrote: post #45

I think Karpathy's microgpt blogpost is the best in this genre in a long time, and it also includes a multi layer perceptron. It's a step up in the hierarchy, so reading both is helpful, of course. https://karpathy.github.io/2026/02/12/microgpt/

I'm not sure if I'd like to declare a best. There are so many different approaches and I think their ability to inform is cumulative, I like the ability of this article to do the tiny training runs in browser. It makes the point of a bias clear. Too many tutorials get sucked into the proof of zero times anything is zero. Everyone knows that. What you should show is where that mstters in the problem at hand. 3blue1bro…

I can't agree more with you, It took me many days to understand the "By we need bias?" I know maths, I know programming, but why was not clear. I love 3blue1brown.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#75

Post by DevarshRanpara »

utopiah wrote: post #69

One day I'll write about my 1-liner physics engine... let gravity = setInterval( _ => { if (projectile.object3D.position.y > 0) projectile.object3D.position.y \*= .99 }, 100) Jokes aside I find that providing ridiculously short toy examples that provide the very limited foundation of a concept are extremely empowering in pedagogy. You "get" it right away because it "fits" in your mind, then you dare tinker with it an…

Yeah, I will try to make more of these, I like to lean things from core, and I like to keep everything as simple as possible.

Re: The Smallest Brain You Can Build: A Perceptron in Python

#77

Post by SubiculumCode »

I wish that the tutorial went just one more step. It presents a one dimensional perceptron. But most perceptrons are multi-input. Adapting the article's 1D perceptron to three-input, for example:

    import random

    learning_rate = 0.1
    EPOCHS = 50
    NUM_INPUTS = 3

    weights = [random.uniform(-1, 1) for _ in range(NUM_INPUTS)]
    bias = random.uniform(-1, 1)

    data = []
    for _ in range(100):
        inputs = [random.uniform(-1, 1) for _ in range(NUM_INPUTS)]
        result = sum(inputs) > 0
        data.append((inputs, result))

    for epoch in range(EPOCHS):
        for inputs, result in data:
            weighted_sum = bias
            for i in range(NUM_INPUTS):
                weighted_sum += inputs[i] * weights[i]
            
            prediction = weighted_sum > 0
            
            if prediction != result:
                error = int(result) - int(prediction)
                for i in range(NUM_INPUTS):
                    weights[i] += learning_rate * error * inputs[i]
                bias += learning_rate * error

    print(f"Final weights: {[round(w, 3) for w in weights]}")
    print(f"Final bias: {round(bias, 3)}")
Post reply on HN