Ask HN: What would you recommend a vibe coder learn about how all this works?

[flagged]

What do you mean with "how all this works"? Do you want to know how program code works, or how LLMs turn natural language into code?

For the former, I think some hand coding will get the best results.

A large part of "thinking like a programmer" is thinking about the sort of invisible structures that code will produce in a computer's memory when it runs - which is not the same as what's literally written in the code file. IMO, the best way to get a feel for that is to write code yourself, run it, examine the results, etc.

But writing code can be a rather dry activity if you don't have motivation for what to write. The usual vibe coded projects are probably too complex to hand code without experience. I think two good starting points could be Code Katas [1] or the Pico8 game engine [2] (and of course any good "intro to Python" or "intro to JavaScript" tutorial on the web)

--

For the latter, you could maybe start with the history of generative text models (like markov chains) and neural networks. Both technologies are several decades old and sort of form the very basic building blocks that made development of LLMs possible.

In particular, neural networks made use of the gradient descent and backpropagation algorithms that are still the basis of most LLM training processes - even if they might now run for months and take exabytes of training data.

Then there is of course the "Attention is all you need" paper that introduced the Transformer architecture - the kind of latest-generation neural network that underpins most LLMs - and basically started the whole LLM revolution. (It's also the "T" in GPT)

(For historical interest, there is also the whole theory of formal languages/grammars and automata - which was the previous attempt to understand human language with computers. That theory gave us a lot of things - such as the Turing Machine and the entire theory of how to build and read a programming language. What it didn't give us though was a way to process human language...)

--

As for the specific question of how an LLM can take this request in human language and generate that code from it that happens to fullfill all the requirements - that's an active area of research and I honestly doubt that right now anyone understands that in all entirety.

Through I believe Anthropic and LessWrong had some interesting blog posts about this topics where they tried to understand the functions of certain specific neurons in an LLM. See also "Golden Gate Claude".

[1] http://codekata.com/

[2] https://www.lexaloffle.com/pico-8.php

You're not getting it. Making app is a solved problem, especially if app function, features, and purpose is derivative of existing things,

Think of it like image generation AI. You can make acceptable if sloppy art with it, using styles that exist. However, you cannot create a new style. You cannot create pictures of things that are truly novel, to do that you have to pick up the brush yourself.

coding with llms is the exact same thing. It can give you copies of what exists, and sometimes reasonable interpolations/mashups, but i have not seen a single succesful example of extrapolation. Not one. You simply leave the learned manifold and everything gets chaotic like in the example i provided.

If AI can make what you want, then the thing you made is not as novel as you thought. You're repurpising solved problems. Still useful, still interesting, just not as ground breaking as the bot will try and tell you.

Honestly scrap AI and do tutorials. Then for each exercise try doing it in a blank folder with nothing to refer to. Then play with the code. Refer to docs only after trying. Yes, like Karate kid. Wax on. Wax off.

Use AI for building still. But have non AI sessions for learning. Good luck!

This seems like the most helpful response for someone like OP. Is the curriculum or syllabus of your class public?