Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Microgpt

karpathy.github.io

Post reply on HN
Print view View original

291–300 of 354 posts

Re: Microgpt

#291

Post by OtherShrezzing »

coldtea wrote: post #281
vadimf wrote: post #259

I’m 100% sure the future consists of many models running on device. LLMs will be the mobile apps of the future (or a different architecture, but still intelligence).

If anything, memory ain't getting cheaper, disks aren't either, and as for graphics cards, forget it. People wont be competing with even a current 2026 SOTA from their home LLM nowhere soon. Even actual SOTA LLM providers are not competing either - they're losing money on energy and costs, hopping to make it up on market capture and win the IPO races.

I don’t think anyone needs to compete with the LLM SOTA to get the benefits of these technologies on-device.

Consumers don’t need a 100k context window oracle that knows everything about both T-Cells and the ancient Welsh Royal lineage. We need focused & small models which are specialised, and then we need a good query router.

Re: Microgpt

#292

Post by CamperBob2 »

Jolter wrote: post #273
matheusmoreira wrote: post #266

Earlier quoted context omitted.

> One of us is an advanced autocomplete engine. The other is a human, capable of making judgements on what is conscious and what is not. What evidence is there that your "judgements" are anything other than advanced autocompletion? Concepts introduced into a self-training wetware CPU via its senses over a lifetime in order to predict tokens and form new concepts via logical manipulation? > Your philosophizing about s…

Surely you’re not seriously saying that you believe AI agents, in their current state of the art, meet whatever criteria you have for being ”alive”? That’s kind of how you’re coming across. I don’t really know how to respond to that, because it’s so preposterous.

You didn't answer the question.

Re: Microgpt

#293

Post by tomjakubowski »

cowlby wrote: post #241
rapnie wrote: post #223

Earlier quoted context omitted.

Yesterday an interesting video was posted "Is AI Hiding Its Full Power?", interviewing professor emeritus and nobel laureate Geoffrey Hinton, with some great explanations for the non-LLM experts. Some remarkable and mindblowing observations in there. Like saying that AI's hallucinate is incorrect language, and we should use "confabulation" instead, same as people do too. And that AI agents once they are launched deve…

Isn’t the sustainability drive a function of how much humans have written about life and death and science fiction including these themes?

Humans, like all animals, have instinctual and biological drives to survive besides, but it's interesting to think how much of our drive to survive is culturally transmitted too.

Re: Microgpt

#294

Post by jonjacky »

I wonder if such a small GPT exhibits plagiarism. Are some of the generated names the same as names in the input data?

Re: Microgpt

#295

Post by camkego »

teleforce wrote: post #94

Someone has modified microgpt to build a tiny GPT that generates Korean first names, and created a web page that visualizes the entire process [1]. Users can interactively explore the microgpt pipeline end to end, from tokenization until inference. [1] English GPT lab: https://ko-microgpt.vercel.app/

I have no affiliation with the website, but the website is pretty neat if you are learning LLM internals. It explains: Tokenization, Embedding, Attention, Loss & Gradient, Training, Inference and comparison to "Real GPT"

Pretty nifty. Even if you are not interested in the Korean language

Re: Microgpt

#296

Post by chongli »

geokon wrote: post #141

> What’s the deal with “hallucinations”? The model generates tokens by sampling from a probability distribution. It has no concept of truth, it only knows what sequences are statistically plausible given the training data. Extremely naiive question.. but could LLM output be tagged with some kind of confidence score? Like if I'm asking an LLM some question does it have an internal metric for how confident it is in its…

Having a confidence score isn't as useful as it seems unless you (the user) know a lot about the contents of the training set.

Think of traditional statistics. Suppose I said "80% of those sampled preferred apples to oranges, and my 95% confidence interval is within +/- 2% of that" but then I didn't tell you anything about how I collected the sample. Maybe I was talking to people at an apple pie festival? Who knows! Without more information on the sampling method, it's hard to make any kind of useful claim about a population.

This is why I remain so pessimistic about LLMs as a source of knowledge. Imagine you had a person who was raised from birth in a completely isolated lab environment and taught only how to read books, including the dictionary. They would know how all the words in those books relate to each other but know nothing of how that relates to the world. They could read the line "the killer drew his gun and aimed it at the victim" but what would they really know of it if they'd never seen a gun?

Re: Microgpt

#297

Post by red75prime »

jaen wrote: post #183
amelius wrote: post #181

Earlier quoted context omitted.

> In short: LLM have no concept, or even desire to produce of truth They do produce true statements most of the time, though.

That's just because true statements are more likely to occur in their training corpus.

The overwhelming majority of true statements isn't in the training corpus due to a combinatorial explosion. What it means that they are more likely to occur there?

Re: Microgpt

#298

Post by danlitt »

geokon wrote: post #141

> What’s the deal with “hallucinations”? The model generates tokens by sampling from a probability distribution. It has no concept of truth, it only knows what sequences are statistically plausible given the training data. Extremely naiive question.. but could LLM output be tagged with some kind of confidence score? Like if I'm asking an LLM some question does it have an internal metric for how confident it is in its…

Can it generate one? Sure. But it won't mean anything, since you don't know (and nobody knows) the "true" distribution.

Re: Microgpt

#299

Post by nickpsecurity »

joefourier wrote: post #204
geon wrote: post #202

Is there a similarly simple implementation with tensorflow? I tried building a tiny model last weekend, but it was very difficult to find any articles that weren’t broken ai slop.

Tensorflow is largely dead, it’s been years since I’ve seen a new repo use it. Go with Jax if you want a PyTorch alternative that can have better performance for certain scenarios.

Also, TPU support. Hardware diversity.
Post reply on HN