[flagged]
Microgpt
111–120 of 354 posts
Re: Microgpt
#112Super useful exercise. My gut tells me that someone will soon figure out how to build micro-LLMs for specialized tasks that have real-world value, and then training LLMs won’t just be for billion dollar companies. Imagine, for example, a hyper-focused model for a specific programming framework (e.g. Laravel, Django, NextJS) trained only on open-source repositories and documentation and carefully optimized with a spec…
We had good small language models for decades. (E.g. BERT) The entire point of LLMs is that you don't have to spend money training them for each specific case. You can train something like Qwen once and then use it to solve whatever classification/summarization/translation problem in minutes instead of weeks.
> The entire point of LLMs is that you don't have to spend money training them for each specific case.
I don’t agree. I would say the entire point of LLMs is to be able to solve a certain class of non-deterministic problems that cannot be solved with deterministic procedural code. LLMs don’t need to be generally useful in order to be useful for specific business use cases. I as a programmer would be very happy to have a local coding agent like Claude Code that can do nothing but write code in my chosen programming language or framework, instead of using a general model like Opus, if it could be hyper-specialized and optimized for that one task, so that it is small enough to run on my MacBook. I don’t need the other general reasoning capabilities of Opus.
Re: Microgpt
#113I'm half shocked this wasn't on HN before? Haha I built PicoGPT as a minified fork with And it's small enough to run from a QR code :)
https://kuber.studio/picogpt/
You can quite literally train a micro LLM from your phone's browser
Re: Microgpt
#114[flagged]
Imagine the people on here spraying their AI takes everywhere while being this oblivious, the code is more or less a standard assignment in all Deep Learning courses. The "reasoning" is two matrix transformations based on how often words appear next to each other.
Re: Microgpt
#115"everything else is just efficiency" is a nice line but the efficiency is the hard part. the core of a search engine is also trivial, rank documents by relevance. google's moat was making it work at scale. same applies here.
Sure, but understanding the core concepts are essential to make things efficient and as far as I understand, this has mainly educational purposes ( it does not even run on a GPU).
yep, agreed. wasn’t knocking the project at all, it’s great for exactly that purpose
Re: Microgpt
#116It’s pretty staggering that a core algorithm simple enough to be expressed in 200 lines of Python can apparently be scaled up to achieve AGI. Yes with some extra tricks and tweaks. But the core ideas are all here.
LLMs won’t lead to AGI. Almost by definition, they can’t. The thought experiment I use constantly to explain this: Train an LLM on all human knowledge up to 1905 and see if it comes up with General Relativity. It won’t. We’ll need additional breakthroughs in AI.
> Train an LLM on all human knowledge up to 1905 and see if it comes up with General Relativity. It won’t.
AGI just means human level intelligence. I couldn't come up with General Relativity. That doesn't mean I don't have general intelligence.
I don't understand why people are moving the goalposts.
Re: Microgpt
#117I'm half shocked this wasn't on HN before? Haha I built PicoGPT as a minified fork with And it's small enough to run from a QR code :) https://kuber.studio/picogpt/ You can quite literally train a micro LLM from your phone's browser
Re: Microgpt
#118This could make an interesting language shootout benchmark.
[flagged]
Re: Microgpt
#119[flagged]
Human internet is dead. I don't know how we can come back from this.