Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

github.com

Post reply on HN
Print view View original

81–90 of 126 posts

Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

#81

Post by homebrewer »

abanana wrote: post #63
simonjgreen wrote: post #38

Earlier quoted context omitted.

Super intrigued but annoyingly I can’t view imgur here

Indeed, part of me wants to not use imgur because we can't access it, but a bigger part of me fully supports imgur's decision to give the middle finger to the UK after our government's censorship overreach.

It blocks many more countries than just the UK because it's the lowest effort way of fighting "AI" scrapers.

imgur was created as a sort of protest against how terrible most image hosting platforms were back then, went down the drain several years later, and it's now just like they were.

Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

#84

Post by dangus »

ben_w wrote: post #65
dangus wrote: post #45

Earlier quoted context omitted.

> It won't write your emails, but it can be trained to play a stripped down version of 20 Questions, and is sometimes able to maintain the illusion of having simple but terse conversations with a distinct personality. You can buy a kid’s tiger electronics style toy that plays 20 questions. It’s not like this LLM is bastion of glorious efficiency, it’s just stripped down to fit on the hardware. Slack/Teams handles com…

> Slack/Teams handles company-wide video calls and can render anything a web browser can, and they run an entire App Store of apps, all from a cross-platform application. The 4th Gen iPod touch had 256 meg of RAM and also did those things, with video calling via FaceTime (and probably others, but I don't care). Well, except "cross platform", what with it being the platform.

Group FaceTime calls didn’t exist at the time. That wasn’t added until 2018 and required iOS 12.

Remember that Slack does simultaneous multiple participants screen sharing plus annotations plus HD video feeds from all participants plus the entirety of the rest of the app continues to function as if you weren’t on a call at all simultaneously.

It’s an extremely powerful application when you really step back and think about it. It just looks like “text” and boring business software.

Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

#85

Post by dangus »

messe wrote: post #48
dangus wrote: post #45

Earlier quoted context omitted.

> It won't write your emails, but it can be trained to play a stripped down version of 20 Questions, and is sometimes able to maintain the illusion of having simple but terse conversations with a distinct personality. You can buy a kid’s tiger electronics style toy that plays 20 questions. It’s not like this LLM is bastion of glorious efficiency, it’s just stripped down to fit on the hardware. Slack/Teams handles com…

> can render anything a web browser can That's a bug not a feature, and strongly coupled to the root cause for slack's bloat.

One person’s “bloat” is another person’s “critical business feature.”

The app ecosystem of Slack is largely responsible for its success. You can extend it to do almost anything you want.

Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

#86

Post by Dylan16807 »

giancarlostoro wrote: post #78

This is something I've been wondering about myself. What's the "Minimally Viable LLM" that can have simple conversations. Then my next question is, how much can we push it so it can learn from looking up data externally, can we build a tiny model with an insanely larger context window? I have to assume I'm not the only one who has asked or thought of these things. Ultimately, if you can build an ultra tiny model that…

For your first question, the LLM someone built in Minecraft can handle simple conversations with 5 million weights, mostly 8 bits.

I doubt it would be able to make good use of a large context window, though.

Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

#87

Post by dangus »

andrepd wrote: post #61
dangus wrote: post #45

Earlier quoted context omitted.

> It won't write your emails, but it can be trained to play a stripped down version of 20 Questions, and is sometimes able to maintain the illusion of having simple but terse conversations with a distinct personality. You can buy a kid’s tiger electronics style toy that plays 20 questions. It’s not like this LLM is bastion of glorious efficiency, it’s just stripped down to fit on the hardware. Slack/Teams handles com…

My Pentium 3 in 2005 could do chat and video calls and play chess and send silly emotes. There is no conceivable user-facing reason why in 20 years the same functionality takes 30× as many resources, only developer-facing reasons. But those are not valid reasons for a professional. If a bridge engineer claims he now needs 30× as much concrete to build the same bridge as he did 20 years ago, and the reason is his/her…

I have great doubts that you were doing simultaneous screen sharing from multiple participants with group annotation plus HD video in your group calls, all while supporting chatting that allowed you to upload and view multiple animated gifs, videos, rich formatted text, reactions, slash command and application automation integrations, all simultaneously on your Pentium 3.

I would be interested to know the name of the program that did all that within the same app during that time period.

For some reason Slack gets criticism for being “bloated” when it basically does anything you could possibly imagine and is essentially a business communication application platform. Nobody can actually name a specific application that does everything Slack does with better efficiency.

Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

#88

Post by fho »

giancarlostoro wrote: post #78

This is something I've been wondering about myself. What's the "Minimally Viable LLM" that can have simple conversations. Then my next question is, how much can we push it so it can learn from looking up data externally, can we build a tiny model with an insanely larger context window? I have to assume I'm not the only one who has asked or thought of these things. Ultimately, if you can build an ultra tiny model that…

You might be interested in RWKV: https://www.rwkv.com/

Not exactly "minimal viable", but a "what if RNNs where good for LLMs" case study.

-> insanely fast on CPUs

Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

#89

Post by andy12_ »

giancarlostoro wrote: post #78

This is something I've been wondering about myself. What's the "Minimally Viable LLM" that can have simple conversations. Then my next question is, how much can we push it so it can learn from looking up data externally, can we build a tiny model with an insanely larger context window? I have to assume I'm not the only one who has asked or thought of these things. Ultimately, if you can build an ultra tiny model that…

This is extremely similar to Karpathy's idea of a "cognitive core" [1]; an extremely small model with near-0 encyclopedic knowledge and basic reasoning and tool-use capabilities.

[1] https://x.com/karpathy/status/1938626382248149433

Re: Show HN: Z80-μLM, a 'Conversational AI' That Fits in 40KB

#90

Post by cwmoore »

nrhrjrjrjtntbt wrote: post #57
Peteragain wrote: post #33

There are two things happening here. A really small LLM mechanism which is useful for thinking about how the big ones work, and a reference to the well known phenomenon, commonly dismissively referred to as a "trick", in which humans want to believe. We work hard to account for what our conversational partner says. Language in use is a collective cultural construct. By this view the real question is how and why we hu…

MAYBE

Universally correct reply, although honestly a bit vague.
Post reply on HN