Viewing profile — halflings

> Youtube charges $10 per month and doesn't produce a single video It is different from Netflix (that pays upfront for production costs), but there's of course a revenue share + th…

LLMs generate their output one token at a time. The first thought when you learn this is that this is a huge performance bottleneck, as we are used to highly parallelized systems. …

The main thing I noticed in the video is that they have heavily sped up all the code generation sections... seems to be on 5x speed or more. (because people got used to how fast an…

Deploying from Antigravity is as easy as say connecting the Firebase MCP [1] and asking it "deploy my app to firebase". [1] https://firebase.google.com/docs/ai-assistance/mcp-serve…

+1, reading through the post, the PR updating the documentation... thanks for being transparent, but also don't be so hard on yourself! That was a very niche error, that you prompt…

"The models perform differently when called via the API vs in the Gemini UI." This shouldn't be surprised, e.g. the model != the product. The same way GPT4o behaves differently tha…

I would also add that search has already moved elsewhere. Less and less people are using search engines to shop, ex:Amazon makes >$57B a year from search ads, but also look at Temu…

That's what the chart says yes. 14.1GB VRAM usage for the 27B model.

> I cannot be the first person to think about such possibilities Differentiable Rendering [1] is the closest thing to what you are describing. And yes, people have been working on …

Training code is only useful to people in academia, and the closest thing to "code you can modify" are open weights. People are framing this as if it was an open-source hierarchy, …

> The world is teetering on the edge of world war The world probably has never been as peaceful as in the last 50 years or so. Same goes for access to drinkable water, food, decent…

Thanks for putting this together! Will give it a watch now

The # of chips is not the most important metric. Most important, even ignoring latency, is throughput (tokens) per $$$. And according to their own benchmark [1] (famous last words …

No HBM because they use tons of fast SRAM instead. Isn't that the main driver for performance here? (the way I understood it => it's still cost effective at scale due to throughput…

Agree re:hallucinations/safety issues, that was likely one of the main blockers. And here's the sad part: they had this back in 2019... see this paper released in Jan 2020: https:/…

This (innovator's dilemma / too afraid of disrupting your own ads business model) is the most common explanation folks are giving for this, but seems to be some sort of post-ration…

uv has been really awesome as a replacement for pip: https://github.com/astral-sh/uv So fast it finally made virtual environments usable for me. But it's not (yet) a full replaceme…

This looks pretty cool! What's the catch? e.g. why isn't this already implemented in accelerators, is it really just a forgotten algorithm, or this has some implications on the cos…

> Your analogy is the same as early Intel engineers completely unaware that those chips would bring on the ramifications of social media Exactly! As they should be. (for both Intel…

This is the definition of strawman. "Advocate for killing all humans" sounds like someone advocating for a genocide, but instead it's just the same transhumanist thinking (which Yu…

Your comment (+ username) reads like what I would have written once upon a time when I was fully in the EA bubble. Truly no offense meant, as I was deeply into the EA movement myse…

The importance (e.g. attention) needs to be dynamic, e.g. one token will be important to some other tokens but not others. tf-idf and similar heuristics are what we were using befo…

Yep that's pretty much it! That's what they call needle in a haystack. See: https://github.com/gkamradt/LLMTest_NeedleInAHaystack