Viewing profile — rryan

No longer the case

ehh, no? definitely not. https://kyutai.org/blog/2024-07-03-meet-moshi/

As a human emdash enjoyer, I'm pretty damn grumpy that everyone assumes I'm a chatbot now. :P

I think the super long timelines in automotive make it really hard to succeed. Really impressed with what ClearMotion accomplished given that

Don't make me tap the sign: There is no such thing as "bytes". There are only encodings. UTF-8 is the encoding most people are using when they talk about modeling "raw bytes" of te…

RMSNorm is pretty insigificant in terms of the overall compute in a transformer though -- usually the reduction work can be fused with earlier or later operations.

ML 101: Do not evaluate on the training data. Yes of course it can, because they fit in the context window. But this is an awful test of the model's capabilities because it was cer…

Thanks for everything, Rich. You inspired me repeatedly.

https://elevenlabs.io/ https://twelvelabs.io/ .... wat

Yea uh, don't know what was going on there but there are roles with direct reports that aren't in mountain view (unless you mean, like in 2004). NYC has 1000s of googlers.

This is ... not what I expected. It's basically wiring up pre-trained models to ChatGPT via a router and "modality transformations" (a.k.a speech-to-text and text-to-speech). I exp…

Except when your doctor won't prescribe it you offlabel because you don't have a condition they were taught about in med school

In the past few months I've been getting much more fake order / payment spam.

I agree that a Transformer is an example of a "reflexive" behavior because it learns to react in a context (via gradient descent rather than evolution as the learning algorithm). I…

There's a huge difference between fitting a probabilistic model to a data distribution then sampling from it (what GPT-3 is) and agents that invent language and use it to communica…

You're using a pointwise loss, which means you are treating each timepoint as conditionally independent. That's a deeply flawed assumption, probabilistically. Try an autoregressive…

"I could go pro, but nah" -- I read this and closed the tab. What a whingefest from a free tier user. The Reddit crawling problem is because Reddit rate limits their crawling so th…

That makes no sense. Why is "YouTube Premium" capitalized?

> The main source of income for youtube isn't ads. YouTube revolves around the merchandise and YouTube Premium subscriptions. > If you are talking about creators who are not earnin…

N=20

* YouTube Kids with allowlist. * App Timer of zero minutes on YouTube app.

Straw man argument. OP was suggesting you pay for YT premium, which does away with the ads completely. Removing your web adblocker is a non sequitur, and exposes your computer to m…