Viewing profile — xscott

> it unrealistically assumes America still has soft power I did caveat by saying, "where it can". I'm very unhappy with what we've been doing here, but until you Canadians bail out…

Lol, I appreciate your cynicism here, but I think it could realistically go worse than that: - OpenAI and Anthropic convince the US gov to ban open weight models, through bribes or…

Yeah, there's room for improvement at every level, but your specific example: How do you get more and more difficult tasks where you can steer the training? To me, that seems limit…

> Don't forget the whole debacle over Fable 5 sabotaging the user for "advanced frontier AI development". Yeah, I've had that happen twice. The second time was about some attention…

Lol, I still think about buying that now, even after the price hike. FOMO.

I've got nothing but hand-waving, but after you've extracted all the smarts from every piece of text ever created, how do you get more? Alpha Go had a game where the models could c…

It's worse though, because you can't really watch them at all. It's very difficult to get quantitative numbers for quality. Even within the same model family, same tokenizer, and c…

There are lots of points in a spectrum of choices. DGX Sparks, Strix Halos, and the surviving Mac Studios can easily run these 30B class models, just not as fast. So maybe just the…

I kick myself a couple times a week for not getting the 512GB Mac Studio in February. I was holding out for an M4 or M5 chip...

Not to mention all the other ways they can screw you: - Middle of the day, servers busy? Swap to Sonnet while pretending it's still Opus. Many people won't notice, and nobody can p…

You might find this video useful or interesting: https://youtu.be/by9lQvpvMIc I've been tempted to do something similar for browser canvas, perhaps with some slightly different cho…

I doubt he plans to convince Xi Jinping to decelerate too, so really this is talking to the Whitehouse about banning Chinese models.

I can't speak to the religious bits or history. That's certainly not my motivation for thinking about this stuff. It's not about computational convenience either. Both of those see…

No matter how small you go, between every two real numbers is a computable number, and between every two computable numbers is a real number that's not computable. If you restricte…

I'm cynical enough that I would suspect all of his statements are duplicitous anyway, but my recent experiences with Claude Fable give weight to it. I asked a question about a seri…

I'm on your side for most of what you say. This topic has been interesting to me for years. I've considered going back to school to build on my math degree, specifically because of…

All the terms are squishy, but being sloppy about it, I think there's intelligence that needs to be in the model, and knowledge that could live in a database. Right now, models are…

As far as I can tell, the good models will use whatever you give them. So it seems we should only give them language features that help humans understand and maintain what the mode…

Exploring compression algorithms for weights is a good idea, and I hope you have a successful product. However, if you can prove this statement: > reduces it down to its minimum en…

> we may need some mechanism to translate the code from terse way to the verbose way I completely agree. Nothing says humans need to read in exactly the syntax the model wrote. We …

Yeah, but sorting algs, FFTs, matrix factorizations, backprop, and many other things just aren't the same with purely functional data structures. Maybe these well known cases could…

> would you rather review LLM-written assembly or LLM-written Haskell? I wish we had a language that was targeted specifically for LLMs to write and humans and LLMs to inspect: - S…

There's a lot that's worth thinking about and discussing on this topic, but it's too loaded with emotional stuff for many people to hope for a productive discussion. I'm a programm…