Viewing profile — calebkaiser

So OpenAI would be proving to the government that they should be trusted to govern models, by failing to govern their models? Plenty of shady stuff goes on in marketing, and I'm su…

I think the commenter is saying that OpenAI committed a false flag operation, not Tailscale. For Tailscale, this may very well be marketing, but it would be strangely self defeatin…

I'm typically very skeptical of most content marketing/corporate PR. In the case of this incident, I struggle to see the clear upshot for OpenAI. It seems pretty unlikely they'd ev…

Implementing models directly from papers is typically pretty doable (and is of course more straightforward when the full implementation is open sourced). Often there is some amount…

OpenAI's head of strategic futures publicly stated that you can't explain the quality of the newest Kimi via distillation. Further, you can just read the papers released alongside …

Here's his original post https://x.com/i/status/2078133895766114412

The guy who said those things just joined OpenAI in the last month. He was previously a senior policy advisor, specifically on AI, to the current administration in the White House.…

The conversation going on in the industry is a bit broader than that. OpenAI's head of strategic futures just said last week that open models are inherently decelerationist, ungove…

Money is a big part of it. The notable thing about Chinese open models is their size. Most open source models out of US labs have not tried to compete with the enormous flagship mo…

I'm sympathetic to this POV. I think specifically in the case of Slack, it's not their responsibility in a cosmic sense, but it is in line with the direction and general promise th…

I am strongly in favor of open models, open source ML more broadly, and am pretty critical of the cynical positions adopted by major US labs vis a vis open models. But this is an i…

OpenAI's Head of Strategic Futures just this week posted this about the latest Kimi release: "It's a very good model! I don't think its performance can be explained away by distill…

This was my initial reaction to reading this post as well. Additionally, as I get older, I find the sentiment of "we're not trying to build a perfect system here" is less about "le…

This is not my experience working in the field the last 8 years. There is not a dearth of talented researchers, engineers, etc. who are willing to contribute to open models. Just l…

Gwern's absurdly catalogued personal site is one of those online artifacts that I hope never changes.

Based on a cursory read of the situation, it seems similar (at least on its face) to the Waymo vs Uber situation. In that case, Uber payed a Waymo an equity stake and signed an agr…

I mean, OpenAI delayed the public release of GPT-2 back in 2019 because it seemed capable of authoring interesting blog posts (that also happened to be untrue). It was a pretty big…

I think the author largely agrees with you re: type systems and LLMs. He's pretty explicit that Haskell should be very well positioned to be a power language for LLM-assisted progr…

I've been a power user of LLMs for software development for a while now, and I've found two things to be true: - The benefits of more "extreme" type systems are more accessible and…

Lots of researchers have done just this! There's a really rich history of research + lots of contemporary work on different encoding/representation strategies. This might be intere…

Nah, optical compression is a thing. You see it in a lot of different areas in ML. In this case, the "trick" has been known for a while, and belongs to a whole world of compression…

This has been a (noble) goal of lots of different projects in the community for a long time. Federated learning projects like Flower have been chipping away at it for a long time. …

This is a good starting point: https://huggingface.co/docs/peft/developer_guides/model_merg... But yes, in general, merging refers to techniques that directly blend the weights of …

I don't understand this line of criticism exactly. By putting new information in the context window, you are materially changing the activations at your point of sampling, which is…