Viewing profile — musebox35

Sadly no. Despite the name, the experts are not routed per concept or topic but per token. So for the same sentence you might activate multiple experts for different tokens. What i…

dev variants are usually cfg distilled which means that directly finetuning isn’t as effective. In the past, for the flux2 klein models,they released base versions that are not dis…

Thanks for posting this. This is the tinkerer mentality. It is not for everyone, but certain things can only be learned in that way. It is the best antidote to AI paranoia. There i…

Training isn’t a single homogeneous step. It starts with pretraining which requires bulk PB of data but you have less quality concerns here. You cover the whole data distribution. …

Thanks it looks promising :)

I also have been using emacs for almost anything for the past 20 years. I had to switch to VSCode for coding over a remote ssh connection to cloud VMs. The client/server split of v…

True enough, but that is true for all the products I buy. I do not expect to control every product I own. For some I prefer to have more control, for others I just need something t…

It is much more reasonable to do it in a visible / flagged way. At least you have visibility over the quality of service you get as a customer. Silent treatment is a breach of trus…

I work on open source text-to-image finetuning of open source models like zimage/flux2 klein 4b and inference time latency optimization. The moment I read the silent treatment, I w…

SFT + RL connection to model/hypothesis search is insightful. Brute force / scalable search is where Sutton's Bitter Lesson also points to. Once your search domain is small compare…

I understand the skepticism. I am worried about the implications of AI as well. The deeper issue at stake is that the depth of scientific knowledge has been increasing for a very l…

The most successful applications like coding are not the result of pure LLM/generative modeling. They come from closing the loop with an agentic harness. The generate-test-selectiv…

I was about to post your last point / quote. Going multigpu is relatively not so though but once you go multi-node you have distributed storage/io/compute system which is highly no…

Not understanding the whole does not completely remove an ability to analyze. An interesting direction is individuality and having a notion of self. It is difficult to demarcate th…

I think the complexity issue in science and engineering has also been growing for some time beyond what can be analyzed/designed by a person or a group with conventional software a…

The cutoff date is early 2025 so make sure to enable web search when experimenting. I was expecting something more recent, took a while to notice this.

Thanks, that sounds like a good direction to try.

Most of your analysis I can easily relate to except “There is evidence that the Chinese models are falling further behind, not gaining.” Where is that evidence? Deepseekv4 claims t…

I watched the last one S5:E17 What jobs are AI jobs and I think it gives the right framing to think about this. It is not prescriptive, it does not give a list which is smart. The …

I am exploring ways to document the design for the agent to read and update. What makes it difficult is the lack of structure. Spec writing is not my core skill. Schemas and APIs a…

I totally agree. I loved coding because of its closed feedback loop. Since last November, I also delegated it mostly to agents. Now I concentrate more on the design part, which is …

Could you briefly describe your workflow for doing that or give a pointer to a blog you wrote/like that aligns with the process? Thanks in any case, happy designing ;-)

Please see Ben Evans’ podcast on a good take on this. Coding is just one of the task you do in your job, it is not the job or at least it probably is not. You do not get paid to co…

I wonder what will be the effect of this on open source software. I do not mean the technical aspects, easy of coding, documenting, explosion of PRs, .. Social aspects scare me a b…

Debating how not to use AI will not get anyone anywhere since negative framing almost never works with humans (it also does not work with llms). Let’s concentrate on how to build c…