Viewing profile — fspeech

Linear layers use decays (like IIR filters) that naturally provide relative positions. Full attention layers can then be free to develop concepts that attend to each other regardle…

That's not true. First there is still a licensing and quota scheme on the US side for the H200s. Secondly China blocked them for use in inferencing. Thirdly Chinese companies don't…

Their kv cache is smaller so they can use less vram and also keep your prefix cached for longer. https://deepseek.ai/blog/deepseek-v4-compressed-attention

Well, so far as the governments get VAT from the manufacturers, they are getting a return on their investments. They are more like mutualizing the companies than subsidizing them, …

Performance is generally limited by the process. Yield not so much. Assuming you can make it at a meaningful level at all, yield is generally a learning process.

Yield is generally not an issue over time, at least not as big as someone outside of the industry would think, if you get enough chips to work in the first place. For high volume c…

What do you mean? Yield is a function of the chip size and density as well as the process. Plus it's a commercial secret so your bet can't be adjudicated.

Yes division is a poor example. It's a poor separation of concerns to try to wrap at this level without usage context. To see the point try to wrap overflows on arithmetic function…

Try ZhiHu(知乎).

If you are just interested in a structural description (so-called netlist) the standard is EDIF.

A lot of Chinese internet commentators are very ignorant of the reality in the US, but the Economist's riposte is weird too. For example how is the Chinese property malaise, which …

Price of a commodity metal can do whatever they want without causing a big problem. It is just a resource allocation signal. However if you base your currency value on it suddenly …

Do you know what questions Newton was asking? https://en.wikipedia.org/wiki/Religious_views_of_Isaac_Newto... Being right is often hindsight and luck.

Your point is right on. And additionally, why would an average Indian refuse the pay package to work in China? The top r&d guy at SMIC is from Taiwan after all. Liang got both Sams…

Humans are also not rewarded for making pronouncements all the time. Experts actually have a reputation to maintain and are likely more reluctant to give opionions that they are no…

I don't think people mind having bigger spaces but market is not clearing. In the US you have slums and bombed out building shells in prime urban locations as well. It is fascinati…

Cheap housing isn't the problem. The problems are people speculating on the appreciation of property, banking system depending on property value as loan collaterals and local gover…

I don't think the repair could be done. It's not about plugging a hole in space. It's about surviving reentry. They can't guarantee the integrity of the glass. Anyway to your point…

It has a fixed capacity of how many different things it can pay close attention to. If it fails on a seemingly less important but easy to follow instruction it is an indicator that…

A human rated vehicle would be much more expensive than one rated for cargo. And there are not many use cases for the vehicle other than rescue missions to the two space stations.

Whatever the reason California averages $4000 in labor cost per Tesla vs less than $300 in Shanghai. That's quite a difference.

It uses 75% linear attention layers so it is inherently lower cost. And it is MOE so active parameters are far lower.

There is also Minimax M2 https://huggingface.co/MiniMaxAI/MiniMax-M2