The fact that AI models can be so easily distilled and replicated is such a stroke of luck. 10 or 15 years ago if one had asked me to envision a future where a private company invents artificial intelligence, I'd have thought for sure they'd have a massive moat, be very difficult to catch, and it would create an almost instant monopoly. Rather, it seems that selling intelligence might end up as a race to the bottom.…
Where is the actual evidence of distillation? I keep seeing this repeated ad nauseam but I must have somehow missed the evidence.
OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
191–200 of 352 posts
Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#192Earlier quoted context omitted.
This viewpoint doesn't make any sense to me. The weights + inference code are the "source code" for AI. I literally don't know what else you are demanding for the "open source" label.
> I literally don't know what else you are demanding for the "open source" label. you need to "literally" go read the definition of open source software or even ask an LLM to define it for you. Weights + inference code are not the source code they're more like the compiled binary. Making modifications to the behavior of a model with additional training is like writing a mod for minecraft. Sure, you can change things…
The actual problem is that we know nothing about the training set of any open-weights model. They could be intentionally biased to influence users, from political censorship to brand advertising, or general shaping of cultural norms. You run the model on own hardware not knowing if it is designed to act against you. Having whole chain open source would allow audit and reproducing the results.
Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#193The fact that AI models can be so easily distilled and replicated is such a stroke of luck. 10 or 15 years ago if one had asked me to envision a future where a private company invents artificial intelligence, I'd have thought for sure they'd have a massive moat, be very difficult to catch, and it would create an almost instant monopoly. Rather, it seems that selling intelligence might end up as a race to the bottom.…
Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#194Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#195Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#196The price difference to Deepseek models (deepseek-v4-flash, deepseek-v4-pro and deepseek-v4-flash-vision-exp) is still significant while the performance difference is not.
Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#197Earlier quoted context omitted.
If you think of LLMs as programs. The weights and inference code are very much a binary. While the training code and data are the true source. Since if you want to robustly modify the LLM that's actually what you need. But since "compilation" (training) is extremely compute intensive this isn't something accessible to anyone without an entire datacenter. Anyway semantics aside having the binary is still infinitely be…
I agree. But models in difference to compiled binaries, are useful as just weights and can be further refined and post-trained, at least. I don't know LLM theory well enough to say if there's some secret sauce they can hold back that makes training ineffective. Less effective I'm sure, we don't have access to their smart training schemes, but post-training should always be possible IIUC.
post-training is like writing a wrapper around the binary. It is closer to building on top of than truly modifying, in that you can tailor things to your needs slightly but cannot make fundamental changes to the underlying thing.
Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#198The fact that AI models can be so easily distilled and replicated is such a stroke of luck. 10 or 15 years ago if one had asked me to envision a future where a private company invents artificial intelligence, I'd have thought for sure they'd have a massive moat, be very difficult to catch, and it would create an almost instant monopoly. Rather, it seems that selling intelligence might end up as a race to the bottom.…
Where is the actual evidence of distillation? I keep seeing this repeated ad nauseam but I must have somehow missed the evidence.
[1] https://www.anthropic.com/news/detecting-and-preventing-dist...
Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#199Earlier quoted context omitted.
I hope it's a good bit cheaper than that, I pay close to $400/mo for electricity and I'm in no way interested in paying anything like that for AI.
Jeeze, what are you doing that uses so much electricity? I live in Germany where people won't stop whining about electricity prices, and I pay 75€/mo.
Re: OpenAI: GPT 5.6 Sol price reduction (until at least Nov 21)
#200Earlier quoted context omitted.
There's something startling about how (relatively) simple these networks are and yet how powerful they are. The main ingredient the AI darlings are using is vast amounts of compute and data. I don't want to take away anything from what the researchers came up with, but I suspect even they are surprised at how capable some of these models have become.
I'm unnerved by how alphazero is more complicated than the "intelligent llms"; it has at least multiple heads and MCTS, a search algorithm. The LLMs seem to just be monolithic (if complicated) architectures where tokens go in the bottom and tokens are spit out at the top.