Is this what the Web was like in the beginning? Something exciting and fascinating every week?
Kinda though things didn't move quite as fast back then. Knowledge didn't spread as fast yet because it was the internet itself that made this possible.
Open-R1: an open reproduction of DeepSeek-R1
51–60 of 246 posts
Re: Open-R1: an open reproduction of DeepSeek-R1
#52Exciting to see this being reproduced, loving the hyper-fast movement in open source! This is exactly why it is not “US vs China”, the battle is between heavily-capitalized Silicon Valley companies versus open source. Every believer in this tech owes DeepSeek some gratitude, but even they stand on shoulders of giants in the form of everyone else who pushed the frontier forward and chose to publish, rather than exploi…
Open source has pragmatic merits and I love the culture. But I don't like associating it with a moral high ground because it doesn't charge you money. By this standard, we should also ask Intel/AMD to open-source their CPUs, video game studios to open-source their code and artifacts, and Google/Amazon for their search engine and infrastructure. Not all business sectors can afford to sustain with the Open Source model.
Re: Open-R1: an open reproduction of DeepSeek-R1
#53Earlier quoted context omitted.
deepseek claims they are "open source" but they are not. They are open weight. IMO a truly "open" AI model should have 3 components publicly available: the weights, the code, and the dataset. Without all 3 the model is not reproducible. Could make the argument that the code and data are sufficient though.
No one will release the dataset because we all know it is gathered through dodgy means.
And likely massaged by the CCP.
Re: Open-R1: an open reproduction of DeepSeek-R1
#54Earlier quoted context omitted.
Just a decade and a half as it turns out! (though things definitely felt dizzyingly fast back then - think Google was launched just 5 years after HTML) - HTML first released in 1993 - AJAX in 1999 - Websocket first proposed in 2008 https://en.wikipedia.org/wiki/HTML https://en.wikipedia.org/wiki/Ajax_(programming) https://en.wikipedia.org/wiki/WebSocket
ActiveX XMLHTTP might have been released in 99, but it didn’t see any sort of real wider usage until 2004, 2005. I’d suggest its usage was really kickstarted when jQuery 1.0 launched in 2006 and standardised the interface to a simple API.
Gmail was the first time I saw a website which could refresh the information without refreshing the page. I was a teen back then but I realized it was something momentous.
Re: Open-R1: an open reproduction of DeepSeek-R1
#55What are some other domains outside of Math and Coding that would be suitable for RL with automated verification?
RFP responses. In enterprise sales, there's a huge amount of back and forth with different teams in a customer when you're selling anything but very simple applications. Most enterprise customers require certified or authoritative responses with backup material that is tested later during formal verification.
Re: Open-R1: an open reproduction of DeepSeek-R1
#56Earlier quoted context omitted.
The thing about open source is that you don't need to trust it. They shared their methodology, so if they are legit, someone else will reproduce what they did very quickly. I expect Meta, Amazon, Google, and Anthropic are on the case right now. From this list, the only one I trust any more than I trust Deepseek is Anthropic. The other three have shown they'll instantly bend the knee to whomever is in power, and that'…
> the only one I trust any more than I trust Deepseek is Anthropic Will you kindly explain why? Thank you.
Anthropic was the only company in that list not to have paid for their CEO or founder to attend the inaugeration of the current ruler of the exectuive branch of the US a week ago.
Re: Open-R1: an open reproduction of DeepSeek-R1
#57Earlier quoted context omitted.
Oh yes, I am firmly on Team China here because US companies got too greedy. Meta is an exception here though and they also propelled AI development massively. DeepSeek is awesome. Any AI task yet implemented in our business can be run from my local PC with just the smaller models. And my PC is fairly crappy to begin with. OpenAI looks quite silly with their "we have to close everything".
The US companies got too greedy? How? They invented this entire space, literally. DeepSeek built their base models off Llama releases and OpenAI outputs (or so it’s thought), and while they added some optimizations on top, it seems like they’ve lied about the costs to produce their models by simply being vague about their base model and training data, and quoting the cost of their final training run. And then there’s…
The US models are also full of censorship. For example the US is much more sensitive to anything related to sexuality and here in Europe it's quite frustrating to deal with that censorship.
Re: Open-R1: an open reproduction of DeepSeek-R1
#58Earlier quoted context omitted.
I still remember when they came out with the 3d dancing baby, blew my mind much more than deepseek, it even played air guitar!
Rounded corners, anyone? Who remembers LISP being new?
I've got an original Apple ][ reference manual (red cover) with the hand annotated ROM listing.
Also have SMALLTALK-80 book with it's railtrack diagrams of syntax on the inside covers.
What's really interesting about the AI mania we're in is that no one has shown that what we have now will get to AGI and how. We have great models that simulate reasoning, but how close are they?
How do we measure their quality? Benchmarks? Tooling?
Re: Open-R1: an open reproduction of DeepSeek-R1
#59Earlier quoted context omitted.
This supposed project is a bit dull, it is just an ongoing HuggingFace community engagement initiative with a misleading headline. Yes R1 itself is fascinating, but there isn't something like it coming out every week.
You mean not like in the last 1-2 months
Every week to me means the frequency, not the duration. So having 52 events in a year that are spread out somewhat evenly but for which many take longer to develop than a week would count. If I count Deepseek as one of these I can’t find another 51 that are on this level. But I’m sure there was at least one per week that was exciting, just not to this degree.
Re: Open-R1: an open reproduction of DeepSeek-R1
#60how is this open vs whatdeepseek did?
deepseek claims they are "open source" but they are not. They are open weight. IMO a truly "open" AI model should have 3 components publicly available: the weights, the code, and the dataset. Without all 3 the model is not reproducible. Could make the argument that the code and data are sufficient though.
This nitpicking is pointless.
DeepSeek's gifts to the world of its open weights, public research and OSS code of its SOTA models are all any reasonable person should expect given no organization is going to release their dataset and open themselves up to criticism and legal exposure.
You shouldn't expect to any to see datasets behind any SOTA models until they're able to be synthetically generated from larger models. Models only trained on sanctioned "public" datasets are not going to perform as well which makes them a lot less interesting and practically useful.
Yes it would be great for their to be open models containing original datasets and a working pipeline to recreate models from scratch. But when few people would even have the resources to train the models and the huge training costs just result in worse performing models, it's only academically interesting to a few research labs.
Open model releases should be celebrated, not criticized with unreasonable nitpicking and expectations that serves no useful purpose other than discouraging future open releases. When the norm is for Open Models to include their datasets, we can start criticizing those that don't, but until then be gracious that they're contributing anything at all.