Earlier quoted context omitted.
https://github.com/ggerganov/llama.cpp/issues/39 40GB for the big llama.
But that's not ChatGPT?
[deleted]
How 'Open' Is OpenAI, Really?
61–70 of 78 posts
Re: How 'Open' Is OpenAI, Really?
#62I wish we could interview those scientists to hear what they think of this AGI goal.
I'm not convinced this will work.
Re: How 'Open' Is OpenAI, Really?
#63Earlier quoted context omitted.
LLaMa [1] works on Apple Mx processors due to unified memory. The 7B model works on a 8GB Apple MacBook Pro M1, ~40 seconds/token. [1] https://dev.l1x.be/posts/2023/03/12/using-llama-with-m1-mac
I get 220ms/token on an M1 8GB Macbook Air, so ChatGPT speeds. on a 32GB M1 Max it babbles faster than my ex-gf.
Weird, I ran it alongside quite a few other processes, didn't expect to even be able to run and didn't bother to optimize it in any meaningful way.
Re: How 'Open' Is OpenAI, Really?
#64Earlier quoted context omitted.
https://github.com/ggerganov/llama.cpp/issues/39 40GB for the big llama.
Even biggest Llama is like 1/3 size of ChatGPT
Might as well compare it to AlpacaGPT, they are trained different. Carefully prompted the output shows flashes of excellence and really whips ass and it is still early days.
We have almost closed the gap in no time whereas on ChatGPT day 1 it seemed they were years ahead. Hopefully there will be more models and developments.
Re: How 'Open' Is OpenAI, Really?
#65Now that it runs on our laptops a lot of the initial shock, mystique, and novelty has already worn off, just like with DALL-E before it. What the leaks show is that the community can run circles around OpenAI and the others when it comes to optimizing these things and taking them in interesting directions on a shoe string budget. For example of what one sesh of tinkering can produce: https://github.com/geohot/tinygra…
People have decent oss solutions for running inference. There are currently no high quality oss code for training and the best checkpoints are still unavailable (gpt3.5+4/palm/claude)…
There are only a few hundred folks who have actual experience training at this scale.
llama is both not oss and not as good as da-vinci-003/etc. Prob best actual oss model is like flan-t5-xxl.
Re: How 'Open' Is OpenAI, Really?
#66OpenAI are legends. They gave us fire. Until then we only heard about this tech from inside Google. It was always like "AI ethicists think this is too dangerous" and so we couldn't have it.
Each time, OpenAI made it happen. They were the driving force behind all of this stuff. Dall E changed the game entirely. ChatGPT changed the game again!
Lots of credit to Stability.AI - absolutely grand work by then. But OpenAI took what was once the tool of wizards in secret towers and gave it to all of us.
Then Stability and Meta gave us the means to build the tools. Who would have thought that Mark Zuckerberg would be crucial to the democratization of cutting edge tech. What a legend!
Re: How 'Open' Is OpenAI, Really?
#67I think one should also acknowledge that OpenAI has been perhaps the leading provider of cutting edge AI to the public.
They let the public play against Dota Five, they provide their GPT playground and API, and they were the first to give something to the public with Dalle2 too. From the perspective of a developer this os mot very open, but from the perspective of the public it is quite open. But more "open access" or "open visibility" instead of "open source".
What they really don't want is that google or whatever develops an AGI secretly.
Re: How 'Open' Is OpenAI, Really?
#68Now that it runs on our laptops a lot of the initial shock, mystique, and novelty has already worn off, just like with DALL-E before it. What the leaks show is that the community can run circles around OpenAI and the others when it comes to optimizing these things and taking them in interesting directions on a shoe string budget. For example of what one sesh of tinkering can produce: https://github.com/geohot/tinygra…
People have decent oss solutions for running inference. There are currently no high quality oss code for training and the best checkpoints are still unavailable (gpt3.5+4/palm/claude)… There are only a few hundred folks who have actual experience training at this scale. llama is both not oss and not as good as da-vinci-003/etc. Prob best actual oss model is like flan-t5-xxl.
So we can't lay eggs yet, at least we got one. Who knows what might hatch.
Re: How 'Open' Is OpenAI, Really?
#69Does it matter? Openai finally built something resembling ai. Big tech wasn’t innovating, I personally don’t care if employees at openai capture a ton of the upside
> Does it matter? Yes. > Openai finally built something resembling ai. Big tech wasn’t innovating, I personally don’t care if employees at openai capture a ton of the upside And if they called themselves ClosedAI I don't think people would mind so much; it's the dishonesty that's grating.
At least they publish their papers!
Re: How 'Open' Is OpenAI, Really?
#70Earlier quoted context omitted.
People have decent oss solutions for running inference. There are currently no high quality oss code for training and the best checkpoints are still unavailable (gpt3.5+4/palm/claude)… There are only a few hundred folks who have actual experience training at this scale. llama is both not oss and not as good as da-vinci-003/etc. Prob best actual oss model is like flan-t5-xxl.
So we can't lay eggs yet, at least we got one. Who knows what might hatch.
I guess I just felt an objection about “[oss] running circles around OpenAI.” The reality is that the core technology is still very unapproachable (making these models). OSS solved like 1% of the total problem/difficulty.
My biggest hope is that meta/google/anthropic actually decide to release training code + data + checkpoint. Meta is most likely to do it but still unlikely
Without that it would take eutherai/carperai to really exceed expectations for OSS to be competitive.