Is there any reason to believe that the first open source chat GPT clone won't consume most mindshare, a la stable diffusion?
Deployment is the significant barrier with these models. They have far more parameters than stable diffusion/DALLE2.
Anthropic's Claude is said to improve on ChatGPT, but still has limitations
31–40 of 58 posts
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#32Is there any reason to believe that the first open source chat GPT clone won't consume most mindshare, a la stable diffusion?
There are already open source LLMs with comparable parameter counts (Facebook's OPT-175B, BLOOM), but you'll need ~10x A100 GPUs to run them (which would cost ~$100K+).
I suspect a big part of why stable diffusion managed to consume so much mindshare is that it can run on ordinary consumer hardware. On that point, I would be excited about an open-source RETRO (https://arxiv.org/pdf/2112.04426.pdf) model with comparable performance to GPT-3 that could run on consumer hardware with an NVMe SSD.
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#33Is there any reason to believe that the first open source chat GPT clone won't consume most mindshare, a la stable diffusion?
My theory is that OpenAI is preying on venture capital, and they don't care who wins long term. As long as they're first to get the freshest ideas on the biggest computers, they can secure a large sum of money.
Sounds like SNL's "First Change Bank" skit:
- You give us a dollar, we'll give you four quarters!
- People ask us how we make money. The answer is simple: _volume_.
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#34In a strange twist of events... their massive Series B round was led by SBF "The [$580M] Series B follows the company raising $124 million in a Series A round in 2021. The Series B round was led by Sam Bankman-Fried, CEO of FTX. The round also included participation from Caroline Ellison, Jim McClave, Nishad Singh, Jaan Tallinn, and the Center for Emerging Risk Research (CERR)." https://www.anthropic.com/news/announc…
When someone pays you with stolen funds, aren't you (morally, if not legally) obliged to pay it back to the victims?
> When someone pays you with stolen funds, aren’t you (morally, if not legally) obliged to pay it back to the victims?
Legally, its complicated, and the reason for that is because it is viewed as both morally complicated (as well simplistic approaches being viewed as creating undesired social incentives.)
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#35Am I the only one who cannot see the embedded tweets? Further, they don’t even link to them elsewhere - you have to accept their cookies to view them. Extremely anti-user.
[deleted]
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#36Is there any reason to believe that the first open source chat GPT clone won't consume most mindshare, a la stable diffusion?
There are already open source LLMs with comparable parameter counts (Facebook's OPT-175B, BLOOM), but you'll need ~10x A100 GPUs to run them (which would cost ~$100K+). I suspect a big part of why stable diffusion managed to consume so much mindshare is that it can run on ordinary consumer hardware. On that point, I would be excited about an open-source RETRO ( https://arxiv.org/pdf/2112.04426.pdf ) model with compar…
They aren't as good as davinci-003, though. There is no open source model competitive with GPT-3.5 yet.
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#37Earlier quoted context omitted.
If it was a bona fide investment and you are without notice of any wrongdoing - no, I’d say you’re not. Presumably your nefarious investor holds some kind of ownership interest that can be sold by their trustees/liquidators to raise funds. If you are able to do so, it seems like a nice thing to try to help source money to help the people caught out, including by assisting in realising any ownership stake, but that’s…
I think the answer should be yes so that those taking investment should feel pressure to ensure that the money from investors is legitimate. Otherwise, it's going to be "whoops, we didn't know ;)" every time.
There seems to be a false dichotomy here between either “always treat the recipient as guilty” and “just let them claim ignorance without questioning it”.
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#38Earlier quoted context omitted.
When someone pays you with stolen funds, aren't you (morally, if not legally) obliged to pay it back to the victims?
> When someone pays you with stolen funds, aren’t you (morally, if not legally) obliged to pay it back to the victims? Legally, its complicated, and the reason for that is because it is viewed as both morally complicated (as well simplistic approaches being viewed as creating undesired social incentives.)
Can you delve into the complexities? To me, if clawback were not available, it seems to create the perverse incentive to start a ponzi scheme and then put the money in a shielded "investment" in my friends' overvalued startup.
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#39Is there any reason to believe that the first open source chat GPT clone won't consume most mindshare, a la stable diffusion?
There are already open source LLMs with comparable parameter counts (Facebook's OPT-175B, BLOOM), but you'll need ~10x A100 GPUs to run them (which would cost ~$100K+). I suspect a big part of why stable diffusion managed to consume so much mindshare is that it can run on ordinary consumer hardware. On that point, I would be excited about an open-source RETRO ( https://arxiv.org/pdf/2112.04426.pdf ) model with compar…
GPT-J-6B is said to be also of comparable performance in certain areas to GPT-3 and can be run (inference) on a RTX3090.
Re: Anthropic's Claude is said to improve on ChatGPT, but still has limitations
#40Is there any reason to believe that the first open source chat GPT clone won't consume most mindshare, a la stable diffusion?
on top of the hardware requirements ($10s of thousands of GPUs are needed for something a language model as big as gpt-3) there's also a lot of work involved in RLHF models like chatgpt. you need to pay people to write and review thousands/tens of thousands of responses for training. see 'methods' here: https://openai.com/blog/chatgpt/