4 years ?!? That’s like a century in AI-dog years. Who knows how the world will be by then.
[flagged]
I think I’ve used it so much that a sentence in I assumed it was ChatGPT. It has a certain way of speaking.
Anthropic’s $5B, 4-year plan to take on OpenAI
21–30 of 504 posts
Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#22Anthropic’s Claude LLM is pretty interesting. In many ways it feels much more limited than GPT4. However, it is suspiciously good at a few edge-case code generation tasks (can’t go into details) that makes me wonder where it got its training data from. It also seems to be much less prone to hallucinating APIs and modules, preferring instead to switch back to natural language and describe the task without pretending i…
I know you said no details, but can you at least share a little bit more about Claude LLM's code generation?
There is a language with massive usage in the enterprise but with very few (if any) high quality code examples on the public internet.
When given a broad task, GPT4 doesn’t just write incorrect code, it tries to do entire categories of things the language literally cannot do because of the ecosystem it runs inside.
Claude does a much better job writing usable code, but more importantly it does NOT tell you to do things in code that need to be done out-of-band. In fact, it uses natural language to identify these areas and point you in the right direction.
If you dig into my profile & LinkedIn you can probably guess what language I’m talking about.
Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#23This is a lot of money going into compute. I will say this again. EU is sleeping on the opportunity to throw money in an opensource initiative, in a field were money matter and the field is still (kind of) level.
Sure, let's make an EU commercial LLM. Let's start by scraping all the Francophone internet. Then let's remove all the PII data and potentially PII-data. Easy-peasy. Then let's train our network so as not to spew out or make up PII data - easy peasy Then let's make it able to delete PII data that it has inadvertedly collected on request. Simultaneously it should be recording all the conversations for safety reasons.…
The secret is not scraping PII in the first place (which is not really difficult, though it requires some planning)
Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#24Anthropic’s Claude LLM is pretty interesting. In many ways it feels much more limited than GPT4. However, it is suspiciously good at a few edge-case code generation tasks (can’t go into details) that makes me wonder where it got its training data from. It also seems to be much less prone to hallucinating APIs and modules, preferring instead to switch back to natural language and describe the task without pretending i…
Didn't they partner with SourceGraph to make Cody? Here's them talking a bit about it: https://www.youtube.com/watch?v=LYuh-BdcOfw. Maybe that's why?
Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#25Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#26Is there a large barrier to entry? I thought that the costs of training, while large, were only a few million so not insurmountable, and the technology is pretty well understood. If this is true it's hard to understand what they'd need $300 million for, and also if there's no moat why they would command a "billions" valuation.
Do we know how much it cost to train GPT-4? (Or would cost, if done by someone not trained in Azure by someone partnered with Microsoft?) My impression, without looking into it now, is training GPT-3 was on the order of $1-10 million. GPT-4 would be higher than that, but you're right still in the ballpark of what lots of ordinary companies could pay.
Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#27Is there a large barrier to entry? I thought that the costs of training, while large, were only a few million so not insurmountable, and the technology is pretty well understood. If this is true it's hard to understand what they'd need $300 million for, and also if there's no moat why they would command a "billions" valuation.
Is there a large barrier to entry?
Yes. OpenAI has raised $1bn (plus $10bn from MSFT to exchange GPT access for Azure services) and has been going 8 years. There are some huge challenges to making it work well and fast. You need money for opex (hiring GPUs to train models on mostly) and talent (people to improve the tech). No one is competing with OpenAI without a good chunk of cash in the bank.
Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#28> billions of dollars over the next 18 months
Is most of that money for hiring people to tag/label/comment on data and the data center costs?
Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#29Earlier quoted context omitted.
Sure, let's make an EU commercial LLM. Let's start by scraping all the Francophone internet. Then let's remove all the PII data and potentially PII-data. Easy-peasy. Then let's train our network so as not to spew out or make up PII data - easy peasy Then let's make it able to delete PII data that it has inadvertedly collected on request. Simultaneously it should be recording all the conversations for safety reasons.…
The secret is not scraping PII in the first place (which is not really difficult, though it requires some planning)
not sure if it can ever be possible. i can ask chatGPT to do stylography analysis on our comments, find our other accounts and go from there. I'm pretty sure most pieces of human-generated data is identifiable at this point
Re: Anthropic’s $5B, 4-year plan to take on OpenAI
#30This is a lot of money going into compute. I will say this again. EU is sleeping on the opportunity to throw money in an opensource initiative, in a field were money matter and the field is still (kind of) level.
> [2008] France has been given the green light by the European Commission for a $152 million government grant for a consortium building a European rival to U.S. internet search giant Google. I don't think government funding to compete with private businesses works well https://www.hollywoodreporter.com/business/business-news/ec-...
They also funded 2 competitors to AWS, just to make sure we own cloud computing. Two is always better than one, right, by ensuring they'll compete with each other.
Another bright idea was to let both projects be managed by large reputable French corporations that everybody trusts. With no software DNA.
How come did both fail?
Edit: One of the largest European provider today, OVH, who existed at the time and was already the leader in France was explicitly left out of both projects... Because the founder is not a guy we can trust you know, he didn't attend the best schools.