181–190 of 287 posts
Slightly better on the NYT Connections benchmark (27.9) than Claude 3 Opus (27.3) but massively improved over Claude 3 Sonnet (7.8). GPT-4o 30.7 Claude 3.5 Sonnet 27.9 Claude 3 Opus 27.3 Llama 3 Instruct 70B 24.0 Gemini Pro 1.5 0514 22.3 Mistral Large 17.7 Qwen 2 Instruct 72B 15.6
Opus remained better than GPT for me, even after the release of GPT-4o. VERY happy to see an even further improvement beyond that, Claude is a terrific product and given the news that GPT-5 only began its training several weeks ago I don't see any situation where Anthropic is dethroned in the near term. There are only two parts of Anthropic's offering I'm not a fan of: - Lack of conversation sharing: I had a conversa…
Both GPT-4 and 4o have been completely useless for coding in the past couple of weeks for me - constant errors, and not just your typical LLM inaccuracies but incapable of producing a few lines of self-consistent code e.g. defines variables foo on one line and refers to it as bar on the next, or it misspells it as foox.
This is amazing - I far prefer the personality of Claude to GPT-4 series models. Also, with coding tasks, Claude-3-Opus and been far better for me vs gpt-4-turbo and gpt-4o both. Looking forward to giving it a spin. Seems like it's doing better than GPT-4o in most benchmarks though I'd like to see if its speed is comparable or not. Also, eagerly awaiting the LMSYS blind comparison results!
GPT4(o) is quite good at advanced math, it's been helpful when I was learning differential geometry. Not sure how Claude compares though, this 3.5 release has tempted me to try it out. Also, it's finally available in Canada!
Earlier quoted context omitted.
Both GPT-4 and 4o have been completely useless for coding in the past couple of weeks for me - constant errors, and not just your typical LLM inaccuracies but incapable of producing a few lines of self-consistent code e.g. defines variables foo on one line and refers to it as bar on the next, or it misspells it as foox.
Waht language? Because I'm guessing they work well for languages with a large amount of training data like Python (in my experience), less well for less used languages like Zig or Clojure (haven't tried them but that's my theory)
Opus remained better than GPT for me, even after the release of GPT-4o. VERY happy to see an even further improvement beyond that, Claude is a terrific product and given the news that GPT-5 only began its training several weeks ago I don't see any situation where Anthropic is dethroned in the near term. There are only two parts of Anthropic's offering I'm not a fan of: - Lack of conversation sharing: I had a conversa…
> GPT-5 only began its training several weeks ago Source?
> OpenAI has recently begun training its next frontier model and we anticipate the resulting systems to bring us to the next level of capabilities on our path to AGI.
Earlier quoted context omitted.
Both GPT-4 and 4o have been completely useless for coding in the past couple of weeks for me - constant errors, and not just your typical LLM inaccuracies but incapable of producing a few lines of self-consistent code e.g. defines variables foo on one line and refers to it as bar on the next, or it misspells it as foox.
Waht language? Because I'm guessing they work well for languages with a large amount of training data like Python (in my experience), less well for less used languages like Zig or Clojure (haven't tried them but that's my theory)
Earlier quoted context omitted.
> GPT-5 only began its training several weeks ago Source?
https://openai.com/index/openai-board-forms-safety-and-secur... (May 28th) > OpenAI has recently begun training its next frontier model and we anticipate the resulting systems to bring us to the next level of capabilities on our path to AGI.
Earlier quoted context omitted.
> GPT-5 only began its training several weeks ago Source?
https://openai.com/index/openai-board-forms-safety-and-secur... (May 28th) > OpenAI has recently begun training its next frontier model and we anticipate the resulting systems to bring us to the next level of capabilities on our path to AGI.
The value of all the AI companies is predicated on high chance of AGI, and gpt5 failing to be revolutionary may pop the whole bubble (+10 trillion of market cap)
Earlier quoted context omitted.
I think Anthropic also uses Google TPUs.
I don't think that is the case. AWS is a very significant investor and if you meet with their business development team they will recommend deploying on bedrock (which is Nvidia). There are also press releases like this[1] stating they use Nvidia. [1] https://nvidianews.nvidia.com/news/aws-and-nvidia-collaborat... and https://press.aboutamazon.com/2023/3/aws-and-nvidia-collabor... search for "anthropic"
From Claude 3's technical report:
Like its predecessors, Claude 3 models employ various training methods, such as unsupervised learning and Constitutional AI [6]. These models were trained using hardware from Amazon Web Services (AWS) and Google Cloud Platform (GCP), with core frameworks including PyTorch [7], JAX [8], and Triton [9].
JAX's GPU support is practically non-existent, it is only used on TPUs.