Last week there we had a customer request that landed in our support on a feature that I partially wrote and wrote a pile of public documentation on. Support engineer ran customer query through Claude (trained on our public and internal docs) and it very, very confidently made a bunch of stuff up in the response. It was quite plausible sounding and it would have been great if it worked that way, but it didn't. While…
OpenAI declares 'code red' as Google catches up in AI race
531–540 of 960 posts
Re: OpenAI declares 'code red' as Google catches up in AI race
#532I've seen a rumor going around that OpenAI hasn't had a successful pre-training run since mid 2024. This seemed insane to me but if you give ChatGPT 5.1 a query about current events and instruct it not to use the internet it will tell you its knowledge cutoff is June 2024. Not sure if maybe that's just the smaller model or what. But I don't think it's a good sign to get that from any frontier model today, that's 18 m…
SemiAnalysis said it last week and AFAIK it wasn't denied. https://newsletter.semianalysis.com/p/tpuv7-google-takes-a-s...
"OpenAI’s leading researchers have not completed a successful full-scale pre-training run that was broadly deployed for a new frontier model since GPT-4o in May 2024, highlighting the significant technical hurdle that Google’s TPU fleet has managed to overcome."
Given the overall quality of the article, that is an uncharacteristically convoluted sentence. At the risk of stating the obvious, "that was broadly deployed" (or not) is contingent on many factors, most of which are not of the GPU vs. TPU technical variety.
Re: OpenAI declares 'code red' as Google catches up in AI race
#533Earlier quoted context omitted.
Dylan Patel joined Dwarkesh recently to interview Satya Nadella: https://www.dwarkesh.com/p/satya-nadella-2
And this is relevant how? That interview is 1.5 hours, not something you just casually drop a link to and say "here, listen to this to even understand what point I was trying to make"
I was trying to make the point that SemiAnalysis is semi-famous.
Re: OpenAI declares 'code red' as Google catches up in AI race
#534(My apologies if this was already asked - this thread is huge and Find-In-Page-ing for variations of "pre-train", "pretrain", and "train" turned up nothing about this. If this was already asked I'd super-appreciate a pointer to the discussion :) ) Genuine question: How is it possible for OpenAI to NOT successfully pre-train a model? I understand it's very difficult, but they've already successfully done this and they…
> Genuine question: How is it possible for OpenAI to NOT successfully pre-train a model? The same way everyone else fails at it. Change some hyper parameters to match the new hardware (more params), maybe implement the latest improvements in papers after it was validated in a smaller model run. Start training the big boy, loss looks good, 2 months and millions of dollars later loss plateaus, do the whole SFT/RL sheba…
Re: OpenAI declares 'code red' as Google catches up in AI race
#535Competition is all you need.
Re: OpenAI declares 'code red' as Google catches up in AI race
#536I've seen a rumor going around that OpenAI hasn't had a successful pre-training run since mid 2024. This seemed insane to me but if you give ChatGPT 5.1 a query about current events and instruct it not to use the internet it will tell you its knowledge cutoff is June 2024. Not sure if maybe that's just the smaller model or what. But I don't think it's a good sign to get that from any frontier model today, that's 18 m…
Any data after that is contaminated with vast amount of AI slop. Is anyone training on anything newer..?
Re: OpenAI declares 'code red' as Google catches up in AI race
#537Re: OpenAI declares 'code red' as Google catches up in AI race
#538Earlier quoted context omitted.
That is.... actually a seriously meaty article from a blog I've never heard of. Thanks for the pointer.
Semi analysis is great, they typically do semiconductors but reporting is top notch.
Re: OpenAI declares 'code red' as Google catches up in AI race
#539Earlier quoted context omitted.
Knowledge cutoff date is usually part of the system prompt. Helps you get useful answers like "I don't know that's too recent" when you ask questions like "who won the basketball game last night".
I opened up the Gemini app and asked: Who are the Dallas Stars playing tonight? Is there anyone that will be out of the lineup for the Stars? What have their recent injuries been like? Can you get the current score? It's response: The Dallas Stars are playing the New York Rangers tonight, Tuesday, December 2, 2025, at Madison Square Garden. Here is the information available regarding the rest of your questions: Out o…
Re: OpenAI declares 'code red' as Google catches up in AI race
#540Earlier quoted context omitted.
I'd be a little bit more nuanced: I think there's something off with their plans right now: it's pretty clear at this point that they can't own the technological frontier, Google is just too close already and from a purely technological PoV they are much better suited to have the best tech in the medium term. (There's no moat and Google has way more data and compute available, and also tons of cash to burn without de…
Being an insane brand means literally nothing if people can trivially switch to competitors, which they can. There isn't even a tenth of enough money if you group together all of advertising. Like, the entire industry. Ads is a bad, bad plan that wont work. Advertising is also extremely overvalued. And even at it's overvalued price tag, it's nowhere near enough.
Logically speaking, yes it is easy to switch between OAI and Gemini, or Coke and Pepsi. But brand loyalty is more about emotions (comfort, familiarity,..) rather logical reasoning.