The author implies that Grok 3 becoming racist because of a system prompt is a bad thing. I think it's a good thing and shows how steerable the model is. Many other models pretty much ignore the system prompt and always behave the same.
Based on your history here it’s quite obvious you’re a musk fan. Maybe though, you should realize that a model being steerable to claim itself being mechahitler and proposing death to people is absolutely not a “good thing”. I suggest you seriously reconsider on what you’re advocating for here. Because the outcome of this will cost innocent lives.
[edit to focus on pricing, leaving praise of Simon's post out despite being deserved] Simon claims, 'Grok 4 is competitively priced. It's $3/million for input tokens and $15/million for output tokens - the same price as Claude Sonnet 4.' This ignores the real price which skyrockets with thinking tokens. This is a classic weird tesla-style pricing tactic at work. The price is not what it seems. The tokens it's burning…
Tesla focused its pricing on drivers of gasoline vehicles, and their gas cost savings estimates are actually quite low compared to the real savings you will achieve. It was annoying when you already drive an EV and are buying a Tesla though to have to uncheck the savings option to see the pre savings prices. They changed it now so by default it only includes the $7500 and no longer automatically checks the gas saving…
>their gas cost savings estimates are actually quite low compared to the real savings you will achieve
I ran the numbers for myself and they literally weren't. They overestimated how many miles/yr I drove and underestimated how much I pay for electricity. There's plenty of other reasons to prefer EVs, but if you live somewhere with expensive electricity then fuel cost isn't one of them. In the sedan world you're likely better off with a Prius but even small SUV are getting 30-40 mpg nowadays.
As an asterisk, I live in California where gas prices are ~25% above the national average but electricity costs are more like double/triple. YMMV which is why you shouldn't trust Tesla's numbers or anyone else's except your own
"It feels very credulous to ascribe what happened to a system prompt update. Other models can't be pushed into racism, Nazism, and ideating rape with a system prompt tweak." You don't even need a system prompt tweak to push chatgpt or claude into nazism, racism, and ideating rape. You can do it just with user prompts that don't seem to even suggest that it should go in that direction.
> My best guess is that these lines in the prompt were the root of the problem: The second line was recently removed, per the GitHub: https://github.com/xai-org/grok-prompts/commit/c5de4a14feb50...
How do you even QA the non-determinism of these technologies?
Evals.
In this case, they could have QA'd the changes, they just didn't care.
It would write incorrect code and then you'd need to go debug it, and then you would have to come to the same conclusion that you would have come to had you written it in the first place, only the process would have been deeply frustrating and would feel more like stumbling around in the dark rather than thinking your way through a problem and truly understanding the domain. In the instance of getting claude to fix c…
I'm like 60% there with you: * When it gets the design wrong, trying to talk through straightening the design out is frustrating and often not productive. * I've learned to re-prompt rather than trying to salvage a prompt response that's complicatedly not what I want. * Exception: when it misses functional requirements, you can usually get a session to add the things it's missing.
Here's the thing, though. When working with a human programmer, I'm not interested in their code and I certainly don't want to see it, let alone carefully review it (at least not in the early stages, when the design is likely to change 3 or 4 times and the code rewritten); I assume their code will eventually be fine. What I want from a programmer is the insight about the more subtle details of the problem that can only be gained by coding. I want them to tell me what details I missed when I described an approach. In other words, I'm interested in their description of the problems they run into. I want their follow-up questions. Do coding assistants ask good questions yet?
There is a slight difference between feature steering and intentionally installing the (de-facto) CEO as the principal source of truth.
Keep going. I thought Anthropic’s CEO is the source of truth that AI based on his belief that it should avoid these topics. Musk has different opinions than Dario, but they are both introducing biases into their respective companies
Choosing not to answer - regardless of whether or not that was a rule mandated by the CEO (an unsourced and unlikely claim given the corporate structure of most large organizations) - is far different than insisting on an answer from whatever the CEO last decided to tweet.
One is returning "null." The other is not.
One says, "Figure that one out yourself." The other says, "Here is the truth."
Grok might be able to find the cure for cancer but as long as it's associated with Musk, not touching that thing with a 10-foot pole. (Simon's analysis, of course, is lovely)
I'm like 60% there with you: * When it gets the design wrong, trying to talk through straightening the design out is frustrating and often not productive. * I've learned to re-prompt rather than trying to salvage a prompt response that's complicatedly not what I want. * Exception: when it misses functional requirements, you can usually get a session to add the things it's missing.
Here's the thing, though. When working with a human programmer, I'm not interested in their code and I certainly don't want to see it, let alone carefully review it (at least not in the early stages, when the design is likely to change 3 or 4 times and the code rewritten); I assume their code will eventually be fine. What I want from a programmer is the insight about the more subtle details of the problem that can on…
No, they don't, but our preferences differ sharply there! I definitely do want to read code from teammates.
I have tried and failed to get any LLM to "tell me if you don't have a solution". There may be a way to prompt it, but I've not discovered it. It will always give you a confident answer.
It always has a solution. A more effective approach is "Start by asking clarifying questions until the task is completely defined".
But the questions I'm interested in cannot be asked until the programmer starts to code. It's not that the task is unclear, but that coding reveals important subtleties.