Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Grok 4 Fast now has 2M context window

docs.x.ai

Post reply on HN
Print view View original

301–310 of 328 posts

Re: Grok 4 Fast now has 2M context window

#301

Post by acuozzo »

otabdeveloper4 wrote: post #213
ggeorgovassilis wrote: post #28

Earlier quoted context omitted.

I came here just to complain about that :-) All LLMs I used seem to give more weight to things at the beginning of the context window and omit many details. Eg. I tried this simple thing: pasted a friend's and my CV into Gemini and asked it to recommend topics for a joint conference presentation. Results depended greatly on the order of CVs pasted in.

That's because when they say "long context window" they're lying and they actually mean that they support a long input prompt that is still compressed into a small context window. (Typically by throwing out tokens in the middle.) An actually large context window is impossible due to how LLM attention works under the hood.

Mamba-2 enters the chat.

Re: Grok 4 Fast now has 2M context window

#302

Post by vaxman »

OpenAI will go to zero unless it agrees to be acquired because they're messing with public company stock valuations using funky purchase orders leaving those public companies no choice but to cancel their credit (at least unless they get a "government backstop" that they say they don't want or need). Those who compete with OpenAI will also "take a hit" if/when that happens, so they would be wise to be looking to make a deal to acquire OpenAI. Dude was from Y Combinator and liked to bank on hope, focusing on capturing market share and worrying about profits later, which is fine in software startups playing with Monopoly money, but when it impacts vendors that are publicly traded companies (to the point that one is now valued at $5T), post-1929 rules come into play. Anthropic has a similar issue, but there, the issue is that their C-suite is making outrageous public statements that are suspected of intending to manipulate the stock values of both private and public competitors and of the publicly held vendors to all of these players. I hope they both go away quietly and someone declares victory rather than the stock market crashing!

As far as xAI, I doubt it will go to zero or run afoul of any of those market manipulation issues because it owns Twitter/X and I think it powers the realtime Tesla cloud, but betting on it is fraught with peril because of the high likelihood that it will wind up under the control of some less capable conglomerate (ergo, GM acquisition of Hughes Aircraft and resale to Raytheon, Boeing and News/DirecTV).

Google, Meta, a handful of B actors and China are where we have to place our bets, but only if we ourselves need (or want to invest on the theory that others need) trillion parameter models (and want to risk having the valuations lowered if/when adverse actions are taken against the above competitors).

Re: Grok 4 Fast now has 2M context window

#303

Post by sigmoid10 »

cronin101 wrote: post #278
sigmoid10 wrote: post #259

Earlier quoted context omitted.

It's not just that some absolutely require it, but a lot of applications hugely benefit from more context. A large part of LLM engineering for real world problems revolves around structuring the context and selectively providing the information needed while filtering out unneeded stuff. If you can just dump data into it without preprocessing, it saves a huge amount of development time.

Depending on the application, I think “without preprocessing” is a huge assumption here. LLMs typically do a terrible job of weighting poor quality context vs high quality context and filling an XL context with unstructured junk and expecting it to solve this for you is unlikely to end well. In my own experience you quickly run into jarring tangents or “ghosts” of unrelated ideas that start to shape the main thread o…

It depends to the extent I already mentioned, but in the end more context always wins in my experience. If you for example want to provide a technical assistant, it works much better if you can provide an entire set of service manuals to the context instead of trying to put together relevant pieces via RAG.

Re: Grok 4 Fast now has 2M context window

#304

Post by enraged_camel »

Geee wrote: post #118
enraged_camel wrote: post #86

Earlier quoted context omitted.

The guy spends most of his time signal-boosting deeply racist, antisemitic, white supremacist stuff on X. He's obsessed with stuff like "replacement theory" and constantly insists white people must make as many babies as possible to maintain their cultural superiority and avoid being outnumbered by other races. You don't have to believe me. Go check for yourself.

It's extremely disrespectful to call the people from minority and diminishing cultures racist or white supremacist for protecting their own culture. Birth rate / demographic / cultural shifts are real problems. Elon has never talked about "white people" or their superiority. These issues have nothing to do with skin color. Same issues are faced by many asian countries as well.

>> Elon has never talked about "white people" or their superiority

I would encourage you to try to avoid making such easily falsifiable claims, and put at least some token effort into your arguments. I was able to find the below with less than five minutes of searching.

https://www.timesofisrael.com/after-musk-prods-adl-says-kill...

Said tweet: https://x.com/elonmusk/status/1686037774510497792

He also endorsed an X post claiming "Jewish communities have been pushing [...] hatred against whites," calling it "the actual truth." https://www.cbsnews.com/news/elon-musk-antisemitic-comments-...

He has also repeatedly advanced a version of replacement rhetoric (e.g. claiming Democrats import immigrants to change power via the census), which is essentially a repackaging of the Great Replacement idea, i.e. a racist conspiracy centered on replacing white populations with those from other races and ethnicities. You can, for example, read the transcript of his interview with Don Lemon.

So yes, Elon does in fact frequently talk about white people. Even when not explicitly mentioning them, he means them. For example when he says people should have more babies, he specifically means white people: https://newrepublic.com/article/181098/elon-musks-weird-obse...

>> Same issues are faced by many asian countries as well.

I find your comparison of this issue to issues faced by various Asian countries to be pretty odd, as it does not stand up to critical scrutiny. Asian countries' demographic crises are about internal low fertility and rapid aging, not about being "replaced" by outsiders. Indeed, the arithmetic makes the comparison impossible: Japan, China, South Korea all have extremely tiny foreign populations. Therefore, pointing to Japan/Korea/China's low birth rates to sanitize "replacement" talk is a bad-faith pivot.

Re: Grok 4 Fast now has 2M context window

#305

Post by Geee »

enraged_camel wrote: post #304
Geee wrote: post #118

Earlier quoted context omitted.

It's extremely disrespectful to call the people from minority and diminishing cultures racist or white supremacist for protecting their own culture. Birth rate / demographic / cultural shifts are real problems. Elon has never talked about "white people" or their superiority. These issues have nothing to do with skin color. Same issues are faced by many asian countries as well.

>> Elon has never talked about "white people" or their superiority I would encourage you to try to avoid making such easily falsifiable claims, and put at least some token effort into your arguments. I was able to find the below with less than five minutes of searching. https://www.timesofisrael.com/after-musk-prods-adl-says-kill... Said tweet: https://x.com/elonmusk/status/1686037774510497792 He also endorsed an X p…

Demographic change can be caused by just a low birth rate, which is more of an economic issue, but it can also be combined with immigration, which may result in changing culture, i.e. "replacement". This issue is currently mostly faced by people who are white Europeans, but these people also represent many different local cultures. Not to mention that Japan and South Korea have also been increasing their immigration, although it has been quite low so far.

All kinds of people have equal right to defend their own culture. It doesn't mean that they're supremacist or racist, even if they think that their culture is better than some other culture. It's only supremacist if it aims to destroy, repress or subject other people, by advocating discrimination and violence.

Thus, "make more white babies" is not supremacist or racist. As isn't calling out violence against white people in South Africa.

Re: Grok 4 Fast now has 2M context window

#306

Post by SOLAR_FIELDS »

manquer wrote: post #296
SOLAR_FIELDS wrote: post #295

Earlier quoted context omitted.

Hmm. I run maybe 3 work streams max in parallel and struggle to keep up with the context switching. I have some level of skepticism that your colleagues are amazingly better and do 4 and produce quality code at a faster rate than 1 or 2 work streams in wall clock time. I consider a workstream to be disparate features or bugs that are unrelated and require attention. Running 8 agents in parallel that are all doing the…

We have similar definition of streams, but It depends on a lot of things from your tooling/ language , stack etc. if your builds take a fair bit of time (incremental builds may not work in worktree first time) or you are working on a item that has high latency feedback like e2e suite that runs on a actual browser etc. Prompt styles also influences this. I like to make fairly detailed prompt that cover a lot of the nu…

Nice answer - all of the above aligns with my experience.

I use sonnet a lot more than openai models and its speed means I do have to babysit it more and get chattier which does make a difference, probably you are right that if I was using codex which is on average 4-6 times slower than claude code that I would have more mental bandwidth to handle more workstreams.

Re: Grok 4 Fast now has 2M context window

#307

Post by giancarlostoro »

solumunus wrote: post #234
giancarlostoro wrote: post #232

Earlier quoted context omitted.

I would argue over censorship is the better word. Ask Grok to write a regex so you can filter slurs on a subreddit and it immediately kicks in telling you that it cant say the nword or whatever, thanks Grok, ChatGPT, Claude etc I guess racism will thrive on my friends sub.

I can’t tell if this is serious or not. Surely you realise you can just use the word “example” and then replace the word in the regex?!

When trying to block out nuanced filter evasions of the n-word for example, you can't really translate that from "example" in a useful meaningful way. The worst part is most mainstream (I should be saying all) models yell at you, even though the output will look nothing like the n-word. I figured an LLM would be a good way to get insanely nuanced about a regex.

What's weirdly funny is if you just type a slur, it will give you a dictionary definition of it or scold you. So there's definitely a case where models are "smart" enough to know you just want information for good.

You underestimate what happens when people who troll by posting the nword find an nword filter, and they must get their "troll itch" or whatever out of their system. They start evading your filters. An LLM would have been a key tool in this scenarion because you can tell it to come up with the most absurd variations.

Re: Grok 4 Fast now has 2M context window

#308

Post by sd9 »

jorvi wrote: post #292
sd9 wrote: post #229

Earlier quoted context omitted.

I am amazed people actually believe this Grok is the most biased of the lot, and they’re not even trying to hide it particularly well

Bias is not the same as censoring. Censoring is "I'm afraid I can't let you do that, Dave". Bias is "actually, Elon Musk waved to the crowd." Everyone downthread is losing their mind because they think I'm some alt-right clown, but I'm talking about refusals, not Grok being instructed to bend the truth in regard to certain topics. Bias is often done by prompt injection whilst censoring is often in the alignement, and…

They are different, but they’re not that different.

If Grok doesn’t refuse to do something, but gives false information about it instead, that is both bias and censorship.

I agree that Grok gives the appearance of the least censored model. Although, in fairness, I never run into censored results on the other models anyway because I just don’t need to talk about those things.

Re: Grok 4 Fast now has 2M context window

#309

Post by giancarlostoro »

Frannky wrote: post #298

I started with ChatGPT, then moved on to Claude, and then discovered Grok. But now I've stopped paying for any of them. Claude edged out ChatGPT in quality, while Grok stood out with its generous usage limits. That all changed, though, once they rolled out the agent system and RLHF. Suddenly, the model slowed to a crawl, veering off on wrong paths and getting lost in its own reasoning. Those endless, super-annoying R…

I guess I joined Claude late, but its been working pretty decent for me. I've been using Claude Code with Zed now that it's a native feature. Honestly, if you're building coding APIs for your LLM and you aren't working with the Zed folks to get your model natively in that editor, you're messing up big in my eyes, its just done so well.

My biggest gripe with Grok is they're not really integrated in all the great tooling I use. I know I can use an API key with Zed, but come on, you want to compete with something like Claude Code? You need to integrate with the tools devs actually use. If they want to rush on anything, get it on more tools.

Re: Grok 4 Fast now has 2M context window

#310

Post by d4rkp4ttern »

throwuxiytayq wrote: post #225
d4rkp4ttern wrote: post #208

Earlier quoted context omitted.

There are “needle in the haystack” benchmarks for long context performance. It would be good to see those.

These aren’t really indicative of real world performance. Retrieving a single fact is pretty much the simplest possible task for a long context model. Real world use cases require considering many facts at the same time while ignoring others, all the while avoiding the overall performance degradation that current models seem susceptible to when the context is sufficiently full.

I agree, retrieving a single fact is necessary but not sufficient.
Post reply on HN