Grok 4 Fast now has 2M context window
1–10 of 328 posts
Re: Grok 4 Fast now has 2M context window
#2Anyone can make a long context window. The key is if your model can make effective use of it or not.
Re: Grok 4 Fast now has 2M context window
#3Anyone can make a long context window. The key is if your model can make effective use of it or not.
Long context window = huge amounts of vacant VRAM = our servers are fucking empty
Re: Grok 4 Fast now has 2M context window
#4Anyone can make a long context window. The key is if your model can make effective use of it or not.
How do they make the context window longer? (serious question, I want to learn how this works)
Re: Grok 4 Fast now has 2M context window
#5Anyone can make a long context window. The key is if your model can make effective use of it or not.
Long context window = huge amounts of vacant VRAM = our servers are fucking empty
But isn't context window dependent on model architecture and not available VRAM that you can just increase or decrease as you like?
Re: Grok 4 Fast now has 2M context window
#6Who here actually uses Grok? It's sad to see Elon's arc but when he doubled down on some of his political ideas he had it coming with the Tesla sales going down and x.ai not taken seriously.
I've always tried to remain apolitical and unbiased but it's hard to overlook who's behind a technology you wanna buy. Not that sama and others are saints either, it's just Elon's very obvious and vocal about it.
It's a shame, really, because Grok is a good model. But Elon promised to open source the previous model and it took them forever to do that with Grok 3. Sorry, but I wanna buy from someone who keeps their promises ("FSD by next year").
Re: Grok 4 Fast now has 2M context window
#7Anyone can make a long context window. The key is if your model can make effective use of it or not.
No they can't, it's a N^2 algorithm, just fitting it in the context window is a challenge.
And sure maybe not 2mil of it is usable, but they're reliably pushing the frontier here.
Re: Grok 4 Fast now has 2M context window
#8What matter is not context or the recod token/s you get.
But the quality for the model. And it seem Grok pushing the wrong metrics again, after launching fast.
Re: Grok 4 Fast now has 2M context window
#9I had a failed refactor with Codex recently and I am wondering if context window size is the cause.
Re: Grok 4 Fast now has 2M context window
#10Earlier quoted context omitted.
Long context window = huge amounts of vacant VRAM = our servers are fucking empty
But isn't context window dependent on model architecture and not available VRAM that you can just increase or decrease as you like?
Most attention implementations can work across an arbitrarily long context.
The limiting factors are typically: 1. Often there are latency/throughput requirements for model serving which become challenging to fulfill at a certain context length. 2. The model has to be _trained_ to use the desired context length, and training becomes prohibitively expensive at larger contexts.
(2) is even a big enough problem that some popular open source models that claim to support large context lengths in fact are trained on smaller ones and use "context length extension" hacks like YaRN to trick the model into working on longer contexts at inference time.