31–40 of 47 posts
Earlier quoted context omitted.
I do, for sure! Yes, we have a few product rollouts lined up. The differentials for latency are posted in our blog post, so that should provide an idea of where the scaling law differentials kick in.
> I do, for sure! Yes, we have a few product rollouts lined up. When, more or less?
Earlier quoted context omitted.
We are SOTA in some ways and not in others, continuously working to make it better! We need a little more time to scale, as we are working on things like disaggregated prefill, etc., the norms of large-scale model infra. I am happy to answer any questions!
I have questions. Can you back up your claims? Why did you not release the white paper in parallel with the product? Feels really fishy.
> The core idea is content-dependent selection. For each query, the model selects which parts of the sequence are worth attending to, and computes attention exactly over those positions. I don't know if this will help for things like understanding code, where the all relevant parts can be the file of 1000 lines that we are analyzing, and where every token is relevant in understanding recursion, loops, function calls,…
- magic.dev claimed 200M context window and it's been two years since and no real product yet. - They are admitting that this is built on top of a Chinese model[1] - They committed a huge chart crime with the Y axis of a chart comparing to Opus on their website that I can't find anymore (Too embarrassing to keep?). The delta between their score (81%) vs. Opus (87%) on SWE bench was hugely minimized - They named the c…
The chart crime was not intentional! We will not make you wait two years. We are O(n), not O(1). O(1) would unfortunately be an impossibility. We may as well do infinite context at that point!
Earlier quoted context omitted.
We are SOTA in some ways and not in others, continuously working to make it better! We need a little more time to scale, as we are working on things like disaggregated prefill, etc., the norms of large-scale model infra. I am happy to answer any questions!
I have questions. Can you back up your claims? Why did you not release the white paper in parallel with the product? Feels really fishy.
If I came up with a novel thing I'd monetise it first, because publishing it makes it part of the training that adds value to billion dollar corps with zero credit to me.
In the old knowledge economy I benefited from the credit assigned to me.
So, to me, nothing fishy at all.
Earlier quoted context omitted.
I have questions. Can you back up your claims? Why did you not release the white paper in parallel with the product? Feels really fishy.
What do you want in a whitepaper that was not in our blog post? There is time to add more before the whitepaper is released.
- magic.dev claimed 200M context window and it's been two years since and no real product yet. - They are admitting that this is built on top of a Chinese model[1] - They committed a huge chart crime with the Y axis of a chart comparing to Opus on their website that I can't find anymore (Too embarrassing to keep?). The delta between their score (81%) vs. Opus (87%) on SWE bench was hugely minimized - They named the c…
i see in the linked post they mention O(n) not O(1). O(1) would basically be impossible and instant. Something like no compute required, constant results...
The name subquadratic is actually good and makes sense to me. Because today's models are usually O(n^2) or worse. Anything equals or less than O(n^1) is basically sub-quadratic.
Meanwhile O(log n) would be logarithmic as the log name indicates. But we have a long way to go there. Maybe with double tokenizer plus extensive caching it may be possible...
What I mean here is tokenizing the user input; then capturing intent; caching intent -> response. So that next time once you get the intent, you don't need to do full transformer inference compute. This can be logarithmic complexity in terms of time complexity.