21–23 of 23 posts
From this thread [0] I can assume that because, while 1.6T, it is A49B, it can run (theoretically, very slow maybe) locally on consumer hardeware, or is that wrong? [0] https://news.ycombinator.com/item?id=47864835
From this thread [0] I can assume that because, while 1.6T, it is A49B, it can run (theoretically, very slow maybe) locally on consumer hardeware, or is that wrong? [0] https://news.ycombinator.com/item?id=47864835
From this thread [0] I can assume that because, while 1.6T, it is A49B, it can run (theoretically, very slow maybe) locally on consumer hardeware, or is that wrong? [0] https://news.ycombinator.com/item?id=47864835
It will be Seconds Per Token instead of Tokens Per Second.
The issue is KV cache