Viewing profile — kiraaa

when there are two commands in a prompt example do A and then do B. the model completely ignores the second task B.

on gpu that is still huge.

super easy to install and use

mistral 7b v0.2 supports 32k

could be true, we can only speculate.

maybe they are using ring attention, on top of their 128k model.

really like the font and great article btw.

the paper does not live up to the quality of model lol

you need 94gb, does not matter which RAM.

https://github.com/sysid/sse-starlette makes token streaming so much easier in python

and also easy to deploy

its not alway about the size, but yeah its really good!

installing it is a nightmare

you could but it will be very slow. oh and make sure the code is cpu compatible.