Viewing profile — ban-lan-gen

If I have a lot of precomputed KV cache values (basically just tensors), how do I store them efficiently on disk? Ideally, I would want to lookup their values by prefix quickly as …

Lookup the meaning of theoretical

Does this only apply to PDFs with many images? If it is mostly text and table, could it just extract plain text?

When I browsed GitHub repos. I came across some repos that implied "Because my thing is written in Rust, it automatically comes with a good performance, good scalability, etc". Jum…

I don't understand all those rust hypes here. What indeed does rust bring to the table?

Something like that 4^30 >>> 8^13?

They just want to sell some expensive hardwares?

Haha?

Minor thing: the link redirects to the org but not the actual repo.

I finally have some evidence to stop my wife's complaints.

Just curious what kinds of models are used in your project? https://arxiv.org/abs/2306.08891 One paper seems to suggest trained specialized model can outperform LLM in some tasks.