Viewing profile — tmostak

GPU-accelerated databases have a long history. I founded HeavyAI (previously MapD/OmniSci) in 2013, but there are or have been many other startups in this space, such as Voltron Da…

Evidence (preferably with recent Teslas/HW4)?

Evidence of this? I own a Tesla (HW4, latest FSD) as well as have taken many Waymo rides, and have found both to react well to unpredictable situations (i.e. a car unexpectedly tur…

Do you have data to back this claim up, specifically with HW4 (most recent hardware) and FSD software releases?

Even without NVLink C2C, on a GPU with 16XPCIe 5.0 lanes to host, you have 128GB/sec in theory and 100+ GB/sec in practice bidirectional bandwidth (half that in each direction), so…

We've made extensive use of perfect hashing in HeavyDB (formerly MapD/OmniSciDB), and it has definitely been a core part of achieving strong group by and join performance. You can …

This looks amazing! Just looking through the code a bit, it seems that the model both supports a (custom) attention mechanism between features and between rows (code uses the term …

You should be able to train/full-fine-tune (i.e. full weight updates, not LoRA) a much larger model with 96GB of VRAM. I generally have been able to do a full fine-tune (which is e…

This assumes that you can linearly scale up the number of TPUs to get equal performance to Nvidia cards for less cost. Like most things distributed, this is unlikely to be the case…

Are you measuring tokens/sec or words per second? The difference matters as generally in my experience, Llama 3, by virtue of its giant vocabulary, generally tokenizes text with 20…

But it's likely to be much slower than what you'd get with a backend like llama.cpp on CPU (particularly if you're running on a Mac, but I think on Linux as well), as well as not s…

Thank you, it's been a major team effort!

We've been working hard for the last year on building HeavyIQ, an LLM-powered plugin to the GPU-accelerated HeavyDB database, that allows users to ask natural language questions an…

More info can be found here: https://www.heavy.ai/heavyiq/overview