Viewing profile — deoxykev

I live there in that city. There are hardly any homeless at all here. Not like other cities at least. I could see it being a major problem in other places.

How about LLM chat over DNS? https://github.com/accupham/llm-dns-proxy

And it typically works on captive portals too before payment.

Meta-commentary always leans nerdier.

Curious to hear what kind of work you do. Because there are definitely fields where productivity as 10x'd because of AI tools.

HTMX and shoelace is an awesome combo. Super fast to prototype things and tweak as needed. Being able to copy paste snippets and directly inject data in a straightforward way is a …

Don't forget to finetune the reranker too if you end up doing the embedding model. That tends to have outsized effects on performance for out of distribution content.

Interesting, I had never heard about min-p until now. From what I understand, it's like a low-pass filter for the token sampling pool which boosts semantic coherence. Like removing…

Yeah, there is a clear bottleneck somewhere in llama.cpp. Even high end hardware is struggling to get good numbers. The theoretical limit should be higher, but it's not yet. Benchm…

I don't think autoregressive models have a fundemental difference in terms of reasoning capability in latent space vs token space. Latent space enables abstract reasoning and patte…

The fundemental challenge of using log probabilities to measure LLM certainty is the mismatch between how language models process information and how semantic meaning actually work…

My take: the distills under 32B aren’t worth running. Quants seem to impact quality much more than other models. 32B and 70B unquantized are very good. 671B is SOTA.

8x 3090 will net you around 10-12tok/s

Have you hit any non-determinism errors keeping workflow state outside temporal?

Hey, I’m building agents on top of temporal as well. One of the main limitations is child workflows can not spawn other child workflows. Are you doing an activity for every prompt …

Imhex is a really great frontend for Capstone. https://github.com/WerWolv/ImHex

Are you able to run 405B? 4Bit quant vram requirements are just shy of 192GB.

4 bit quants should require 85GB VRAM, so this will fit nicely on 4x 24G consumer GPUs, plus some leftover for KV cache optimization.

How does this compare to LayoutLMv3? Was it trained on forms at all?

Hi there, I would be interested in a chat about those back-office patterns and use cases. Could you send an email to a2V2aW4gQCBkZW94eSAuIG5ldA==

I use ansible to deploy and sync scripts, services, etc. I think it would work well for you use case as well.

Here is the list from the thread: --- 1. A Man Named Pearl 2. Once Upon a Time in Northern Ireland 3. Microcosmos 4. Crip Camp! 5. Keep The River On Your Right 6. All The Beauty An…

A default key expiry of 3 days would be helpful here too, as it would mitigate the threat of someone compromising the endpoint and extracting secrets from browser history.