Viewing profile — abetlen

I would add the following two numbers if you're generating realtime text or speech for human consumption: - Human Reading Speed (English): ~250 words per minute - Human Speaking Sp…

Also worth checking out https://github.com/saharNooby/rwkv.cpp which is based on Georgi's library and offers support for the RWKV family of models which are Apache-2.0 licensed.

You can see for yourself (assuming you have the model weights) https://github.com/abetlen/llama-cpp-python I get around ~140 ms per token running a 13B parameter model on a thinkpa…

This is not true, GPT-3 can perform chain-of-thought reasoning through in-context learning either by one/few-shot examples or zero-shot by starting a prompt with "let's think step …

I think you're focusing on a few narrow examples where LLMs are underperforming and generalising about the technology as a whole. This ignores the fact that Microsoft already has a…

My go-to heuristic is three point estimation, basically a weighted average of the best, worst, and average case [0]. (Best + Worst + 4 * Average) / 6 One nice property is that it i…

If you run a Kubernetes cluster for self-hosting software or development I highly recommend setting up a Tailscale subnet router [1]. This will allow you to access any IP (pods or …

pikchr is awesome. A project I did recently was a WASM-compiled pikchr library to generate diagrams directly in the browser [1]. Here's a very early demo of a live editor you can p…

Cool trick, thanks for sharing. I don't get why there isn't a suitable snscanf function that takes the buffer length as an argument and returns the number of bytes parsed?

If you're getting into this stuff a great resource I found is the ZipCPU Tutorial [0] by Dan Gisselquist. The tutorial covers both Verilog design and Formal Verification methods. I…

> I'm pretty sure you could also maintain a temp table and use some kind of "insert...where...returning" construct to squeeze that into a recursive query. I'm not sure if this is p…

The issue I found with WITH RECURSIVE queries is that they're incredibly inefficient for anything but trees. I've looked around and there doesn't seem to be any way to store a glob…

I've actually been working on an extension to perform breadth first search queries in SQLite on general graphs [0]. The extension is actually based off of the transitive closure ex…

My brand new laptop was stolen while I was in school and I had to finish the year with a used 10-inch netbook I borrowed from a friend. Because of the tiny display and underpowered…

I may be misreading that, but isn't the table saying that 82% of residents with dementia require assitance in their daily living activities? From having spent time volunteering at …

I'm not sure if you're from outside North America, but I don't think nursing homes here are likely to have LAN parties unless we make some big medical breathroughs in the next few …

They really were. Here's a version of that video that includes Lorinda Cherry writing a talking calculator by piping together 3 unix commands. https://youtu.be/XvDZLjaCJuw?t=828

You're right. There is also Chilean town / research station that requires all residents to have their appendix removed [1]. [1]: https://en.wikipedia.org/wiki/Villa_Las_Estrellas

Fun fact, all doctors who plan to winter at an Antarctic station must have their appendix removed beforehand [1]. The reason is that they're usually the only doctor available. [1]:…

While you can use recursive common table expressions to perform graph queries, certain queries like breadth-first traversal are not very efficient for anything but trees (not even …

OTP = Open Telecom Platform [1], though I think the full name is avoided nowadays. On an unrelated note, I've always been fascinated with the Erlang VM. The idea that you could hot…

This is true, but won't help very much either. According to Google Support [1], they don't decide what is or is not fair use, leaving it up to the courts. This means the default po…

When I was a teenager, my mom (a psychiatrist) gave me her copy of Frankl's treatise on Logotherapy titlled The Will to Meaning . That book was so over my head, but something that'…