Viewing profile — metawake

Great suggestion!! this is exactly the right methodology for establishing confidence intervals. I've added this to the roadmap as `--bootstrap N`: ragtune simulate --queries querie…

Author here. Built RagTune to stop guessing at RAG configs. Surprising findings: 1. On legal text (CaseHOLD), 1024 chunks scored WORST (0.618). The "small" 256 chunks won (0.664). …

Thanks! To answer your questions: *Backends:* Currently supports Qdrant, pgvector, Weaviate, Chroma, and Pinecone. Adding more is straightforward since it's just implementing a Sto…

CLI tool to debug and benchmark RAG retrieval without LLM calls. - `ragtune explain "query"` → see what was retrieved with scores - `ragtune simulate` → batch eval with recall/MRR …

I am using a vector DB using Docker image. And for debugging and benchmarking local RAG retrieval, I've been building a CLI tool that shows what's actually being retrieved: ragtune…

I made a small project ( https://github.com/metawake/puppetry-detector ) to detect this type of LLM policy manipulation. It's an early idea using a set of regexp patterns (for spee…

Hi HN, I’ve been exploring whether prompt compression — done before sending input to LLMs — can help cut down on token usage and cost without losing key meaning. Instead of using a…