Viewing profile — scoresmoke

Before I buy, can you confirm the bridge is GDPR-compliant, AI-Act-ready, has a digital product passport, and passed its environmental impact assessment? Otherwise the local compli…

Yes.

I am working on post-training and evaluation tasks mostly, and I built Evalica as a convenient tool for my own use cases. The computation is fast enough to not bother the user, but…

I am thinking about this for a while and I think I’ll vibecode them. Not sure about WASM, though, as the underlying libraries should support it, too, and I am not sure about all of…

You might also consider a fast implementation of Elo and Bradley–Terry that I have been developing for some time: https://github.com/dustalov/evalica (Rust core, Python bindings, 1…

GPT-2 follows the very well-studied architecture of Transformer decoder, so the outcomes of this study might be applicable to the more complicated models.

Ruff and uv are both excellent tools, which are developed by a VC-backed company, Astral: https://astral.sh/about . I wonder what their pitch was.

The most important changes are deprecations of certain public APIs: https://numpy.org/devdocs/release/2.0.0-notes.html#deprecati... One new interesting feature, though, is the supp…

Thank you! I excluded the coding tasks as most annotators don't possess this expertise. I trust them in comparing pairs of dissimilar model outputs that don't require any specific …

In September 2023, I noticed a tweet [1] on difficulties with LLM evaluation, which resonated with me a lot. A bit later, I spotted a nice LLMonitor Benchmarks dataset [2] with a s…

Yet they have a copyright notice in the footer: Copyright © 2023, Oracle and/or its affiliates.

It's all about timing and communities. In my echo chamber, everybody is currently making them a part of the product, but I'm interested in a more representative sample. Hiring usua…

I found code LLMs to be very useful for rewriting nested for-loops in Python as nice vectorized operations in NumPy. One has to be careful about unexpected array materialization, b…

I have seen that many teams are trying to perform model routing by sending simpler queries to cheaper models and complex queries to more complex and expensive models, e.g., https:/…

Since the LLMs and other generative models had been around for a while, many companies claimed to build products on top of them or launch new AI features based on them. What happen…

Discussions about LLM alignment often miss topics of data quality and quantity. It turns out that current models like Llama 2 use 10K+ prompts and responses for supervised fine-tun…