Viewing profile — remilouf

Rémi here, really cool! It kind of turned into a rabbit hole on my end: https://github.com/rlouf/sigil

Of course: https://github.com/rlouf/sigil

It was indeed inspired from my IRC days :)

Original author here, the project has evolved quite a bit since then, you can follow here if that interests you: https://github.com/rlouf/sigil (The $7k was sarcasm)

Author here. Sorry my writing is tedious. Next time I’ll use AI to make it more readable.

> Ironically LLMs solve the MxN problem he's complaining about Enlighten me please

Ooops sorry

Author here. You're right, it's not a hard problem, but a particularly annoying one.

I haven't always done this, and the knowledge base used to visibly degrade over time. Reviewing a PR does not take a long time, maybe a few minutes, and this compounds over time.

This is actually pretty funny.

That’d be a pretty inefficient way to generate bullshit at scale

LLM evaluations are very sensitive to the details of the prompt's structure. This post shows how using structured generation reduces the results' variance and the ranking shifts.

Looks like it’s quite the opposite: http://blog.dottxt.co/performance-gsm8k.html

What do you mean by "semantic dimension"?

That whole structured generation line of work looks promising. I hope someone else takes this and runs evaluations on other benchmarks. Curious to see if the results translate!