I'm working on lots of projects. My favorite is what I call "context bonsai" where I'm giving LLM harnesses the ability to surgically edit the context. It's available as a tool. You can say "remove that failed debugging session and write a summary of what we learned." Or you can take a more hands-on approach and say "remove messages msg_ID1 through msg_ID2". The removal leaves a summary and keywords, and the original…
I did some experiments by exposing the raw latent states, using hooks, of a small 1B Gemma model to a large model as it processed data. I'm curious if it is possible for the large model to nudge the smaller model latents to get the outputs it wants. I desperately want to get thinking out of tokens and into latent space. Something I've been chasing for a bit.