Viewing profile — olliepro

The authors have some inconsistencies with training token length… Most errors are probably responses that didn’t finish before their 3K token limit. They’ve measured how well RL is…

This is the classic pattern of LLM generated MCQs.

With super high res onboard camera footage too.

They do quite a lot of distillation. As we've seen from the American open weight models from AI2 (OLMo series of models). They have a lot of incentive to distill beyond just copyin…

A lot of distillation happens. E.g. OLMo models have a completely open dataset and they are heavily distilled. It only makes sense to try to absorb behaviors from the best models o…

decentralized training makes a lot more sense when the required hardware isn't a $40K GPU...

This would likely only get used for small finetuning jobs. It’s too slow for the scale of pretraining.

I bet they lack good long context training data and need to start a flywheel of collecting it via their api (from willing customers)

Tensors are in no shortage nowadays. I did read this a tensors though and got a good laugh.

There’s a section of I-15 in Utah’s Salt Lake County which reliably has a crash on weekdays at 6pm. It was unfortunately at a pinch point in the mountains with no good alternate ro…

Much of the scientific medical literature is behind paywalls. They have tapped into that datasource (whereas ChatGPT doesn't have access to that data). I suspect that were the medi…

It depends on your thing. If the marathon was just the motivation, your thing is running... if the marathon was the bucketlist item, it is the thing.

Getting everyone to fall in love with the thing is not doing the thing... learned this as a data scientist brought in to work on a project which ended soon thereafter. A team of 20…

Everyone's threshold is different. I aspire to "move fast and break things", but more often than not, I obsess over the rough edges.

The more I use AI to do the thing, the more it feels like I didn't do the thing.

What abstraction levels do you expect will remain only in the Human domain? The progression from basic arithmetic, to complex ratios and basic algebra, graphing, geometry, trig, ca…

I made a skill that reflects on past conversations via parallel headless codex sessions. Its great for context building. Repo: https://github.com/olliepro/Codex-Reflect-Skill

I was thinking about something like this, but I don't have codex running on a server. Keep me posted on how it goes!

This skill is useful for identifying agent friction points and brainstorming new skills, developing context of past work for a new conversation, identifying code bloat from failed …

I believe the idea is that it “files away” the files into folders.

Lol

Can Claude code jump through the hoops for you?

Three things that shook me awake to the idea that the information barrage of the internet is a tranquilizer/red herring: - Bad Mental Health: At the start of the war in Ukraine I r…

Although there are many examples of troubling sycophantic responses confirming or encouraging delusions, this document is the original complaint (the initial filing) in a lawsuit a…

It feels like this should work, but the breadth of knowledge in these models is so vast. Everyone knows how to taste, but not everyone knows physics, biology, math, every language……