Viewing profile — bradhilton

The problem is that they have a lot of time to report their purchases. If they were required to report before they purchased the problem would probably resolve itself.

The SP500 is probably the most popular investment in America, perhaps aside from housing. Wouldn't hurt to have lawmaker's fortunes broadly aligned versus narrowly aligned with spe…

Awesome! If you run into any problems or have questions feel free to open an issue or drop by the discord [1] server. [1] https://discord.gg/zbBHRUpwf4

Hi, we don't have reliable documentation for the HTTP API endpoints yet, mostly as they are still subject to change. However, to briefly provide some context, `/_train_model` retur…

Contributor here, we developed the Agent Reinforcement Trainer (ART) library to make it easy to train LLMs for anything. No callbacks or straitjacket flows. Instead we serve an Ope…

I could see training your own email agent being beneficial for products like this: https://x.com/advaitpaliwal/status/1913290027897131084

I know Google DeepMind ran experiments with 10M a while ago, but I think this will be the first legit, released 10M context window model.

Yes, pedantically, it is! But as I said, everything's on a spectrum. Online-ish data can still work just fine.

We used about 58 hours on 4xH100s and about 19 hours on 8xH100s to get the very best result with the 32B model. We trained for about another 16 hours before finishing the run, but …

Well, in this case there is a much more straightforward method with the same CP-SAT solver used to create the puzzles. This is more of a fun experiment to see if we can train LLMs …

Technically yes, only if you do a gradient step with data sampled from the exact same weights is it an online step. With our training recipe this can be easily done by accumulating…

The model is rewarded for accuracy. For each puzzle there are a few multiple choice questions. If it got 1 out of 4 correct, for example, its reward would be 0.25. Then group relat…

Yeah, the takeaway shouldn't be "our model is smarter," but that we were able to train weak models to as good or better than the best for this specific task. Depends on what you're…

We updated the first paragraph to define the acronym. Thanks again for the feedback!

Great question! So the dataset includes prompts and solutions, but no "gold" answer per se to use for SFT. You could sample responses from larger models and then train the smaller …

Great point! Thanks for the feedback.

Yeah, it may help. In this paper[1], the author used a KL penalty of 0.01 for general tasks and 0.001 for mathematical. I tend to think it's probably not very important unless you'…

We trained all the parameters. Those would definitely be interesting ablations. I would also like to see how much of a performance hit we would take with PEFT methods like LoRA.

No meaningful changes to the hyperparameters, just changed the tasks per iteration to 16 and trained on the same first 16 training tasks each iteration. We only tested this with th…

As for why they dropped suddenly , I don't really know. Sometimes models develop degenerate behaviors, but even when forking from the best checkpoint and lowering the learning rate…

Hi, other author here. I think the models converged on shallow/greedy strategies that improved performance up to a point, but are ultimately shortsighted, especially for harder puz…

They need to be trained with a small length penalty

These are distillation fine-tunes of two different models: - Qwen2.5 7B - Llama3.1 8B Though the sizes are similar, they will probably have different strengths and weaknesses based…

Beats Gemini 1.5 Pro at all but two of the listed benchmarks. Google DeepMind is starting to get their bearings in the LLM era. These are the minds behind AlphaGo/Zero/Fold. They c…

Okay, that was really impressive. Well done!