Viewing profile — umjunsik132

I think RL as a method which produces training data by model's predictions — It directly leads the model to extend its output range because of increased diversity of the data. Howe…

Hi dang, sorry to ping you here again. I emailed back about a week ago but haven't heard back. My posts are still showing up as [dead] immediately after submission even with non-de…

[dead]

Great project!

I built this because most RL environments are either interesting or accessible, rarely both. Engaging environments like MineRL or Dota 2 either demand massive compute or involve pa…

Overview This model is built on Qwen2.5-Coder-1.5B-Instruct and has been fine-tuned exclusively on the official Roblox Luau corpus. It is designed to assist developers with code ge…

Hi HN, author here. I've always felt that standard benchmarks focus too much on final accuracy, while the architectural choices that get us there are often treated like a dark art.…

For this initial version, I kept the gating static to keep the model as simple as possible while validating the core idea. Making the gate dynamic based on the input is a great sug…

Hi HN Author here. I built FactorizedAttention - a new attention mechanism based on the GWO framework. Instead of simple QK^T dot products, it uses factorized quadratic forms to mo…

Thank you for your comment and for sharing your interesting work. I'll take a look.

I used AI to polish my response. The idea was mine though. My apologies.

That's a fantastic question, and you've hit on a perfect example of the GWO framework in action. The key difference is the level of abstraction: GWO is a general grammar to describ…

Hi HN, author here. For years, it bothered me that convolution (the king of vision) and matrix multiplication / self-attention (the engine of Transformers) were treated as complete…