Viewing profile — angusturner

I find the presentation of this as "factoring training" very confusing. Factoring is something we do to distributions, which we then parameterize with our neural net. An autoregres…

I immediately thought of IWAE as well.

In 2017 I worked tirelessly with my colleagues to implement and replicate the first transformer paper. Yesterday I left Opus 4.8 to go do some architecture research, with GPU acces…

For small to medium projects, an LLM can write functional (if not well crafted) Rust. Considering how easy this is now, why choose a heavier, slower and less typesafe language?

why read that, vs an actually well-written compiler though?

[flagged]

I thought exceptions tended to be made when its highly relevant to the technical topic at hand and also non controversial. Outside a few weird online bubbles and pockets of the US,…

I feel this. I've had a few tasks now where in honest retrospect I find myself asking "did that really speed me up". Its a bit demoralising cause not only do you waste time, you ha…

I think most SWEs do have a good idea where I work. They know that its a significant, but not revolutionary improvement. If you supervise and manage your agents closely on well sco…

Interesting... Maybe I need to investigate PayPal as an option here. Best case would be my bank eventually adds tap to pay natively

I recently made the shift to graphene from iOS and am mostly enjoying it. The user profiles was slow to set up and not having shared filesystem between the user profiles creates fr…

Yeah, worlds slowest and most in-efficient write-only database. And as soon as you need to interact with goods or services in the real world, then you still need trust anyway. All …

Fuck google for this. Awful decision. Guaranteed to be abused when Google or government despots decide that certain apps (or developers) aren't aligned with their interests. Feelin…

You mean the guy that bans people from twitter for disagreeing with him? And has made a chatbot that spouts right-wing conspiracies in the name of being "anti-woke"?

As an Aussie, I was feeling somewhat consoled about the state of the US by the fact that the EU and UK still seem to have their heads screwed on. OSA and chat control have made me …

I wish people would stop parroting the view that LLMs are lossy compression. There is kind of a vague sense in which this metaphor holds, but there is a much more interesting and r…

There is an excellent talk by Jack Rae called “compression for AGI”, where he shows (what I believe to be) a little known connection between transformers and compression; In one vi…

By the definition you have provided though, someone that has access to stable, safe or functional housing but then chooses to not to use it (eg opting to camp instead), is not home…

Agree the media is having a field day with this and a lot of people will draw bad conclusions about it being sentient etc. But I think the thing that needs to be communicated effec…

One under appreciated / misunderstood aspect of these models is they use more compute than an equivalent sized autoregressive model. It’s just that for N tokens, autoregressive mod…

You assume that for small steps (I.e taking some noisy code and slightly denoising) you can make an independence assumption. (All tokens conditionally independent, given the curren…

Hm suppose for argument sake that feeding a batch of data through some moderately large FF architectures takes on the order of 100ms (I realise this depends on a lot parameters - b…

I’m really glad to see people converging on this view because I feel a bit insane for not understanding all the hype. Like, yeah, we need a standard way to connect LLMs with tools …