Viewing profile — mrlongroots

In my experience, code is a small fraction of the work. I'm in an infra team and for the last 2 weeks or so I've been trying to understand whether a particular workload will catch …

Opinionated take: ingestion should be treated as a streaming reorganization workload, separate from whatever a "database" is. You do not even need to change the Iceberg spec, altho…

> And what might be the outcome? DoE compute budgets are ~10B USD across labs. AI training is a trillion-dollar workload. Different league.

All that would not help you with an AI training cluster interconnect. See Amin Vahdat's keynote at HotInterconnects 2025. Everyone is building a fabric for this stuff from scratch …

It is partly this and partly a funding vehicle for American next-gen computing. It is not that hard to estimate FP64 ballpark from a whole bunch of public statistics. And it takes …

Today's limits are known and undisputable. Tomorrow's limits are a promise: some promises over-deliver, others under-deliver. :) Regardless, to bring the discussion back to the cla…

> With alzheimer's an autopsy can tell for sure but that's not much help for a patient. Ok let us unpack this statement. For your point to hold, I would have to be saying "all kind…

Pyschiatry gets complicated because the failures are not mechanical. Even if you could image every single neuron in a person's head we do not have a very good way to define an algo…

> I think „the diagnosis” is over simplification and lots of professionals would disagree that there’s always a single one. "The Diagnosis" does not mean "one root cause". Situatio…

> There is no guarantee that the LLM will help you converge on anything. Absolutely. The guarantee does not come from the LLM. The LLM is a simply an improved version of Google Sea…

Yeah I think the OP is muddling the point by conflating "physician's version of the diagnosis" with "The Diagnosis". There is absolutely one "The Diagnosis". Human body is a machin…

Maybe I am missing something but I just find this wrong. Everything is a puzzle: there is one "Truth" or one diagnosis. You (a smart human) should be able to converge on it by cros…

> Yes, they don't realize it or lie to themselves because ~50% dropout. I think there's some misinterpretation here. Not staying on in academia after PhD (common/modal) is not the …

As someone who graduated with a 7.5 year long PhD last month, I feel like PhD stipends are not a major problem. Like I got $40K in a low CoL area, but accounting for tuition and ov…

That training is compute-bound and inference is memory-bound is well-known, but I don't think Nvidia deployments typically specialize for one vs the other. One reason is that most …

MapReduce is nice but it doesn't, by itself, help you reason about pushdowns for one. Parquet, for example, can pushdown select/project/filter, and that's lost if you have MapReduc…

Algebras are also nice for implementations. If you can decompose a domain into a few algebraic primitives you can write nice SIMD/CUDA kernels for those primitives. To your point, …

Yes, GPT5-series thinking models are extremely pedantic and tedious. Any conversation with them is derailed because they start nitpicking something random. But Codex/5.2 was substa…

While Arrow is amazing, it is only the C Data Interface that can be FFI'ed, which is pretty low level. If you have something higher-level like a table or a vector of recordbatches,…

Hyperscalers do not need to achieve parity with Nvidia. There's a (let's say) 50% headroom in terms of profit margins, and plenty of headroom in terms of the complexity custom chip…

Yeah unfortunately no amount of manoeuvering is a substitute for a kill chain where a distributed web of sensors and relays and weapon carriers can result in an AAM being dispatche…

The appropriate comparison point for aggregate cluster storage bandwidth would be its bisection bandwidth. (I do HPC, IIRC ANL Aurora is < 1PB/s DAOS and 20 PB/s bisection).

I think I'm talking about cluster-scale network bisection bandwidth vs attached storage bandwidth. With replication/erasure coding overhead and the economics, the order of magnitud…

Yep I think the value of the experiment is not clear. You want to use Spark for a large dataset with multiple stages. In this case, their I/O bandwidth is 1GB/s from S3. CPU memory…

> LDL-C is much much cheaper to measure. ApoB costs 36x times as much, so Insurance Companies don't like to pay for it Unfortunately American retail prices might as well be generat…