Viewing profile — danking00

Wow! At first, I expected this to be a demonstration of an AI playing rocket league, but I rapidly realized this is actually a model simulating rocket league. Wild! It feels just l…

Feather appears to just be block compressed Arrow IPC [1]. Lightweight compression techniques generally achieve two orders of magnitude faster random access compared to block compr…

cppreference says [1]: A prvalue of floating-point type can be converted to a prvalue of any integer type. The fractional part is truncated, that is, the fractional part is discard…

I think one of my favorite examples of avoiding decompression for great profit (really: health) is PLINK. If you’re not aware, it’s a tool for doing statistical genetics (things li…

We’ll post a blog post soon with specific, benchmarked numbers, but, in this case, you can have your cake and eat it too! The compression and decompression throughputs of Vortex (a…

Thanks for introducing me to these other formats! I hadn't heard of them yet. All three of fst, JDF, and Vortex appear share the goal of high throughput (de)serialization of tabula…

Yeah we should be more clear in our description about how our footers differ from Parquet. Parquet is a bit more prescriptive; for example, it requires row groups which are not req…

Yeah, you and us are on the same page (heh). We don’t want the format to require row grouping. The file format has a layout schema written in a footer. A row group style layout is …

See also one of the author's twitter threads: https://x.com/cureffi/status/1806404950055862547

Unless I’m misreading, the first two references support the idea that vacancy rates are too low but don’t comment on why . The last reference indeed argues in favor of too much fin…

Thanks for the reply and clarification! That’s an impressive throughput. Am I interpreting the behavior correctly to think that: your code has one branch per int and that branch wi…

If I’m following, this presents an alternative format and implementation to LEB128 which encodes and decodes substantially faster. Notably, the implementation is quite simple. Cool…

I think it might help readers to include a narrative about an example application. Perhaps I’m in the minority but I tend to think of Bloom filters as a way to reliably know someth…

There’s also a lighter weight Nature News article about it https://www.nature.com/articles/d41586-024-01449-y

I was surprised by the last clause in your last sentence, but you’re absolutely right. https://www2.census.gov/geo/docs/reference/ua/ua_list_all.tx... LA edges out NYC: 7000 to 530…

I gathered a table for Google and Amazon's options. I do not have experience with on-prem solutions so I don't know how to compare these prices to the cost of owning and operating …

It'd be interesting to see a peak-sequential-bandwidth by cost-per-gigabyte plot. The number I keep in my head is 500 MiB/s, but you're right that there are much faster drives out …

Most effective learning experience I ever had was reading the relevant book chapter before class, taking some broad notes to structure my understanding, and then asking questions d…

Thanks for all your comments thus far! I have an axe to grind with the US immigration system but it’s best to grind it on a stone of facts. Yeah, I was a bit loose with the parents…

For India alone, Wikipedia says ~8% or ~100 million have a BS. I was more focused on PR/citizenship in my post. I do not know what qualifies you for an H1-B. In the second point, I…

Fair on the first point, I’ll adjust the text to clarify the need for a BS. On the second point, first claim, you can be in an EB-1 queue without an H1-B (e.g. OPT or TN [1]). On t…

I suppose majority is factually true, but I think it’s worth being specific that nearly three billion people are in that situation.

I’d be delighted to be proven wrong but the ten year minimum experience comes from the second reference: > Letters from current or former employers documenting at least 10 years of…