Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Fire-Flyer File System (3FS)

github.com

Post reply on HN
Print view View original

71–80 of 106 posts

Re: Fire-Flyer File System (3FS)

#71

Post by jpgvm »

cyanf wrote: post #48
jpgvm wrote: post #31

Earlier quoted context omitted.

Well current popular systems are pretty much limited to Lustre and the new kid Weka, mostly Lustre though tbh. You can try to use "standard" options like MinIO/Ceph(RADOS)/SeaweedFS but you will very quickly learn those systems aren't remotely fast enough for these usecases. AI training is what this is used for, not inference (which has absolutely no need for any filesystem at all). What makes the workload somewhat s…

They’re using the FS for caching the KV caches of past requests. It’s why they’re able to charge so little on prompt cache hit.

Ahh I missed that. Yes prefix caching and RAG are 2 cases were you will want something like this during inference time.

Re: Fire-Flyer File System (3FS)

#72

Post by smallmancontrov »

tway223 wrote: post #57
codingwagie wrote: post #51

I think the difference between deepseek and OpenAI/Anthropic is one of the difference between practitioners and academics. Ofcourse there is world class talent at OpenAI. But there are also alot of "I went to Harvard and want to work in AI", and those types of people just simply dont have the technical exposure to even think of building something like this.

I would say most if not every large company in China has their own AI infra stack, partially because tech talent is relatively more abundant and partially some of the tech leads have been exposed to western tech via open source and work experience so they have a good success rate (which makes it a more common practice). Anecdotally, specifically Google, FB ex-employees from oversea offices, MSFT and Intel ex-employee…

As opposed to the US, where every large company has its own AI infra stack, often extending down to the silicon and up to large open source projects?

What's going on here, why are people forgetting what's around them? Does familiarity breed contempt? Are attention spans so shot that failure to participate in this week's news cycle is enough for "out of sight, out of mind"? Or is HN full of Chinese bots now?

Re: Fire-Flyer File System (3FS)

#73

Post by bugglebeetle »

mustpax wrote: post #55
codingwagie wrote: post #51

I think the difference between deepseek and OpenAI/Anthropic is one of the difference between practitioners and academics. Ofcourse there is world class talent at OpenAI. But there are also alot of "I went to Harvard and want to work in AI", and those types of people just simply dont have the technical exposure to even think of building something like this.

Someone should write a blog post about the prestige/effectiveness negative feedback loop. This is also the Achilles heel of top tier SV VCs including YC.

The problem isn’t the prestige it’s that prestigious institutions in America don’t produce high-quality talent. They’re instead mostly corrupt credentialing mills for the rich and well-connected. From what I understand, DeepSeek also only hires from the best universities in China, but “best” actually means something relative to how difficult entrance to those organizations is to achieve and their coursework.

Re: Fire-Flyer File System (3FS)

#75

Post by htrp »

hintymad wrote: post #69

A distributed file system is honed as one of the trickiest software to write, and we are usually advised not to write a file system from scratch (even on top of FUSE), let alone a highly optimized one. When a silicon value company is having the 100th meeting to align god-knows-what, a team of fewer than 60 already came up with a production-grade highly efficient parallel file system. Have we in the valley companies l…

> team of fewer than 10

the highflyer team are pretty well resourced.... think they have more than 10 people

Re: Fire-Flyer File System (3FS)

#76

Post by cma »

codingwagie wrote: post #51

I think the difference between deepseek and OpenAI/Anthropic is one of the difference between practitioners and academics. Ofcourse there is world class talent at OpenAI. But there are also alot of "I went to Harvard and want to work in AI", and those types of people just simply dont have the technical exposure to even think of building something like this.

Weren't the flash attention authors not just from academia but in academia at the time?

Re: Fire-Flyer File System (3FS)

#77

Post by htrp »

bee_rider wrote: post #60
genewitch wrote: post #54

Earlier quoted context omitted.

Do you, though? Need the best of the best?

As someone who did some simulation focused engineering grad school stuff; there is a tendency for some of the best to go become quants. Does the field need it? I don’t know. But for whatever reason the draw of the “print money using math tricks” seems to attract some hardcore folks, haha. It is really frustrating to see good engineers go to play trading games. We should study how exactly it is China managed to unlock…

Government effectively banned unproductive tech (adtech, fintech etc) and told ppl to go do stuff like robotics and AI

Re: Fire-Flyer File System (3FS)

#78

Post by rfoo »

Forbo wrote: post #74
digdugdirk wrote: post #46

Earlier quoted context omitted.

996 work culture?

That's been illegal for three and a half years?

That's been illegal since May 1995 (before that China had six working days week).

Does it really matter whether it's illegal or not, if there is no enforcement? Pinduoduo (in other name, Temu) has been doing 70 hours week since they started. Yes, they are still doing it right now.

Re: Fire-Flyer File System (3FS)

#79

Post by rfoo »

jpgvm wrote: post #31
yalogin wrote: post #30

It’s not clear to me where and how the current popular systems fall short. Do they talk about I anywhere? Also, what specifically is the data access patterns for training and inference that are different from traditional use cases?

Well current popular systems are pretty much limited to Lustre and the new kid Weka, mostly Lustre though tbh. You can try to use "standard" options like MinIO/Ceph(RADOS)/SeaweedFS but you will very quickly learn those systems aren't remotely fast enough for these usecases. AI training is what this is used for, not inference (which has absolutely no need for any filesystem at all). What makes the workload somewhat s…

> What makes the workload somewhat special is

I'll add that latency also doesn't matter that much. You are doing batched data loading for batch n+1 on CPU when GPUs are churning batch n-1 and copying batch n from host memory at the same time.

So as long as your "load next batch" doesn't run for like >1s it would be fine. But one single "load next batch" on one worker means thousands (if not more) random read.

Re: Fire-Flyer File System (3FS)

#80

Post by rfoo »

startupsfail wrote: post #64
rfoo wrote: post #19

Earlier quoted context omitted.

Huh? What kind of RDMA has a completion latency of 20 nanoseconds? It's more like 5 microseconds. I agree that a lot of "modern" storage stack is way too slow though, tried to find a replication-first object storage for crazy-fast random read in small number of objects last year and found none.

I was talking about, thinking in terms of 20 nanoseconds intervals, rather than completing a request in 20 nanoseconds. To get 1 microsecond wire-to-wire latency you do need to count your nanoseconds. Why this number - this is because it’s roughly the time it takes to read 64 bytes from L3 cache. And NICs tend to be able to push data into L3 (or equivalents). Current state of the art - look up nanoPU, from Stanford.…

Ah, makes sense. Sadly RDMA isn't that fast for now, or at least commercial RNICs/switches don't :( Once you left your host in data center network, everything counts in microseconds.
Post reply on HN