221–230 of 416 posts
anybody care to speculate on how long this is likely to last? is this a blip that will resolve itself in six months, or is this demand sustainable and we are talking years to build up new manufacturing facilities to meet demand?
Anyone want to start a fab with me? We can buy an ASML machine and figure out the rest as we go. Toronto area btw
Hear me out- artisanal DRAM.
I know this is mostly paranoid thinking on my behalf, but it almost feels like this is a conscious effort to attempt to destroy "personal" computing. I've been a huge advocate for local, open, generative AI as the best resistance to massive take-over by large corporations controlling all of this content creation. But even as it is (or "was" I should say), running decent models at home is prohibitively expensive for m…
My kids use personal computing devices for school, but their primary platform (just like their friends) is locked-down phones. Combining that usage pattern with business incentives to lock users into walled gardens, I kind of worry we are backing into the destruction of personal computing.
Earlier quoted context omitted.
large language models are large and must be loaded into memory to train or to use for inference if we want to keep them fast. older models like gpt3 have around 175 billion parameters. at float32s that comes out to something like 700GB of memory. newer models are even larger. and openai wants to run them as consumer web services.
I mean, I know that much. The numbers still don't make sense to me. How is my internal model this wrong? For one, if this was about inference, wouldn't the bottleneck be the GPU computation part?
Suppose some some parallelized, distributed task requires 700GB of memory (I don't know if it does or does not) per node to accomplish, and that speed is a concern.
A singular pile of memory that is 700GB is insufficient not because it lacks capacity, but instead because it lacks scalability. That pile is only enough for 1 node.
If more nodes were added to increase speed but they all used that same single 700GB pile, then RAM bandwidth (and latency) gets in the way.
Every shortage is followed by a glut. Wait and see for RAM prices to go way down. This will happen because RAM makers are racing to produce units to reap profits from the higher price. That overproduction will cause prices to crash.
I think the OpenAI deal to lock wafers was a wonderful coup. OpenAI is more and more losing ground against the regularity[0] of the improvements coming from Anthropic, Google and even the open weights models. By creating a chock point at the hardware level, OpenAI can prevent the competition from increasing their reach because of the lack of hardware. [0]: For me this is really an important part of working with Claud…
Please explain to me like I am five: Why does OpenAI need so much RAM? 2024 production was (according to openai/chatgpt) 120 billion gigabytes. With 8 billion humans that's about 15 GB per person.
This reminds me of the recent LaurieWired video presenting a hypothetical of, "what if we stopped making CPUs": https://www.youtube.com/watch?v=L2OJFqs8bUk Spoiler, but the answer is basically that old hardware rules the day because it lasts longer and is more reliable of timespans of decades. DDR5 32GB is currently going for ~$330 on Amazon DDR4 32GB is currently going for ~$130 on Amazon DDR3 32GB is currently goin…
Earlier quoted context omitted.
I made a website: https://ontario-chips.vercel.app/
I suppose it would help if I could read the whole page: I cannot see the left few characters on Firefox on Android. What did you make this with? Note that fixing the site won't increase my chances of donating, I'm from the ASML country ;)
Earlier quoted context omitted.
Micron has said they're not scaling up production. Presumably they're afraid of being left holding the bag when the bubble does pop
Why are they building a foundry in Idaho? https://www.micron.com/us-expansion/id
>help ensure U.S. leadership in memory development and manufacturing, underpinning a national supply chain and R&D ecosystem.
It's more political than supply based