Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

ECC and DDR5

etbe.coker.com.au

Post reply on HN
Print view View original

111–120 of 162 posts

Re: ECC and DDR5

#111

Post by etbe »

tliltocatl wrote: post #85
dsr_ wrote: post #50

Earlier quoted context omitted.

The primary source of uncorrelated bit-flips is, effectively, cosmic radiation. (Correlated bit-flips are likely from a manufacturing error... or, at least in one case, from excessively radioactive ceramic packaging.) The more atmosphere you have to randomly absorb high-energy photons, the better. Higher density memory drops fewer electrons in each well, which means a lower-energy photon can change the state. Corolar…

> Corolary: submarine datacenters are much better than orbital datacenters Yea, except datacenters also need connectivity. And submarine connectivity, while not particularly expensive, is quite vulnerable - even more so than on land.

People outside the US have a significant lack of Internet functionality when there are problems with submarine cables, but it hardly ever fails. Apart from when certain countries sabotage things.

If you only need 100M deep water then you don't need to go far offshore, there are places where it's only about 1KM out. 12M of water is used for storing spent nuclear fuel assemblies and less than half of that is needed for protection so 5M should be adequate for stopping cosmic rays and that depth is common in harbours for recreational boating, such harbours shouldn't have issues of cable breakage.

But the real issue for comparing submarine and orbital computers is cooling. Space is cold but doesn't allow easy transmission of heat.

Re: ECC and DDR5

#112

Post by etbe »

loeg wrote: post #53
inigyou wrote: post #22

Earlier quoted context omitted.

My Linux installation came with an AMD MCE driver that reports them to dmesg. It logs a few corrected errors a day.

Check if those are all coming from DIMMs, as opposed to other subsystems (on-die caches).

https://www.theregister.com/on-prem/2021/06/04/fyi-todays-co...

Google and Facebook report a few unreliable CPU cores out of thousands of systems. Meaning maybe a 1/10,000 rate of repeatable CPU errors.

https://www.cs.toronto.edu/~bianca/papers/sigmetrics09.pdf

Google reports 8% of DIMMs being affected by errors. While the differences in metrics prevent comparing the two things directly it seems that DRAM errors are more common than CPU errors.

Re: ECC and DDR5

#113

Post by etbe »

hangonhn wrote: post #101

"I have used more than a few systems with ECC UDIMMs and I recommend avoiding them if convenient as ECC UDIMMs are expensive on the second hand market while ECC RDIMMs can get very cheap." Boy did I learn that the hard way. I found an ASRock server motherboard that fit my needs for a NAS perfectly. I just assumed all ECC memory were registered so I didn't even bother to check. I had already put everything together mi…

A very large portion of people who go from working on consumer grade systems to getting enterprise grade stuff for their home lab make a mistake like that, I did some similar things, my biggest mistake was spending $300 on non-refundable ECC RDIMMs without realising that my server (the cheapest Dell Txxx server) took ECC UDIMMs.

I hope that my blog posts on these topics will save some people some money.

Re: ECC and DDR5

#114

Post by etbe »

wpm wrote: post #52
Havoc wrote: post #5

Got ECC udimm for my ddr4 server and was surprised to see it picking up errors occasionally (once every couple months). Likely from a weakness in one of the sticks. This far it’s always corrected it though so opted to keep them anyway (nobody wants to be minus 32gb in these trying memory times) With normal sticks I’d not have know that there is a potential issue.

Depending on your server's CPU sockets, reseating the CPU might help if they were ever swapped before (esp if you got the server used). I have a marginally bad RAM slot on one of my Supermicro X11 boards because my dumb ass dropped the CPU into the socket and bent a few pins. I bent them back, but you can only do so much and the furthest RAM slot won't work without throwing errors daily if I run it at 2400MHz.

I have a HP Z640 that only has 3/4 DIMM slots working. Reseating the CPU didn't help and the same CPU works perfectly in another Z640. I never found out the cause. It was sold as "unknown condition for parts" so $50 for a server that works perfectly apart from one DIMM slot is still a good deal.

Re: ECC and DDR5

#115

Post by etbe »

toast0 wrote: post #39
zargon wrote: post #18

Earlier quoted context omitted.

This is the attitude that needs to change. If RAM is 25% of the system cost (say for a home server in your example), then the cost for ECC is 3% of the system cost. There is no world where accepting data corruption in order to save 3% cost is a reasonable tradeoff. ECC should be standard, period.

Let me kmow where I can get ECC RAM that's only 12% more than the price of non-ECC and I'll strongly consider it the next time I do a build. (Otoh, prices are high, so I'm not building anything for a while unless stuff breaks)

For second hand gear RDIMMs are significantly cheaper than UDIMMs. As all RDIMMs have ECC that means generally ECC is cheaper for second hand systems.

Aliexpress had $50 single socket motherboards for Xeon E5-26xx CPUs and $150 dual socket motherboards last time I checked that use DDR4 RDIMMs.

Before the prices went up due to AI foolishness DDR4 RDIMMs were $1.20AU/G means means something like $US0.80/G.

Re: ECC and DDR5

#116

Post by etbe »

snvzz wrote: post #49

I run a 9800x3d with 96GB ECC (2x 48GB DDR5-5800). It was a pain to select a compatible board and source the RAM but glad I did, in June 2024... the RAM prices went up immediately after. non-ECC should be a kernel taint in Linux. Newer versions of Windows (12?) should also refuse to boot without ECC. It should be mandatory in DDR6. The insanity must end.

Yes a kernel taint is a really good idea!

Re: ECC and DDR5

#117

Post by etbe »

monster_truck wrote: post #55

This is not a very good post. Claiming that every issue you've experienced is from seating errors? Come on man, at least deliberately reproduce that scenario once. I've tortured motherboards for fun and never managed that on purpose. We did pry every cap off of an old K6 board while it was on and... it just kept running. It didn't come back after we turned it off The amount of time it takes to reseat memory, even if…

I can only report what I have seen. Feel free to write your own blog post reporting what you have seen.

Re: ECC and DDR5

#118

Post by pseudalopex »

Wowfunhappy wrote: post #21
zargon wrote: post #18

Earlier quoted context omitted.

This is the attitude that needs to change. If RAM is 25% of the system cost (say for a home server in your example), then the cost for ECC is 3% of the system cost. There is no world where accepting data corruption in order to save 3% cost is a reasonable tradeoff. ECC should be standard, period.

When I last looked at this (a couple of years ago), it wasn't just the cost difference—the ECC memory available to purchase was also slower than non-ECC offerings, or at least cost significantly more if I actually wanted the same speed. At least for me, going ECC simply didn't make sense. Caveat that I was building a gaming PC at that time, I might have gone differently for a home server. And, this was before the mem…

Intel and AMD segmented the market. Workstations and servers use registered memory. Desktops use unregistered memory. Most desktop motherboards or CPUs do not support ECC. And most people who want ECC want reliable more than fast. So ECC unregistered above standard speed memory is made and priced for a niche. But this would change if ECC was standard.

Re: ECC and DDR5

#119

Post by HDBaseT »

bartleeanderson wrote: post #109
ck2 wrote: post #7

16GB ECC DDR5 was $50 early 2025 it is now $500 when the AI/datacenter bubble finally bursts we are going to have some amazing hardware deals

I had a conversation with chatgpt about that very thing, whether it was a bubble or not and how the big players could absorb the cost and where it would all land and the interesting thing was if China gets its foothold and grabs enough share because the prices are too high even being subsidized, that could cause real trouble. But the point was its not a bubble in the conventional sense. Have your own conversation abo…

I'd rather just have the conversation here. I don't need some hallucinated random seed response to tell me what China can and cannot do.

Of course an AI is going to tell you AI isn't a bubble.

Re: ECC and DDR5

#120

Post by etbe »

procaryote wrote: post #9
graemep wrote: post #6

Earlier quoted context omitted.

> ECC RAM is not needed for every application and mandating it for every gaming rig and machine used to run Chrome will just increase prices unnecessarily. The price increase will be very small if manufacturers are no longer able to segment the market to make a higher margin on those who need it. No one wants even games or web browsers to crash randomly. > If they want to eat that cost, it should be up to them. They…

Without quantifying the cost to society that someone's web-browser crashes sometimes, it's hard to make a rational case that this issue is so dire that it needs specific regulation. I'm not sure making it illegal to sell consumer grade hardware is the boon for the people you imagine

No-one ever proposed making non-ECC RAM illegal, that's a straw-man.

Restricting risky items is standard practice for all governments. All the "use by" dates on food etc are from government regulation. The current experiments with "raw milk" shows that you can get away with flouting food safety regulations and not have immediate problems.

Post reply on HN