Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Putting out the hardware dumpster fire

dl.acm.org

Post reply on HN
Print view View original

91–100 of 115 posts

Re: Putting out the hardware dumpster fire

#91

Post by AnthonyMouse »

hedgehog wrote: post #90
AnthonyMouse wrote: post #87

Earlier quoted context omitted.

I'm trying to imagine why wear leveling would depend on the chips. The number of writes you can get out of them would, but the optimal wear leveling algorithm should be to spread the writes around as much as possible regardless of that, shouldn't it? Likewise, some chips could require more error correction because they expect more errors. But the amount of error correction is a tunable in the algorithm; it's a space…

Wear leveling: Just search Google Scholar for "wear leveling" to get an idea for how deep that rabbit hole goes. At minimum think about the difference between placing an infrequently written logical sector vs a frequently written one, and what the optimal strategy would be given physical blocks with varying write life left on them. Then how do you partition between regions you treat as SLC vs MLC? And on and on and o…

> At minimum think about the difference between placing an infrequently written logical sector vs a frequently written one, and what the optimal strategy would be given physical blocks with varying write life left on them.

That's not a simple answer but why would it depend on whose chips you have?

> What is the behavior vs temperature and what are you running the thing at?

This is where you might start seeing differences between manufacturers. But this is also optimization. If you don't have this information the chip should still meet a minimum spec and you can use a conservative value. If you do know it you can do something more efficient. And if this becomes popular the chip makers will start publishing what's necessary to do the optimization.

Re: Putting out the hardware dumpster fire

#92

Post by hedgehog »

AnthonyMouse wrote: post #91
hedgehog wrote: post #90

Earlier quoted context omitted.

Wear leveling: Just search Google Scholar for "wear leveling" to get an idea for how deep that rabbit hole goes. At minimum think about the difference between placing an infrequently written logical sector vs a frequently written one, and what the optimal strategy would be given physical blocks with varying write life left on them. Then how do you partition between regions you treat as SLC vs MLC? And on and on and o…

> At minimum think about the difference between placing an infrequently written logical sector vs a frequently written one, and what the optimal strategy would be given physical blocks with varying write life left on them. That's not a simple answer but why would it depend on whose chips you have? > What is the behavior vs temperature and what are you running the thing at? This is where you might start seeing differe…

> That's not a simple answer but why would it depend on whose chips you have?

How do you estimate how much life is left on any given part of the physical flash?

This is one of those areas where neglecting the optimizations qualitatively changes the result. It's like Twitter without scalability, sure you can build it in a weekend but what problem does that solve?

Don't get me wrong, I would like to see all of this stuff get better standardized, documented, and open source, but for the most part this is a cost-sensitive commodity and if you want to roll your own SSD qty 1 from bare chips it's just going to be orders of magnitude more expensive than letting SanDisk or whoever do it. Maybe it will be possible to get the hyperscalers and cloud operators to push for the kind of standardization you describe, it would benefit them.

Re: Putting out the hardware dumpster fire

#93

Post by Qwertious »

CrazyStat wrote: post #31
jrockway wrote: post #28

Earlier quoted context omitted.

If I'm ever going to get on the conspiracy theory bandwagon, it will be on two topics: 1) Return to office. 2) Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. Manufacturers say "no, we have a super secret secret sauce that nobody else could possibly improve upon". Uh huh, sure.

> 1) Return to office. In the spirit of unbridled conspiracy theory (i.e. I have no evidence for this and don't believe it is generally true): Landlords are paying CxOs kickbacks to push return to office in order to prop up commercial real estate.

I think that's basically true but it's not a conspiracy - take Apple, for instance. They have a $5 billion HQ on their balance sheet. If everyone works from home, then their investors will ask the CEO "why aren't you using this asset that costs $5 billion? It's inefficient - sell it off."

The problem here is that if the CEO sells off the HQ (in a market where everyone has a bunch of unused real estate) they'll almost certainly take a loss (and potentially a loss of billions of dollars, if they e.g. can only sell it for $3B), whereas if they keep it then it's technically valued at what it originally cost to build. The CEO's job is to make the company (look) profitable, so the CEO wants to find any excuse to use the HQ, even if it loses a bit of money or doesn't otherwise make much sense.

For companies that lease, it's the same problem or even worse - corporate office leases tend to be e.g. 3 years, and they're locked into paying actual dollars every quarter until the lease term ends, so if they can't claim to be using the office space then they're directly losing cash. Renegotiating the lease is a bad bet as the landlords have basically zero incentive to do so in the current environment. Thus, they want to appear to use their office space.

There are some other niche benefits too - forcing return to office is a great way to reduce employee count without needing to claim you're downsizing (although if you want to downsize then you can totally make the claim), and it makes office perks more valuable.

There are also some tax breaks from local governments that are contingent on having X number of workers in the office, and WFH could potentially require reversing the tax break.

None of these are "conspiracies", they're just sparkling incentives.

Re: Putting out the hardware dumpster fire

#94

Post by a1369209993 »

mikewarot wrote: post #24
ForOldHack wrote: post #18

Earlier quoted context omitted.

"Dumpster fire?" You are too kind. It's a cesspool, and a leaky one a that. Mostly Microsoft fault, and it has been now known to be getting worse.

>Mostly Microsoft fault, It's the lack of capability based security, and a common tragic misunderstanding of what it is, that is the root cause of this dumpster fire. This effects all operating systems in common use , including all versions of Linux, Mac OS, Windows, etc. Imagine if your only choice to buy an ice cream cone was to give the other person full access to your checking account, forever . That's what phone…

> all you have to do to port code over is change the calls to file selection dialogs (which return a list of file names) to "powerbox" calls (which return handles),

Actually you don't even necessarily need that. Assuming you have operating-system-provided file selection dialogs, you can just return "/app/d41d8cd98f00b204e9800998ecf8427e/spool/whatever-the-fuck-it-is.ext", and only allow the app to read and write that directory. Although that's a bit more balky in general, it can handle most use cases without even needing to modify the offending program.

Re: Putting out the hardware dumpster fire

#95

Post by srcreigh »

hoosieree wrote: post #50
srcreigh wrote: post #47

Earlier quoted context omitted.

In my city office buildings are being converted to residential housing. 3 buildings recently in an area of population size 250k.

What part of the world? I've heard that the cost of converting office spaces to residential is more than building new.

Waterloo Canada

Re: Putting out the hardware dumpster fire

#96

Post by AnthonyMouse »

hedgehog wrote: post #92
AnthonyMouse wrote: post #91

Earlier quoted context omitted.

> At minimum think about the difference between placing an infrequently written logical sector vs a frequently written one, and what the optimal strategy would be given physical blocks with varying write life left on them. That's not a simple answer but why would it depend on whose chips you have? > What is the behavior vs temperature and what are you running the thing at? This is where you might start seeing differe…

> That's not a simple answer but why would it depend on whose chips you have? How do you estimate how much life is left on any given part of the physical flash? This is one of those areas where neglecting the optimizations qualitatively changes the result. It's like Twitter without scalability, sure you can build it in a weekend but what problem does that solve? Don't get me wrong, I would like to see all of this stu…

> How do you estimate how much life is left on any given part of the physical flash?

In proportion to how many times it's been rewritten already, or by checking the error rate for what's already written there.

> This is one of those areas where neglecting the optimizations qualitatively changes the result.

It mostly changes how much error correction you have to use. But the nature of it also allows other optimizations.

Suppose you're going to use multi-drive array. Now you could stripe the error correction across devices and use the same system to recover from device failures, which at the same level of total error correction is more resilient.

Re: Putting out the hardware dumpster fire

#97

Post by mkj »

hlandau wrote: post #52
gchadwick wrote: post #36

Earlier quoted context omitted.

We're aiming to push things in the other direction with OpenTitan: https://github.com/lowRISC/opentitan/ It's an Open Silicon root of trust, all RTL (the actual hardware design in SystemVerilog), firmware, documentation and verification environment is open source and in the repository I just linked. We're closing in on our first discrete chip (details here https://opensource.googleblog.com/2023/06/opentitan-rtl-free.…

Can you explain how the root of trust is configured? Is it efuses or some sort of onboard mutable nonvolatile storage? If I buy a system with one of these chips in, am I likely to be able to set my own root of trust, or will the OEM have irreversibly set the root of trust to their own key?

It looks like the chips themselves should support change of ownership. Whether an OEM ships them in an unlocked (transferrable) state is up to them.

https://opentitan.org/book/doc/security/specs/ownership_tran...

Re: Putting out the hardware dumpster fire

#98

Post by adrian_b »

jpleger wrote: post #14

I think one of the primary reasons that it is such a dumpster fire is there traditionally hasn't been an "open" ecosystem in the hardware world, though now they are being forced towards that direction kicking and screaming. Every part of the hardware ecosystem has traditionally been done in closed, NDA ridden environments and only over the last 5-10 years has that even started to change. Designing chips has required…

Traditionally there hasn't been an "open" ecosystem in the hardware world only if the "traditions" are understood to be no older than three decades ago.

The IBM Personal Computer has been amazingly open and this has been tremendously important for the evolution of the computer industry, by creating de facto hardware standards.

Unfortunately many managers have been less impressed by the huge benefits for the entire society that this openness has caused, than by the fact that later IBM has failed to exploit as well as their competitors the open standards created by themselves, for increasing their profits.

Then, slowly after 1990 and more and more after 2000, the ugly fashion of secret documentation and NDAs designed to prevent competition on the markets, with the hope of enabling higher product prices, has spread everywhere.

It is impossible to estimate whether this secrecy has ever been profitable for the companies that practice it, because while it has prevented the apparition of competitors that would have lowered the prices, it has also limited the sizes of the markets in which their products are sold by limiting their customer list mostly to the already existing, because many of the potential customers cannot evaluate whether a product is suitable for their needs.

In order to accept the harassment of an NDA, you must be already convinced that you need that product. Before this annoying NDA practice, it was normal to evaluate a much greater number of products for any new project and it was much more frequent to decide for new suppliers.

The NDAs may be beneficial for those who strongly dominate a market, so they do not hope to grow, but only to retain their captive customers, but they prevent the growth of the smaller companies. Nevertheless, most small companies appear to stupidly imitate the behavior of the big ones and they are equally secretive about their products, which does not have any rational justification.

Re: Putting out the hardware dumpster fire

#99

Post by adrian_b »

jrockway wrote: post #27
kevin_thibedeau wrote: post #17

Earlier quoted context omitted.

Most electrical components have traditionally had open datasheets. It is only a relatively recent phenomenon that mass market ICs are locked behind NDAs.

It really depends on the complexity. Sure, 555 timers and even microcontrollers have datasheets. CPUs that run Linux, Wifi chips supporting recent standards, etc.? Rare. This is why Linux in the 90s was a big pain; you get some random Ethernet chip and there is no documentation on how to talk to it, so you're just dead in the water. (I think Android was the turning point. From then on, stuff had to support Linux, bec…

Complex CPUs and peripherals up to around 1995, e.g. up to Pentium, had mostly open documentation, not much different than for a 555.

Only after that the public documentation has become progressively more and more restricted.

Some restrictions are quite recent, e.g. only since the first Zen AMD has stopped publishing the BIOS and Kernel Developer's Guides and Intel has stopped publishing even summary datasheets for the mobile CPUs only a few product generations ago (summary datasheets are still published for desktop CPUs).

Re: Putting out the hardware dumpster fire

#100

Post by [deleted] »

AnthonyMouse wrote: post #56
jrockway wrote: post #28

Earlier quoted context omitted.

If I'm ever going to get on the conspiracy theory bandwagon, it will be on two topics: 1) Return to office. 2) Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. Manufacturers say "no, we have a super secret secret sauce that nobody else could possibly improve upon". Uh huh, sure.

> Why we need a dedicated CPU and DRAM attached to flash memory. You can garbage collect and wear level in your OS if you want to. The thing I don't get is, the chips are all commodities, and it's not like soldering them to a board is rocket science. Why isn't one of the companies that makes fully-specified inexpensive RISC-V chips selling one attached to some commodity flash chips and an NVMe connector? Include some…

[deleted]
Post reply on HN