Live data from Hacker News

The Next Backblaze Storage Pod

backblaze.com

81–90 of 118 posts

Re: The Next Backblaze Storage Pod

#81

Earlier quoted context omitted.

You'd think they could build a ARM-powered, credit card sized controller for them with a disk breakout card and network IO. PC motherboard and full-sized cards seem like overkill.

They're running a fair bit of math (probably Reed Solomon matrix multiplications for error correction) over all the data accesses. Given the bandwidth of 60+ hard drives (150MB/s per hard drive x 60 == 9GB/s in/out), I'm pretty sure you need a decent CPU just to handle the PCIe traffic. At least PCIe 3.0 x16, just for the hard drives. And then another x16 for network connections (multiple PHY for Fiber in/out that ca…

That's not really how it's done.

RS is normally used as erasure code: It's used when writing (to compute code blocks), and when reading _only when data is missing_. Checksums are used to detect corrupt data, which is then treated as missing and RS used to reconstruct it. Using RS to detect/correct corrupt data is very inefficient.

Checksums are also normally free (CRC + memcpy on most modern CPUs runs in the same time that memcpy does: it's entirely memory bound).

The generation of code blocks is also fairly cheap: Certainly no large matrix multiplications! This is because the erasure code generally only spans a small number of blocks (e.g. 10 data blocks), so every code byte is only dependent on 10 data bytes. The math for this is reasonably simple, and further simplified with some reasonable sized look-up tables.

That's not to say that there is no CPU needed, but it's really not all that much, certainly nothing that needs acceleration support.

Re: The Next Backblaze Storage Pod

#82
post #75

Any idea what Dell is actually selling them? The DVR's we buy (Avigilon) are white Dell 7x0's with a custom white bezel, but those only fit 18 3.5" drives.

The author recently commented: https://www.backblaze.com/blog/next-backblaze-storage-pod/#c... It's apparently the "Dell PowerEdge R740xd2 rack server".

https://ifworlddesignguide.com/entry/281015-poweredge-r740xd... super interesting design.

Re: The Next Backblaze Storage Pod

#83

> That’s a trivial number of parts and vendors for a hardware company, but stating the obvious, Backblaze is a software company. Stating the obvious: Backblaze wants investors to value them like a SaaS company. This blog post suggests they’re more of a logistics and product company— huge capex and depreciating assets on hand. As a customer, I like their product, but they’re no Dropbox. If they would allow personal NA…

I sync my NAS to Backblaze B2 without any issues, and pricing is great.

Yeah but I don't want to pay B2 prices. I don't have a PB, but I have more than I'd really like to hang off one machine.

Re: The Next Backblaze Storage Pod

#84
post #69

Earlier quoted context omitted.

Definitely limited by the 1gbps or even 10gbs network connection

Any pod like this would normally have at least 1x40gbps uplink minimum? Like most blade setups, like (random example): https://www.storagereview.com/review/supermicro-x11-microbla...

I don't think that makes sense for a backup application.

Re: The Next Backblaze Storage Pod

#85

Did the surge in Chia mining affect global hard drive prices at all?

Doesn't look like it, at least in the EU. You can check for yourself here[0]. Just find a drive such as this one[1] and check the chart on the upper-right corner for price history.

[0] https://geizhals.eu/?cat=hde7s&xf=1080_SATA+1.5Gb%2Fs~1080_S...

[1] https://geizhals.eu/toshiba-enterprise-capacity-mg08aca-16tb...

Re: The Next Backblaze Storage Pod

#86

Did the surge in Chia mining affect global hard drive prices at all?

In general, for larger (> 10TB) drives, most definitely. mostly unaffected.

The direct impact on larger drives is entirely dependent on brand (Toshiba's Enterprise drives appear to be less in demand than Seagate's Exos, for instance), recording process (CMR versus SMR), and, to a lesser extent, power consumption. In virtually all instances though, the price has jumped significantly [0]. 16TB Exos more than doubled, for instance [1]. In large parts of Europe, large drives have been back-ordered since the beginning of April; my orders are scheduled for delivery in August, yet I would be surprised if I saw anything before September.

[0] https://www.tomshardware.com/news/analysis-hdd-prices-skyroc...

[1] https://camelcamelcamel.com/product/B07SPFPKF4

Re: The Next Backblaze Storage Pod

#87
May just be me, but I feel like it’s probably a bad idea to outsource one of your core competencies. Backblaze is quite literally built on their storage pods.

It’s like outsourcing the EU telecommunications, which we’ve seen a bunch of articles about recently.

Re: The Next Backblaze Storage Pod

#88

> That’s a trivial number of parts and vendors for a hardware company, but stating the obvious, Backblaze is a software company. Stating the obvious: Backblaze wants investors to value them like a SaaS company. This blog post suggests they’re more of a logistics and product company— huge capex and depreciating assets on hand. As a customer, I like their product, but they’re no Dropbox. If they would allow personal NA…

It's easy to buy a bunch of hard drives and connect in a data center. Managing petabytes per user for thousands of users is the hard part, and it's a software problem. BackBlaze is definitely a SaaS company... though the quality of their offering certainly lags behind Dropbox, both in terms of feature set and user experience. They're also in a very competitive industry. Storage/backup is basically a commodity nowaday…

> BackBlaze is definitely a SaaS company

agreed, but I think s3 is the more realistic comparison than dropbox

Re: The Next Backblaze Storage Pod

#89

I have to imagine that by making Storage Pods 1.0 through 6.0, maybe they "encouraged" Dell (and other manufacturers) that this particular 60+ hard drive server was a good idea. And now that multiple "storage pod-like" systems exist in the marketplace (not just Dell, but also Supermicro) selling 60-bay or 90-bay 3.5" Hard drive storage servers in 4U rack form factors, there's not much reason to build their own? At le…

You'd think they could build a ARM-powered, credit card sized controller for them with a disk breakout card and network IO. PC motherboard and full-sized cards seem like overkill.

With 60+ hard drives in a single enclosure, you want all of the processing power, PCIe lanes, and bandwidth you can get.

There are a small number of high end ARM server boards that could do it, but you’re not saving much money at that point. Might be more expensive due to lack of scale.

Amortize a server-grade CPU and motherboard across 60+ high capacity drives and it’s not really worth pinching pennies at the risk of lower performance.

Re: The Next Backblaze Storage Pod

#90
post #82
post #75

Earlier quoted context omitted.

The author recently commented: https://www.backblaze.com/blog/next-backblaze-storage-pod/#c... It's apparently the "Dell PowerEdge R740xd2 rack server".

https://ifworlddesignguide.com/entry/281015-poweredge-r740xd... super interesting design.

That is really interesting ...

We deploy our storage with a 2U "head unit"[1] that has the actual computer and all of the SSDs for boot and SLOG and L2ARC and "zfs special"[2] devices. Then we attach 60-drive JBODs externally to those "head units".

With this dual-front design you could have 24 2.5 drives in the first row and 12 3.5 drives in the second row and that would really be helpful when we sometimes need to quickly spin up an adjacent zpool without attaching another whole JBOD.

Sadly, I do not see any configurations with 2.5" drives on the Dell config page:

https://www.dell.com/en-us/work/shop/dell-poweredge-servers/...

If one row was 24x 2.5 drives and the other was 12x 3.5 drives I think we would start deploying these immediately ...

EDIT: Huh ... even the two, rear, boot drives are 3.5" which is sort of weird and gratuitous ...

[1] https://www.supermicro.com/en/products/chassis/2U/216/SC216B...

[2] Yes, it's actually called that ...

Post reply on HN