Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

anandtech.com

Post reply on HN
Print view View original

81–90 of 97 posts

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#81

Post by mrep »

cookiecaper wrote: post #79
mrep wrote: post #73

Earlier quoted context omitted.

Or if you are a team like mine that develops/manages 200 servers, 15 RDS database instances, 500 TB of compressed S3 data, and provides accessibility to all of that through API's with only 5 developers because of how easy AWS makes it for us. While our server costs are probably higher than going bare metal, how many developers would it take you to manage all of that on bare metal hosts?

The expense of one's outlay to Amazon is not a persuasive statistic. It only shows that you have a lot of incentive to keep confirmation bias in gear. AWS is specifically designed to maximize instance count (that is, cost). The reason "devops" has exploded since EC2 hit critical mass is that before EC2, people were reasonable with the number of servers they needed. With EC2, it's all water, and it's so easy to press…

Our EC2 costs are about the same as S3, RDS is the most as I forgot to mention the backup servers we have in each region but we are trying to move all of that data to S3, and more than our SQS costs.

I'm pretty new to the backend design cost/benefit (have only used AWS since I graduated college) but I'm curious to here about bare metal storage solutions for 500 TB of compressed json as I have not read any compelling solutions for that data requirement (Outside of MongoDB but I have heard good and bad things about that).

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#82

Post by agumonkey »

mtanski wrote: post #44
agumonkey wrote: post #26

Earlier quoted context omitted.

Optical discs are relegated to back shelves too. Hard to beat cheap rewritable nand flash. ps: I'd be curious about a tiny, 2017 laptop friendly optical disc storage. Say a 2.5" RW BD layer in cartridge. A modern mini disc.

There are people using optical media for long term storage. So I am not counting it out as a storage medium. In my classification I would put it in cold storage tier.

Surely. Although I'd have thought for long time storage magnetic tapes were cheaper.

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#83

Post by zxcmx »

saosebastiao wrote: post #5

Impressive. On a semi-related note, the fact that cloud providers offer managed Postgres databases is great, but things like this keep pushing me to think about bare metal in colo. A $20,000 server/backup combo with a couple of these will give me 5x the performance of a server that costs me $70k/year on AWS before provisioned IOPS . That's a huge gap to play with for taking care of your own maintenance costs.

Yep but the FTEs you need are 100k+ each, managing baremetal servers is almost certainly not your core business, and you're running the risk of gitlab style fat fingering dinking around with low level stuff that you did to save money.

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#84

Post by zxcmx »

chx wrote: post #35
bkeroack wrote: post #33

Earlier quoted context omitted.

With public cloud, you're paying for the control plane. Period. Yes the raw compute cost is much higher (and the performance often less) than bare metal, but software development is really, really expensive. With the public cloud you get the result of literally millions of programmer-hours "for free". To many that's worth it (at least at first, below a certain scale). This is also one reason why Kubernetes is really…

I really badly do not understand what EC2 gives you that a dedicated box doesn't. Yes, if you use all the other services, then there's a value but then again for example SQS is a particularly shitty queue.

It's not ec2, it's the ecosystem.

DDOS, multi-path, multi-az, dr, snapshots, hardware redundancy, elasticity, freedom from datacenter contracts, fiber contracts, driving to rack stuff, jammed fingers, staff costs, firmware bugs, bad hardware batches, cdn, compliance costs. I have more but I'll stop there.

It doesn't make sense at very small or large scales but it captures a hell of a lot of the middle.

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#85

Post by [deleted] »

chx wrote: post #35
bkeroack wrote: post #33

Earlier quoted context omitted.

With public cloud, you're paying for the control plane. Period. Yes the raw compute cost is much higher (and the performance often less) than bare metal, but software development is really, really expensive. With the public cloud you get the result of literally millions of programmer-hours "for free". To many that's worth it (at least at first, below a certain scale). This is also one reason why Kubernetes is really…

I really badly do not understand what EC2 gives you that a dedicated box doesn't. Yes, if you use all the other services, then there's a value but then again for example SQS is a particularly shitty queue.

[deleted]

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#86

Post by [deleted] »

chx wrote: post #35
bkeroack wrote: post #33

Earlier quoted context omitted.

With public cloud, you're paying for the control plane. Period. Yes the raw compute cost is much higher (and the performance often less) than bare metal, but software development is really, really expensive. With the public cloud you get the result of literally millions of programmer-hours "for free". To many that's worth it (at least at first, below a certain scale). This is also one reason why Kubernetes is really…

I really badly do not understand what EC2 gives you that a dedicated box doesn't. Yes, if you use all the other services, then there's a value but then again for example SQS is a particularly shitty queue.

[deleted]

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#87

Post by justinclift »

benlwalker wrote: post #55

I'm the technical lead for the Storage Performance Development Kit ( http://spdk.io ). I have one of these in my development system and we're hoping to post benchmarks fairly soon. SPDK further reduces the latency at QD 1 by several microseconds. It's an impressive device!

Any word on real world endurance?

There's some speculation these devices are massively overprovisioned due to the er... cells (or equivalent) wearing out much faster than early hype/info/etc.

So, it'd be interesting to get a real world idea if these things are likely to explode badly ;) 6 months after purchase or not (etc). :)

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#88

Post by cookiecaper »

mrep wrote: post #81
cookiecaper wrote: post #79

Earlier quoted context omitted.

The expense of one's outlay to Amazon is not a persuasive statistic. It only shows that you have a lot of incentive to keep confirmation bias in gear. AWS is specifically designed to maximize instance count (that is, cost). The reason "devops" has exploded since EC2 hit critical mass is that before EC2, people were reasonable with the number of servers they needed. With EC2, it's all water, and it's so easy to press…

Our EC2 costs are about the same as S3, RDS is the most as I forgot to mention the backup servers we have in each region but we are trying to move all of that data to S3, and more than our SQS costs. I'm pretty new to the backend design cost/benefit (have only used AWS since I graduated college) but I'm curious to here about bare metal storage solutions for 500 TB of compressed json as I have not read any compelling…

In 2017, 500TB is easily within the realm of colocated bare metal.

There are many build it yourself options, which would involve things similar to this SuperMicro JBOD [0] + some other servers/external RAID controller to handle distributions over the disks. You could also go really barebones and do a couple of home-built "RAIDZilla"-style devices. [1]

For the cost of one month's S3 storage, you can buy a pre-built Backblaze Pod from a third-party supplier [2]. I've found this to be the case with Amazon; the monthly cost is about 50%-100% of the permanent cost for the actual hardware (which will usually last at least 3-5 years). Even if you have to hire a couple of your own hardware jockeys, you're going to be saving 6x.

For a less DIY route, any SAN provider will be able to accommodate 1PB (for redundancy) without breaking a sweat. Of course, this will be a large upfront expenditure, but it should still easily be cheaper than S3 over the long run. There are tons of options and storage engineering is a big field. Look around and I'm sure you'll find something acceptable.

-----

All this said, I really have to be skeptical that you need 500TB of (compressed!) JSON data. I would look into how much of that data you really need to keep, set up some retention policies, and seriously consider reworking your storage format to make these numbers more reasonable (JSON is obviously not space-efficient), which will not only greatly reduce infrastructure costs but also make the project much easier to handle.

You may also wish to look into modern compression codecs if you haven't already. LZMA provides the best ratio but the compression cycle is slow (decompression is fast). Brotli and zstd are new compression options that are at least comparable to gzip in ratio and much faster. Data deduplication should also help.

[0] https://www.cdw.com/shop/products/Supermicro-SC417-BE1C-R1K2...

[1] https://www.glaver.org/raidzilla25/

[2] https://www.backblaze.com/blog/open-source-data-storage-serv... (estimates cost of third-party pod at $12,849.40; S3 cost calculator says 500TB of storage in us-east-1 is $12,407.17 without any bandwidth, request costs)

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#89

Post by chx »

zxcmx wrote: post #84
chx wrote: post #35

Earlier quoted context omitted.

I really badly do not understand what EC2 gives you that a dedicated box doesn't. Yes, if you use all the other services, then there's a value but then again for example SQS is a particularly shitty queue.

It's not ec2, it's the ecosystem. DDOS, multi-path, multi-az, dr, snapshots, hardware redundancy, elasticity, freedom from datacenter contracts, fiber contracts, driving to rack stuff, jammed fingers, staff costs, firmware bugs, bad hardware batches, cdn, compliance costs. I have more but I'll stop there. It doesn't make sense at very small or large scales but it captures a hell of a lot of the middle.

Most of what you described is colo and not dedicated boxes...

Re: The Intel Optane SSD DC P4800X (375GB) Review: Testing 3D XPoint Performance

#90

Post by Toast_ »

toast0 wrote: post #67
Toast_ wrote: post #10

Earlier quoted context omitted.

>where you just don't have the ability to run hardware (increasingly the case) Softlayer has 'on demand' bare metal plans. I'm not sure how cost effective it would be compared compared to running your own box, but the simplicity is nice and they had me up and running in under 2 hours (assembled with customizations). http://www.softlayer.com/bare-metal-servers

How custom can you get with the hourly machines? The order page (without logging in) looks like you have to pick a ram size (and they don't go beyond 256GB), and you can only go up to 4 disks. That said, I know the order page for monthly has maximum ram sizes listed below some of the monthly machines I manage, so maybe the hourly page is a simplified version of the truth too.

I'm not really sure about hourly plans. When I was configuring my machine (~1-2 years ago) I was able to select hourly or monthly billing (hourly would've been outrageously expensive).
Post reply on HN