Live data from Hacker News

Testing disks: Lessons from our odyssey selecting replacement SSDs

bbc.co.uk

61–70 of 78 posts

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#61

Earlier quoted context omitted.

The BBC has a very strong product prominence policy[0] (i.e. avoid naming brands when possible), being government funded is a large driver of this policy. [0] http://www.bbc.co.uk/editorialguidelines/guidelines/editoria... EDIT: fixed policy name and added link

It's a shame this is so selectively applied. See, for example: - The constant mention of speaking to people 'over Skype' on the News - Publicization of Twitter hashtags on Questiontime and other programs - Hours worth of Top Gear footage (and the entire Arctic Special) that were effectively Toyota Hilux advertisements

Many, many hours worth of prime time TV is advertising for movies/books/etc.. "So you're over here promoting your new movie" says eg Mr Norton, "Yeah, I'm getting US$20M for this movie so I appreciate the free advertising here ..." (or some bs the publicist wrote for them) says the guest, "cue trailer".

When I was on the BBC I had to change my logo polo-shirt so as not to advertise my single-location SME business. Yet Nike, et al., logos are fine, as is advertising by sports teams, etc..

It's all very inconsistent.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#62
post #7

Earlier quoted context omitted.

It's not Government funded. There's a mandatory TV tax (for TV viewers), the fee of which is set by Government.

"Taxes" are money paid to the government for use in government funded activities.

So, you agree it is taxation?

(Genuinely not clear to me, not trying to be snarky.)

FWIW when I last researched the issue (maybe 3 years ago) there was also funding from direct taxation used for BBC activities; it was in the published accounts at least.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#63
post #52

Earlier quoted context omitted.

BBC published a technical (but really PR) article written by amateurs posing as pros, instead of consulting reputable experts?

To be clear, this isn't a news article written by our journalists - it is a piece written by the team themselves that we felt may be of interest to others, and that might help us do things better in the future. While I enjoyed reading it, I can assure you that SSD performance testing doesn't move the BBC PR needle compared to the identity of the new Doctor Who. We're acutely aware that we've still got much to learn i…

Do you accept American Engineers? /s

I found the piece to be wonderful. I don't do large scale storage work, so I'm very un-knowledgeable in the area, but it's great to see someone else's struggles other than Amazon or a backup service. And it is yet another indicator that the BBC cares about quality content instead of just pushing up some stock price.

Thanks for you write up

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#64
post #60

Earlier quoted context omitted.

> In our experience SSD write performance goes to sh t if you don't regularly TRIM them. Interesting, is that because of the load? It seemed "modern" SSDs have GCs good enough that trim isn't quite necessary anymore to ensure good performances in consumer loads. > RAID cards not passing TRIM is a big problem for us too... Are there NVMe RAID cards? I assume they'd necessarily pass the command along considering deallo…

> Interesting, is that because of the load? It seemed "modern" SSDs have GCs good enough that trim isn't quite necessary anymore to ensure good performances in consumer loads. A drive has no way to tell whether filesystem is using a given block or not. TRIM is a way for the filesystem to tell it that. So I would imagine the GC that you're referring to is working on the blocks marked with TRIM. BTW, besides running fs…

> So I would imagine the GC that you're referring to is working on the blocks marked with TRIM.

Not necessarily. Since flash doesn't support in-place modification of data, any change to a portion of a file (or other FS data structure) that writes less than a contiguous 16MB (depending on the flash) will create a need for GC on the drive with or without TRIM. You can put a drive into a state of needing to do a lot of GC even without changing the quantity of live data.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#65

Earlier quoted context omitted.

> I noticed they didn't mention any brands by name though, why is that? As a public service broadcaster in the UK, the BBC must be very careful about naming specific brands and products due to rules laid out in their legal remit (and fear of legal action if from a party that feels unfairly disadvantaged by a competitor getting a good mention or them getting a bad one). This is way Blue Peter always use "sticky backed…

> like Google -> to google Hang on! Surely, "to google" refers to performing a search specifically with Google, right?

Not a chance. To google means "to search online [using a search engine]".

Have you never witnessed someone googling something using Bing?

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#66
post #31
post #11

Earlier quoted context omitted.

I have barely any knowledge of hardware, but I have built plenty of pieces of software that have helped people.

Civil Engineers don't understand chemistry or quantum mechanics - how they build a bridge.

They must know some chemistry surely, like weathering effects on concrete; effects of potential chemical spills, eg on roadways, metal-concrete-surfacing interactions.

?

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#68
This reminds me of testing I did years ago on ... CD-ROMs. Funny how lessons from old technology can apply to new technology.

Around 15 years ago my company did a Linux distribution on CDs: KRUD. It was updated monthly, and we had something like 400 subscribers. For various reasons we burned these CDs in house on a cluster I built.

We would burn, eject, read and checksum, and if the read test succeeded we would ship it out. We found some users with some discs had problems reading them. We contacted these users and paid them to return the CDs and did further testing on them.

Our initial test was using dd, and we found that the discs that were not obviously damaged in shipping, would tend to pass tests on some of our CD-ROM drives, but fail on others. But when they did succeed, they would tend to take longer than normal.

I wrote a new test program that instead of using dd directly used SCSI read commands, and timed every one. It would then count the number of reads that were "slow" (like 2x normal) and those that were "really slow" (like 5x), and if these got over a certain threshold we would throw away the disc.

Being able to time the raw operations was incredibly useful, and seems like it could have shown the authors of this paper problems before being deployed to production.

Except, they didn't really seem to do very thorough testing of the drives. Running stress testing on a 1TB drive for an hour seems pretty short.

Also in my above job we did hosting. We found that if we burned in disks by reading/writing to them 10 times ("badblocks -svw -p 10"), we would almost never experience drive failures on the Hitachi drives we were using. If we didn't do this, the drives would have a fairly high chance of falling out of the RAID array in production.

As drive sizes increased from 20GB to 200GB to 1TB, these tests started taking weeks to complete. But, they were totally worth it.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#69
You could give blkreplay a go next time you decide on which disks to buy. I find the additional effort is worth it, but ymmw. Use one of the shipped loads for a quick test, but you really want to run blktrace against your current setup and feed that data to blkreplay.

Re: Testing disks: Lessons from our odyssey selecting replacement SSDs

#70
post #52

Earlier quoted context omitted.

BBC published a technical (but really PR) article written by amateurs posing as pros, instead of consulting reputable experts?

To be clear, this isn't a news article written by our journalists - it is a piece written by the team themselves that we felt may be of interest to others, and that might help us do things better in the future. While I enjoyed reading it, I can assure you that SSD performance testing doesn't move the BBC PR needle compared to the identity of the new Doctor Who. We're acutely aware that we've still got much to learn i…

Don't let the gratuitous negativity get to you! I may only see a very small slice of what the BBC does, what with being one of those continental imperialist and all. But what does find its way to me has always seemed to be excellent.

(Not a job application. Unless you know something about the EU/British future that I don't.)

Post reply on HN