Live data from Hacker News

Toyota blames factory shutdown in Japan on ‘insufficient disk space’

theguardian.com

151–160 of 223 posts

Re: Toyota blames factory shutdown in Japan on ‘insufficient disk space’

#151
post #52

Earlier quoted context omitted.

This is such a uninformative comment. Business/Enterprise storage is far, far more expensive than a drive you pick up at Best Buy or Fry's.

I'd bet the salary of the guy who is making this a pain could buy enough storage that they don't have to worry about it anymore.

Depends on the org. Some orgs are well managed, others don't want to pay for stuff behind the scenes until it bites them in the ass. They don't know what they don't know.

Re: Toyota blames factory shutdown in Japan on ‘insufficient disk space’

#152
post #13

I was responsible for some dev ops stuff at a state's health department and one of the more infuriating things about working at that place was that it was like pulling teeth getting more storage allocated. Our backups would be running out of disk and they'd allocate me 50 or 100 GB at a time. I'm sure someone Toyototian was yelling that this was going to happen for the past 6 months.

I can say the same thing but as a sysadmin working at a place where it was pulling teeth to get money to buy additional storage. Upper management was so daft/cheap and they can't see past six months so they decide to buy the cheapest enterprise solution with the least amount of upgrade capacity.

My experience as an admin was quite similar. Developers just couldn't seem to grasp the need for someone to foot the bill for their constant requests for 2 TB LUNs. I'm just glad I'm done dealing with that shit.

Re: Toyota blames factory shutdown in Japan on ‘insufficient disk space’

#153
post #6

Earlier quoted context omitted.

To spell it out, I think the joke here is the assertion that they are probably using old operating system and software. The screenshot is for windows XP.

Why is that a joke? Factory equipment is ordered and has a lifetime of 20 years. Of course it has old systems. These are grownup industries. They have no time for development hipster nonsense and extraction games for unwanted features to stave of unemployment in some glass-towers.

If that is so, factories should maybe not buy software that has a much shorter lifespan than their 20 years. In essence, they are the ones making the mistake, not the OS vendor. Heck, they should probably fork an existing OS and maintain it themselves if this lifetime requirement is such an important thing.

Re: Toyota blames factory shutdown in Japan on ‘insufficient disk space’

#154
post #150

Earlier quoted context omitted.

So don't waste your money on it then! Why in the world would you buy one "Enterprise" disk instead of three "consumer" disks?

Because if you're providing services to a large business, you're typically not managing servers with physical disks attached. You're using a SAN with fiber connections or NFS mounts. Most SANs require specific drives sold by the vendor with firmware they've tested, mounted to custom sleds etc. You can't just connect a WD Mybook drive. Mom and Pop businesses using single servers can do all they want in regards to driv…

If you are in that situation, then obviously you have to do what you have to do.

Absolutely none of that has to apply to the sort of situation we are talking about here.

The only real reason to spend more per disk is when you know all your disks are going to fail, and extending the lifespan per average disk will definitely save you more than the enterprise markup costs. So you better have dozens or hundreds of disks in the first place.

For any truly valuable data, you should at minimum have a backup in one different physical location. That backup should be include at least one redundant disk. None of that is worth spending a dime on more expensive hardware.

Re: Toyota blames factory shutdown in Japan on ‘insufficient disk space’

#158
post #42

Just-in-time will cost you far more than it will save you. The problem is that when it fails, you judge those failures as "backwards looking one offs" as opposed to a cost that is part and parcel to being JIT. Factory downtime is the single biggest cost a manufacturing shop can incur. No amount of working capital savings from carrying fewer screws, widgets, and bolts can offset the costs of taking 14 plants down due…

It would be nice if you could back that up with some facts, considering Toyota has been using some variant of this system since 1948. Please provide a citation for how JIT manufacturing cost TM "far more than it saved them"?

Bad business practices can survive for a long time.

I do not have any skin in on the game as to how effective or not JIT is, but wouldn’t the correct question be, how did Toyota fair vs non-JIT companies? As GP was implying, JIT is fine day-to-day, but can make failure conditions worse.

Re: Toyota blames factory shutdown in Japan on ‘insufficient disk space’

#159

Earlier quoted context omitted.

Just wondering, why pi?

I've tested both π and g, and while they both work well, g results in far fewer disk full errors. I've heard c works even better, though I haven't tried it yet.

> I've tested both π and g, and while they both work well, g results in far fewer disk full errors. I've heard c works even better, though I haven't tried it yet.

Good to know. FWIW i should also be avoided. It's tempting to use, since most programs use it as a counter, so it /should/ standardize the logfilesizes. But in practice it's very tricky to get a definitive disk space requirement with it.

Re: Toyota blames factory shutdown in Japan on ‘insufficient disk space’

#160
post #18

Earlier quoted context omitted.

I wonder, wouldn't it be more of a question of how much later to shut down. Or in other words, how big is the time buffer you get for not being "just in time"? The article suggests (the way I read it, at least) that it is some fundamental time span, but I feel like it's more like half a day?

In some (American) automotive plants I've worked in, JIT lifetimes are about 10 hours. The order for a seat or mirror or door handle doesn't exist at the tier 1 suppliers where my equipment gets installed until 10 hours before it is needed. At that point, my PLC gets a request to induct a particular part number and paint color into the manifest, inducts it and acknowledges that request, the machine builds it, laser-m…

Do the plants have capacity to store more if they wanted? I am thinking of foreseeable short term disruptions which could break the 10 hour windows.

Hurricanes, blizzards, train/truck worker strikes, cargo ship stuck in a canal, etc. For a multinational, those are not black swan events, something of that nature is going to break the pipeline on a regular basis. Potentially for days of backlog.

Clearly it must mostly work, but that is some crazy thin margin. I assume whoever works logistics is just perpetually on the edge of a heart attack.

Post reply on HN