Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

All my servers have an 8 GB empty file on disk

brianschrader.com

Post reply on HN
Print view View original

671–680 of 715 posts

Re: All my servers have an 8 GB empty file on disk

#671

Post by nybble41 »

cassepipe wrote: post #371
jsmith99 wrote: post #231

Earlier quoted context omitted.

The difference is that market mechanisms are meant to allocate items to those that value them the most (assumed to be the same as their willingness to pay). Claiming a licence by contrast has little cost.

I agree on the cost of claiming of the license on this specific case enables depletion of ressources for those who need it. But market mechanisms allow speculation. If you can mobilize capital quicker than others for a resource when you know it's going up, you are just pocketing the difference. Seems hard to justify as a mechanism that is beneficial to society as a whole.

The profit in speculation comes from correctly recognizing that a good will be worth more if consumed in the future than it would be if consumed in the present. It can be annoying to those who would have preferred to use the good in the present and are unable to afford the anticipated future prices, but in the end society benefits by having the good still available in the future for the more valuable use.

Re: All my servers have an 8 GB empty file on disk

#673

Post by cellularmitosis »

ericbarrett wrote: post #319
abruzzi wrote: post #267

Earlier quoted context omitted.

Fuel injected motorcycles don’t have reserve (at least, none that I’ve seen.) instead they have low fuel lights or full fuel gauges. I’m guessing it’s because the fuel pumps are in the tank and the fuel injection system needs high pressure.

Fuel injectors require filtered gas because even small particles can clog them, and said filter is more likely to be clogged or even compromised by sucking up the last drops of fuel (and scale and debris) in the tank, so the low-fuel warning is required. Carb jets can get clogged, too, but are wider since they're not under as much pressure. Also, since they're a wear item they're a lot easier to clean and/or replace.

I think grandparent commenter had it right: it's because the pump is in the tank. There's just no good way to have an external petcock determine where a tank-internal pump gets its fuel from.

Re: All my servers have an 8 GB empty file on disk

#674

Post by nybble41 »

abduhl wrote: post #662
nybble41 wrote: post #646

Earlier quoted context omitted.

> Say you budget $100 for dinner tonight and you go out and it costs $75. Do you owe the restaurant $25? No, but if you accurately forecast that dinner will cost $100 on average , and this time it only happened to cost $75, you should put most of the savings aside for the other times when it will cost $125 and not reallocate it to be spent on something else. Consistent over-budgeting is still an issue which would nee…

We’re arguing about different things it appears. This thread started with someone saying that a team coming in under budget should is “owed” that money in the future by management. I said this isn’t so and that it’s a self centered and myopic viewpoint. You are talking about punishment and reallocation, presumably by reducing the budget the next cycle. I’m not in favor of that unless it’s clear that the team is consi…

> For example, if a team says they need $100 a year and comes in at $90 then I don’t think next year’s budget should be $110 while some people in this thread think it should be.

IMHO it depends on why the expenses were less than the budget. If it's a matter of probability or essential uncertainty then the savings should be set aside for other occasions where luck isn't as favorable. If the department realized cost savings by improving business practices then most or all of the savings should stay with the department to be invested in future improvements (a one-time carry-over into the next budget period) and/or distributed as reward to those responsible for the improvements, as an incentive to continue making such improvements. If costs were lower because the department didn't accomplish everything they set out to do then that might be a justification for reallocating their budget, and/or implementing more drastic changes to get them back on track.

> Your point about average cost just means that you’re budgeting on the wrong timeframe.

The timeframe for the budget would generally be predetermined (e.g. one fiscal year) and not set by the department itself.

> If you estimate your average dinner is $100 but you’re spending $75 most of the time except for one huge dinner every month then you should be budgeting $75 for dinner and then budget separately for one large dinner a month.

Sure, but I was referring to probabilistic variation due to uncertainty in the forecast, not a predictable mix of large and small expenses. And the "dinners" in this analogy would be once per budget period (i.e. annual for most organizations), not frequent enough to average out.

Re: All my servers have an 8 GB empty file on disk

#676

Post by scottlamb »

neop1x wrote: post #590
scottlamb wrote: post #346

Earlier quoted context omitted.

You recall incorrectly; swap is not needed. It's not just me who runs without it; Google production machines did for many years. "The behavior is often worse without swap" is more vague / subjective. I prefer that a process die cleanly than everything slow to a crawl semi-permanently. I've previously written about swap causing the latter: https://news.ycombinator.com/item?id=13715917 To some extent the bad behavior c…

>> It's not just me who runs without it; Google production machines did for many years. But then you need other mechanisms monitoring "out of memory" situations. For example Kubernetes also require nodes to have swap disabled. But running workloads should have resource request/limits defined and then kubelet config specifies minimums on free disk and RAM memory. If there is not emough RAM, workloads are evicted to an…

I agree with your comment if you remove the words "but then" and "other". It's a good idea for orchestration software to detect/avoid unhealthy nodes and enforce the resource limits it uses for bin-packing. It's also a good idea for monitoring systems (by which I mean ones which do alerting and visualization) to track memory usage and pressure stalls. [1] I wouldn't say swap is a substitute for these in any way. A swapping machine is an unhealthy machine.

[1] https://www.kernel.org/doc/html/latest/accounting/psi.html

Re: All my servers have an 8 GB empty file on disk

#677

Post by sillysaurusx »

kortex wrote: post #632
sillysaurusx wrote: post #531

Earlier quoted context omitted.

Yay academic machine learning docker containers. What's a few 30GB docker images between friends? :) Hopefully docker will lose the fight for mindshare in the ML space.

What do you mean, "lose mindshare"? Docker is one of the absolute best things to happen to ML. My work is largely algo integration - I take academics' ML code and bundle it for API consumption. Before this, I was a bench chemist, so I kind of have a thing for reproducing experiments. The academic ML scene has a reproducibility crisis that makes other science reproducibility crises look like Phys 100 labs. These thing…

Ah, you're a docker zealot. I misread the joke.

You had a flash of self-awareness. Docker containers take massive amounts of disk space, and you need arcane knowledge just to use them. I'd rather focus on doing ML, not learning docker. In other words, "I'm getting to old for this." (33 is up there.)

It bugged me in the game industry when some old programmer said "shaders are a young person's game," implying that they wouldn't even look into how they worked. Now that I am that older programmer, or getting there, I see what they mean. Docker arcana is a young person’s game.

If Docker solved the reproducibility crisis, you might have a point. But it doesn't. Most of the crisis is the fact that (a) datasets are trapped behind institutions that won't make them available, (b) the models themselves aren't made available (OpenAI), and (c) the code itself isn't available (also OpenAI).

Those three things are the main problem. Forcing everyone to use a 30GB docker container just to do basic ML isn't going to do anything but waste time and turn newbies away from ML.

You're a fine debator though. It was an enjoyable read; have an upvote.

Re: All my servers have an 8 GB empty file on disk

#678

Post by abduhl »

nybble41 wrote: post #674
abduhl wrote: post #662

Earlier quoted context omitted.

We’re arguing about different things it appears. This thread started with someone saying that a team coming in under budget should is “owed” that money in the future by management. I said this isn’t so and that it’s a self centered and myopic viewpoint. You are talking about punishment and reallocation, presumably by reducing the budget the next cycle. I’m not in favor of that unless it’s clear that the team is consi…

> For example, if a team says they need $100 a year and comes in at $90 then I don’t think next year’s budget should be $110 while some people in this thread think it should be. IMHO it depends on why the expenses were less than the budget. If it's a matter of probability or essential uncertainty then the savings should be set aside for other occasions where luck isn't as favorable. If the department realized cost sa…

I think we agree in general and are just quibbling about the details of how to budget correctly (timeframes, line items, etc.). Most of the issues that come up with these stories of people getting their budgets slashed if they don't spend enough or having to buy a bunch of bullshit at the end of the year are just a result of poor budgeting at some point which has been allowed to continue.

Re: All my servers have an 8 GB empty file on disk

#680

Post by dang »

legulere wrote: post #125

[flagged]

Please don't take HN on generic ideological tangents. They lead to threads that are repetitive and therefore uninteresting, and often turn nasty as well.

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...

https://hn.algolia.com/?dateRange=all&page=0&prefix=true&sor...

We detached this subthread from https://news.ycombinator.com/item?id=26585073.

Post reply on HN