Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Be Aware of the Makefile Effect

blog.yossarian.net

Post reply on HN
Print view View original

281–290 of 347 posts

Re: Be Aware of the Makefile Effect

#281

Post by bluGill »

PaulHoule wrote: post #278
kragen wrote: post #167

Earlier quoted context omitted.

Steam engines currently power most of the world's electrical grid. The main reason for this is that, completely contrary to what you said, they are more efficient and more reliable than diesel engines. (Electric motors of course are not a heat engine at all and so are not comparable.) Steam engines used to be very inefficient, in part because the underlying thermodynamic principles were not understood, but also becau…

Steam engines are thoroughly obsolete in the developed world where there are natural gas pipeline networks. People quit building coal burning power plants in North America at the same time they quit burning nuclear power plants for the same reason . The power density difference between gas turbines and steam turbines is enough that the capital cost difference is huge. It would be hard to afford steam turbines if the…

Gas is often used to power the boilers because steam is so much better.

Re: Be Aware of the Makefile Effect

#282

Post by fargle »

cassepipe wrote: post #184
fargle wrote: post #138

calling this "Makefile" effect is a terrible disservice. one could as easily call it "PHP" effect, "YAML" effect, etc. pick whichever language you'd personally like to denigrate. there is nothing that makes makefiles inherently more or less susceptible to this. if it's more common, it's because people don't want to take the time doing more solid engineering and clean design for something like a ci/cd config or a make…

The reason why it seems to apply to makefiles in particular is because most people think life is too short to bother learning and understanding makefiles so it seems to happen there more than anywhere else. Also no matter how complicated and subtle you think your makefile is, true experts will tell you it's wrong and you instead copy their apparently over-engineered, hard to understand makefile

> Also no matter how complicated and subtle you think your [thing] is, true experts will tell you it's wrong and you instead copy their apparently over-engineered, hard to understand [thing]

not unique at all to makefiles, probably not even in the top ten [things] that "true" experts like to "help" with

Re: Be Aware of the Makefile Effect

#283

Post by chuckadams »

stouset wrote: post #276
IgorPartola wrote: post #234

Make and Makefiles are incredibly simple when they are not autogenerated by autoconf. If they are generated by autoconf, don’t modify them, they are a build artifact. But also, ditch autoconf if you can. In the broader sense: yes this effect is very real. You can fall to it or you can exploit it. How I exploit it: write a bit of code (or copy/paste it from somewhere). Use it in a project. Refine as needed. When start…

They are also extremely limited. Timestamp-based freshness is often broken by modern VCSes. Git doesn’t record timestamps internally, so files can (and often do) have their mtime updated even when their contents are the same, causing unnecessary rebuilds. They also are utterly unable to handle many modern tools whose inputs and/or outputs are entire directories or whose output names are not knowable in advance of run…

I started using ccache to speed up Make, but soon found that allowed me to replace Make entirely with a bash script using a few functions.

Re: Be Aware of the Makefile Effect

#284

Post by kragen »

PaulHoule wrote: post #278
kragen wrote: post #167

Earlier quoted context omitted.

Steam engines currently power most of the world's electrical grid. The main reason for this is that, completely contrary to what you said, they are more efficient and more reliable than diesel engines. (Electric motors of course are not a heat engine at all and so are not comparable.) Steam engines used to be very inefficient, in part because the underlying thermodynamic principles were not understood, but also becau…

Steam engines are thoroughly obsolete in the developed world where there are natural gas pipeline networks. People quit building coal burning power plants in North America at the same time they quit burning nuclear power plants for the same reason . The power density difference between gas turbines and steam turbines is enough that the capital cost difference is huge. It would be hard to afford steam turbines if the…

There is some truth in what you say. Though steam engines still power most of the power grid (especially in the "developed world") their capital costs are indeed too high to be economically competitive.

However, there are also some errors.

In 02022 24% of total US electrical power generation capacity was combined-cycle gas turbines (CCGT), https://www.eia.gov/todayinenergy/detail.php?id=54539 which run the exhaust from a gas turbine through a boiler to run a steam turbine, thus increasing the efficiency by 50–60%. So in fact a lot of gas turbines are installed together with a comparable-capacity steam turbine, even today.

Syngas is not a technology that "just doesn't work". It's been in wide use for over two centuries, though its use declined precipitously in the 20th century with the advent of those natural-gas pipeline networks. The efficiency of the process has improved by an order of magnitude since the old gasworks you see the ruins of in many industrial cities. As you say, though, that isn't enough to make IGCC plants economically competitive.

The thing that makes steam engines economically uncompetitive today is renewable energy. Specifically, the precipitous drop in the price of solar power plants, especially PV modules, which are down to €0.10 per peak watt except in the US, about 15% of their cost ten years ago. This combines with rapidly dropping prices for batteries and for power electronics to undercut even the capex of thermal power generation rather badly, even (as you say) if the heat was free, whereas typically the fuel is actually about half the cost. I don't really understand what the prospects are for dramatically cheaper steam turbines, but given that the technology is over a century old, it seems likely that its cost will continue to improve only slowly.

Re: Be Aware of the Makefile Effect

#285

Post by kragen »

bluGill wrote: post #279
kragen wrote: post #168

Earlier quoted context omitted.

Yeah, I write most things in Python or JavaScript because it's much more practical.

Both have strong limits for writing complex code. Typescript is one attempt of an answer because bad as javascript is for large programs the web forces it. I prefer a million lines of c++ to 100k lines of python - but if 5k lines of python sill do them c++ is way too much overhead. (rust likely plays better than c++ for large problems from scratch but most large probles have existing answers and throwing something el…

I agree with all of this.

Re: Be Aware of the Makefile Effect

#286

Post by gjadi »

Terr_ wrote: post #274
RHSeeger wrote: post #266

Earlier quoted context omitted.

This. I have no urge to have git "clean" my project, because I'll lose a ton of files I have created locally. Rather, I want the project know what it creates when it builds and have the ability to clean/purge them. It's a never ending source of frustration for me that "gradlew clean" only cleans _some_ stuff, and there's no real "gradlew distclean".

Hmm, I wonder what's the best way™ to, er "locally backup" those files, in such a way that no git-clean invocation will remove them without promoting. All I can think of are things like periodically copying them to another folder, or give them a different ownership needed for edit/delete, etc. Unless there's some kind of .gitpreserve feature...

You can have locally ignores files in .git/info/exclude IIRC.

Re: Be Aware of the Makefile Effect

#287

Post by godelski »

huijzer wrote: post #64
mianos wrote: post #29

I have an alternate theory: about 10% of developers can actually start something from scratch because they truly understand how things work (not that they always do it, but they could if needed). Another 40% can get the daily job done by copying and pasting code from local sources, Stack Overflow, GitHub, or an LLM—while kinda knowing what’s going on. That leaves 50% who don’t really know much beyond a few LeetCode p…

> That leaves 50% who don’t really know much beyond a few LeetCode puzzles and have no real grasp of what they’re copying and pasting. Small nuance: I think people often don’t know because they don’t have the time to figure it out. There are only so many battles you can fight during a day. For example if I’m a C++ programmer working on a ticket, how many layers of the stack should I know? For example, should I know h…

Funny enough I'm an A̶I̶ML researcher and started in HPC (High Performance Computing).

  >  if I’m a C++ programmer ... should I know how the CPU registers are called?
Probably.

Especially with "low level"[0] languages knowing some basics about CPU operations goes a long way. You can definitely get away without knowing these things but this knowledge will reap rewards. This is true for a lot of system based information too. You should definitely know about things like SIMD, MIMD, etc because if you're writing anything in C/C++ these days it should be because you care a lot about performance. There's a lot of stuff that should be parallelized that isn't. Even stuff that could be trivially parallelized with OpenMP.

  > what should an AI researcher working always in Jupyter know?
Depends on what they're researching. But I do wish a lot more knew some OS basics. I see lots of things in papers where they're like "we got 10x" performance on some speed measurement but didn't actually measure it correctly (e.g. you can't use time.time and be accurate because there's lots of asynchronous operations). There's lots of easy pitfalls here that are not at all obvious and will look like they are working correctly. There's things about GPUs that should be known. Things about math and statistics. Things about networking. But this is a broad field so there are of course lots of answers here. I'd at least say anyone working on AI should read at least some text on cognitive science and neuroscience because that's a super common pitfall too.

I think it is easy to not recognize that information is helpful until after you have that information. So it becomes easy to put off as not important. You are right that it is really difficult to balance everything though but I'm not convinced this is the problem with those in that category of programmers. There's quite a number of people who insist that they "don't need to" learn things or insist certain knowledge isn't useful based on their "success."

IMO the key point is that you should always be improving. Easier said than done, but it should be the goal. At worst, I think we should push back on anyone insisting that we shouldn't be (I do not think you're suggesting this).

[0] Quotes because depends who you talk to. C++ historically was considered a high level language but then what is Python, Lua, etc?

Re: Be Aware of the Makefile Effect

#288

Post by MindSpunk »

mdaniel wrote: post #191
pantalaimon wrote: post #95

Earlier quoted context omitted.

Auto-generated Makefiles that CMake and Autotools produce really leave a bad impression on how complex Makefiles need to be.

Good news, you can change the output then! And for as much as you might not like its generated Makefiles, I assert that $(cmake -G Ninja) is 100,000,000x more "wtf" than -G Makefiles I disagree about the autotools ones, I find them very sane although autotools itself can die in a rotting dumpster out back. And take m4 with it.

Ninja isn't really a reasonable comparison to make. Ninja is explicitly not designed to be human authored like Makefiles are. Ninja is designed to be an output from some build system generator like CMake and doesn't make affordances for humans as it's intended for machine generation and machine consumption so it can be _very_ fast (which it is).

Re: Be Aware of the Makefile Effect

#289

Post by PaulHoule »

kragen wrote: post #284
PaulHoule wrote: post #278

Earlier quoted context omitted.

Steam engines are thoroughly obsolete in the developed world where there are natural gas pipeline networks. People quit building coal burning power plants in North America at the same time they quit burning nuclear power plants for the same reason . The power density difference between gas turbines and steam turbines is enough that the capital cost difference is huge. It would be hard to afford steam turbines if the…

There is some truth in what you say. Though steam engines still power most of the power grid (especially in the "developed world") their capital costs are indeed too high to be economically competitive. However, there are also some errors. In 02022 24% of total US electrical power generation capacity was combined-cycle gas turbines (CCGT), https://www.eia.gov/todayinenergy/detail.php?id=54539 which run the exhaust fr…

Yeah, and people are talking about renewables as if the storage is free. Or people quote case 17 out of

https://www.eia.gov/analysis/studies/powerplants/capitalcost...

as if 1.5 hours of storage was going to cut it. I've been looking for a detailed analysis of what the generation + storage + transmission costs of a reliable renewable grid is that's less than 20 years old covering a whole year and I haven't seen one yet.

Re: Be Aware of the Makefile Effect

#290

Post by rixed »

mauvehaus wrote: post #251
Quekid5 wrote: post #244

Earlier quoted context omitted.

They are simple but very often wrong . It's surprisingly hard to write Makefiles that will actually do the right thing under anything other than "build from scratch" scenarios. (No, I'm not joking. The very existence of the idea of "make clean" is the smoking gun.)

The idea that git offers a 'clean' command was revelatory to me. Your build system probably shouldn't need to know how to restore your environment to a clean state because your source control should already know what a clean state is . That's sort essential to serving its purpose, after all. I haven't yet run into a scenario where there was a clean task that couldn't be accomplished by using flags to git clean, usual…

make clean is supposed to clean the intermediary files only but keep the actual build targets (typically what you want to install)
Post reply on HN