Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

fabricate: The better "make". Finds dependencies automatically for any language.

code.google.com

Post reply on HN
Print view View original

41–44 of 44 posts

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#41

Post by benhoyt »

modeless wrote: post #38
joe_bleau wrote: post #27

Earlier quoted context omitted.

Can it tell (using access times under windows) that the files were modified by that stage of the build command, and not by another unrelated process? How does it know which drives and directories to scan for recently modified files? Scanning all of them would take a while.

It is possible to do strace-like stuff on Windows too as evidenced by Process Monitor. http://technet.microsoft.com/en-us/sysinternals/bb896645.asp... If scanning atimes is too slow or unreliable, perhaps fabricate could be updated to use procmon?

Here's a version of strace for Windows: http://www.intellectualheaven.com/default.asp?BH=projects&#3...

I've hacked it to print the actual file names on CreateFile() calls, but it's failing because it doesn't recurse and trace sub-processes. It should be theoretically possible by hooking CreateProcess(), but it's probably hard.

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#42

Post by sorbits »

vr wrote: post #32
sorbits wrote: post #30

I have a machine with two quad-core CPUs, I want to run 8 parallel jobs at a time! Most of these make replacements start with design decisions (like using Python) that pretty much rule out ever reaching what make does best, and which is essential to get my build times improved by at least a factor of 6. Using a tool (like CMake) to generate your Makefiles is an excellent solution, heck, even a Python script that gene…

Using Python does not prevent parallel execution of tasks. It's just the build tasks graph that gets created by a single process, but then the build system can use all the available CPUs to actually execute it. See e.g. -j flag to SCons http://www.scons.org/doc/production/HTML/scons-user.html#AEN... .

As I said, generating a Makefile (i.e. the dependency graph) from Python would be fine, which conceptually sounds like what SCons do (from your comment).

But look at the build file example on the fabricate page, it is a user supplied build file with a build function which calls a compile function which loops over sources. Having the fabricate system take input like that and do parallel execution is as hard as solving the halting problem.

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#44

Post by gchpaco »

bryhoyt wrote: post #37
gchpaco wrote: post #26

The central problem with these sorts of "make replacements" is they miss out on the whole thing that makes make useful. I care not a whit about implicit rules, and I would welcome a real language over GNU make's function apparatus (filenames with spaces in them tend to get split in the functions, due to the fundamentally macro style), but without pattern matching rules and a real dependency engine I won't even think…

gchpaco, could you give an example of a situation where you'd use a real dependency engine, which could not be solved (at least not easily) by using fabricate.py? I know such cases exist in theory, but have never run across a need, so I'm interested.

Since fabricate makes Python, a full language available, it's not that there's things that cannot be expressed, it's really about ease. So I'll tell you about something I did when I was a grad student.

I was using an up to date TeX install. My advisor wasn't. To placate him, I had to copy all of my packages, and all the packages they referred to, and so on down the line so that he could compile my papers. I did this entirely within make + some small shell scripts. The fact that I could write scanners for %.tex and %.sty and %.cls and have GNU make (obviously) automagically use them to build the dependency scripts I instructed it to include was immensely convenient. This is not impossible to do procedurally, but it was inconvenient, and the all-make solution had the attractive side that whenever there were bugfixes to the packages they got automatically included in what I was doing.

A better example, come to think of it, is the typical LaTeX woe; you can't build the final .dvi until you have accurate page numbers, references, tables of contents, indexes. You get all of those by--building the .dvi. So there's a strange circular dependency here. Make doesn't deal with this especially well, although it can be done. Ant and SCons don't even try.

Post reply on HN