Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

fabricate: The better "make". Finds dependencies automatically for any language.

code.google.com

Post reply on HN
Print view View original

31–40 of 44 posts

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#31

Post by benhoyt »

kragen wrote: post #18
benhoyt wrote: post #3

We've been using memoize to build stuff for a while now, but we needed something that worked under Windows too. So here's our "fabricate" build tool. It's BSD licensed -- enjoy!

Does it scan your entire filesystem for atimes so as not to miss the .h files over in /usr/include, and if not, how does it know which directories need scanning?

No, it defaults to recursively scanning the current directory, but you can easily override this if you want, eg:

    setup(dirs=['.', '../lib'])

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#32

Post by vr »

sorbits wrote: post #30

I have a machine with two quad-core CPUs, I want to run 8 parallel jobs at a time! Most of these make replacements start with design decisions (like using Python) that pretty much rule out ever reaching what make does best, and which is essential to get my build times improved by at least a factor of 6. Using a tool (like CMake) to generate your Makefiles is an excellent solution, heck, even a Python script that gene…

Using Python does not prevent parallel execution of tasks. It's just the build tasks graph that gets created by a single process, but then the build system can use all the available CPUs to actually execute it. See e.g. -j flag to SCons http://www.scons.org/doc/production/HTML/scons-user.html#AEN....

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#33

Post by benhoyt »

thingie wrote: post #16

Do I miss something? That example is like three times longer than an equivalent Makefile would needed to be, and I need to know Python, obviously. For a trivial project, I guess that Make would be easier to use, and for something more complex, I think I would have to do quite a lot of programming, with Fabricate…

To be fair, the equivalent Makefile with autodeps generation (using gcc's -M options) is about the same length.

However, as soon as you start adding more complex stuff the fabricate gets much easier to work with and maintain. Make's $< $@ stuff, but also because you're forced into a targets-are-files mould, and because you don't have a real programming language to work in.

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#34

Post by benhoyt »

sorbits wrote: post #30

I have a machine with two quad-core CPUs, I want to run 8 parallel jobs at a time! Most of these make replacements start with design decisions (like using Python) that pretty much rule out ever reaching what make does best, and which is essential to get my build times improved by at least a factor of 6. Using a tool (like CMake) to generate your Makefiles is an excellent solution, heck, even a Python script that gene…

Fair call. So far that hasn't been a problem for us -- but if you want to make it one, please donate us some cores! :-)

Seriously, as vr said, Python's not stopping us. And I imagine it wouldn't be too hard to add code to spawn off a few background tasks based on the dependency list.

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#35

Post by benhoyt »

tsally wrote: post #14

You don't need to enter in dependencies with 'make' either (suffix rules or makedepend). Don't get me wrong, fabricate looks great, but let's not misrepresent the power of make in the right hands. From the docs: files.o : files.c defs.h buffer.h command.h cc -c files.c I don't know a programmer beyond the introductory level that still writes this in their make file. :-p

True, one normally doesn't -- and shouldn't. But I saw a surprising number of Makefiles on Google Code Search done this way.

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#36

Post by benhoyt »

st3fan wrote: post #17
thingie wrote: post #16

Do I miss something? That example is like three times longer than an equivalent Makefile would needed to be, and I need to know Python, obviously. For a trivial project, I guess that Make would be easier to use, and for something more complex, I think I would have to do quite a lot of programming, with Fabricate…

What misses in my opinion is 'build by convention'. Most things that you want to build have recipes that are mostly the same. You have some sources that you want to compile and then link against some libraries. Using some platform specific compiler and linker flags. So, instead of writing a program to do the build I would rather see a simple DSL that allows me to specify just the minimal amount of configuration to ge…

Well, our experience says that as soon as you step outside that "minimal amount" you have to write all the build steps explicitly anyway, so we're specifically avoiding implicit rules. And we end up being explicit pretty quickly in all our projects -- maybe more than some folks because we're embedded developers and tend to need full control, but even so.

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#37

Post by bryhoyt »

gchpaco wrote: post #26

The central problem with these sorts of "make replacements" is they miss out on the whole thing that makes make useful. I care not a whit about implicit rules, and I would welcome a real language over GNU make's function apparatus (filenames with spaces in them tend to get split in the functions, due to the fundamentally macro style), but without pattern matching rules and a real dependency engine I won't even think…

gchpaco, could you give an example of a situation where you'd use a real dependency engine, which could not be solved (at least not easily) by using fabricate.py? I know such cases exist in theory, but have never run across a need, so I'm interested.

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#38

Post by modeless »

joe_bleau wrote: post #27
silentbicycle wrote: post #13

Earlier quoted context omitted.

Yes, via access times and strace: "When this build stage ran, it checked these files, so remember them as dependencies." Not magic at all, and should automatically work for languages that haven't even been invented yet.

Can it tell (using access times under windows) that the files were modified by that stage of the build command, and not by another unrelated process? How does it know which drives and directories to scan for recently modified files? Scanning all of them would take a while.

It is possible to do strace-like stuff on Windows too as evidenced by Process Monitor. http://technet.microsoft.com/en-us/sysinternals/bb896645.asp...

If scanning atimes is too slow or unreliable, perhaps fabricate could be updated to use procmon?

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#39

Post by kragen »

benhoyt wrote: post #31
kragen wrote: post #18

Earlier quoted context omitted.

Does it scan your entire filesystem for atimes so as not to miss the .h files over in /usr/include, and if not, how does it know which directories need scanning?

No, it defaults to recursively scanning the current directory, but you can easily override this if you want, eg: setup(dirs=['.', '../lib'])

Is there a way to override it to just use strace?

Re: fabricate: The better "make". Finds dependencies automatically for any language.

#40

Post by benhoyt »

kragen wrote: post #39
benhoyt wrote: post #31

Earlier quoted context omitted.

No, it defaults to recursively scanning the current directory, but you can easily override this if you want, eg: setup(dirs=['.', '../lib'])

Is there a way to override it to just use strace?

Sure is. This will do it:

    setup(runner='strace_runner')
Post reply on HN