Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

The convergence of compilers, build systems and package managers

blog.ezyang.com

Post reply on HN
Print view View original

11–20 of 42 posts

Re: The convergence of compilers, build systems and package managers

#11

Post by spc476 »

johncolanduoni wrote: post #7

I think a good way to handle this convergence would be to move away from repeated invocations of the compiler. Instead, your build system/IDE could keep an active instance of the compiler running that keeps the last generated ASG/intermediate representation/machine code in memory, along with bookkeeping information that remembers the relations between these. When a file changes/a rebuild is requested, this lets the b…

At work I do multi-language development. One program has a mixture of C, C++ and Lua as a (more or less) static executable (Lua modules written in C and Lua are included in the executable). It was easy enough to add the steps to make to ensure the Lua files were recompiled (into an object file) if they changed. I'm bullish (skeptical) that this will be easy with an all-singing-all-dancing compiler/build/package program.

Re: The convergence of compilers, build systems and package managers

#12

Post by jimrandomh »

I'd prefer to see package managers be more separate. Downloading dependencies involves interacting with third parties and has security implications. That's fine when you're downloading software intentionally from a source you trust, but I've had the experience of downloading source code and noticing that the build system is downloading from sources I don't recognize or without https, and that's not so good.

Re: The convergence of compilers, build systems and package managers

#13

Post by Ace17 »

henrikschroder wrote: post #5
fizixer wrote: post #4

There is a problem but you're looking at it the wrong way. What's in the compiler's machine code generation phase that the build system needs to know about? If nothing, then making a monolithic system is only going to make your life miserable. Well-designed compilers are already split into (at least) two subsystems: frontend and backend. Frontend takes the program and spits an AST (very roughly speakign, although the…

> I think AST and semantic-analyzer are going to play an increasing role in a variety of software development activities It would be fantastic if source control systems would work on the AST instead of the plain text files, so many annoying problems could be solved there.

There are simpler and less intrusive ways to solve outside-AST issues (like an automatic reformatting pass before each build or each diff)

What you're suggesting raises a bunch of new (non-trivial) issues:

- What would you do with code comments? Things like "f(/+old_value+/new_value)".

- How to store code before preprocessing (C and C++) ?

- How to store files mixing several languages (PHP, HTML, Javascript) ?

- How do you store code for a DSL?

Re: The convergence of compilers, build systems and package managers

#14

Post by cat-dev-null »

Configuration management systems (chef, puppet, cfe2/3) is a superset of package management#, which is, in turn, depends on frozen static artifacts of each project's build system.

# This is because configuration management installs files, packages, templates files and runs commands pre/post, similar to how most package managers work, but at a fine-grain level of user customized as opposed to maintainer customized.

The meta is that one could consider a "system" or approach by where the project build and configuration management systems were seamless. One main challenge in doing so would be that the staticness of artifacts allows for reproducible compatibility, whereas end-to-end configurability can easily become Gentoo.

Re: The convergence of compilers, build systems and package managers

#15

Post by GFK_of_xmaspast »

henrikschroder wrote: post #5
fizixer wrote: post #4

There is a problem but you're looking at it the wrong way. What's in the compiler's machine code generation phase that the build system needs to know about? If nothing, then making a monolithic system is only going to make your life miserable. Well-designed compilers are already split into (at least) two subsystems: frontend and backend. Frontend takes the program and spits an AST (very roughly speakign, although the…

> I think AST and semantic-analyzer are going to play an increasing role in a variety of software development activities It would be fantastic if source control systems would work on the AST instead of the plain text files, so many annoying problems could be solved there.

What kinds of problems you talking about.

Re: The convergence of compilers, build systems and package managers

#16

Post by GFK_of_xmaspast »

Tyr42 wrote: post #10
haberman wrote: post #8

This analysis misses one major part of the equation: configuring the build. Almost every non-trivial piece of software can be built in multiple configurations. Debug vs. Release, with/without feature X, using/not using library Y. The configuration of the build can affect almost every aspect of the build. Which tool/compiler is called, whether certain source files are included in the build or not, compiler flags (incl…

"simple make" is ninja, right? https://ninja-build.org/ Already works with cmake and so on.

Doesn't work with fortran, which I found out to my chagrin just this afternoon.

Re: The convergence of compilers, build systems and package managers

#17

Post by GFK_of_xmaspast »

haberman wrote: post #8

This analysis misses one major part of the equation: configuring the build. Almost every non-trivial piece of software can be built in multiple configurations. Debug vs. Release, with/without feature X, using/not using library Y. The configuration of the build can affect almost every aspect of the build. Which tool/compiler is called, whether certain source files are included in the build or not, compiler flags (incl…

I hadn't heard of CML2, but it looks like even several years later it was remembered as a "no don't do that": https://lkml.org/lkml/2007/7/28/145 (also: http://www.linuxtoday.com/developer/2002021700120NWKNDV " What ESR should do is try create less controvery. Do not change megabyte to mebibyte in the help files. Find out why Linus doesn't include the updated configure.help files, then fix them and get them included right away. Do not change the UI, then old UI is documented and people are familiar with it." )

Re: The convergence of compilers, build systems and package managers

#18

Post by doug1001 »

is sbt an early prototype of what the OP has in mind? I like sbt because it seems to me that by combining (at least parts) of each of compiling, building, and packaging, the entire workflow is streamlined. So for instance, sbt has an incremental re-compiler which applies a suite of heuristics to minimize the code required to be recompiled and is triggered automatically by any change to the source. In practice this is a huge time saver, but it wouldn't work without relying on sbt's obviously detailed knowledge of the dependency graph. Another example: sbt can also handle package management and deployment, largely via plugins (eg, "native-packager", "sbt-ghpages", "assembly" (uber jar), "sbt-elasticbeanstalk"

Re: The convergence of compilers, build systems and package managers

#19

Post by anewhnaccount »

spc476 wrote: post #11
johncolanduoni wrote: post #7

I think a good way to handle this convergence would be to move away from repeated invocations of the compiler. Instead, your build system/IDE could keep an active instance of the compiler running that keeps the last generated ASG/intermediate representation/machine code in memory, along with bookkeeping information that remembers the relations between these. When a file changes/a rebuild is requested, this lets the b…

At work I do multi-language development. One program has a mixture of C, C++ and Lua as a (more or less) static executable (Lua modules written in C and Lua are included in the executable). It was easy enough to add the steps to make to ensure the Lua files were recompiled (into an object file) if they changed. I'm bullish (skeptical) that this will be easy with an all-singing-all-dancing compiler/build/package progr…

Bullish means optimistic.

Re: The convergence of compilers, build systems and package managers

#20

Post by spc476 »

anewhnaccount wrote: post #19
spc476 wrote: post #11

Earlier quoted context omitted.

At work I do multi-language development. One program has a mixture of C, C++ and Lua as a (more or less) static executable (Lua modules written in C and Lua are included in the executable). It was easy enough to add the steps to make to ensure the Lua files were recompiled (into an object file) if they changed. I'm bullish (skeptical) that this will be easy with an all-singing-all-dancing compiler/build/package progr…

Bullish means optimistic.

You're right. I mean bearish. Sigh.
Post reply on HN