Kids today are so spoiled. :P I remember back in my Ericsson days when an incremental build would take 40 minutes, and then another 20 or so to load the resulting 100 MB monolithic executable onto hardware and boot it up. That kind of turnaround time teaches you to think through your code thoroughly, and use the type system to catch bugs early.
The Build is Always Broken
51–60 of 75 posts
Re: The Build is Always Broken
#52People love to complain about Webpack, but it pretty much does all of this out of box, all the way down to hot-reloading UI components. In that sense it's way ahead of all C++, C#, Java, etc build systems I know. Doing an `npm run dev` type of command in a reasonably set up JS project is very much this staged execution model the author talks about. Everything gets cached, on disk and in memory, only changes are recom…
I’ve never had anything close to this experience with Webpack. In my experience all the magic “it just works” of incremental updates and live reloading is extremely fragile and just gives up silently all the time in large-ish projects. Adding to that, it’s incompetent at what’s supposed to be its core feature: bundling JavaScript modules. I would count it among the least reliable software systems I have to work with regularly.
Re: The Build is Always Broken
#53People love to complain about Webpack, but it pretty much does all of this out of box, all the way down to hot-reloading UI components. In that sense it's way ahead of all C++, C#, Java, etc build systems I know. Doing an `npm run dev` type of command in a reasonably set up JS project is very much this staged execution model the author talks about. Everything gets cached, on disk and in memory, only changes are recom…
You won't get what true "liveness" is until you work in Lisp or Smalltalk. The ability to change a function in a running program, without restarting it. Automatic reload might seem cool, but reload means your application restarts and loses state. Imagine you have some Web app and you open some menus/dialogs, perhaps it's connected to some server via WebSockets etc. and you find a bug somewhere, you go to the code to…
Huh?
I do procedural music in JS/web audio, and my projects are set up so that changes to the logic and instruments all happen live, while the music continues to play.
How is that different from what you're describing? What specifically can't webpack/HMR do?
Re: The Build is Always Broken
#54Earlier quoted context omitted.
You won't get what true "liveness" is until you work in Lisp or Smalltalk. The ability to change a function in a running program, without restarting it. Automatic reload might seem cool, but reload means your application restarts and loses state. Imagine you have some Web app and you open some menus/dialogs, perhaps it's connected to some server via WebSockets etc. and you find a bug somewhere, you go to the code to…
> You won't get what true "liveness" is until you work in Lisp or Smalltalk. The ability to change a function in a running program, without restarting it This can be done in a number of other common languages today such as Java (JVM languages) and C# (probably all .NET languages). Most IDEs I've seen for those platforms support it. Substantial enough changes to the code can require an application restart, but most ch…
You'd be surprised to find out that Common Lisp has actually quite a strong type system (not Haskell-level but much better than C++/Java); just that it's optional. You get the best of both worlds — fast prototyping, and then when you decide upon the types, you can add type declarations, which usually result in (much!) faster code and robustness.
Re: The Build is Always Broken
#55As others have noted, the title is confusing; this is really talking about live code reloading. "The build" => offline compilation in general, "is always broken" => is not as good as live reloading.
I've used live reloading in HyperCard, where your application (the "stack") is always running and you can interactively edit the code for each UI element. I think this can work really well, but only under certain constraints:
- You have a very clean separation between code and data (in HyperCard, the data is the set of cards and backgrounds, and the contents of all the fields; the code is the event handlers attached to each object).
- The data format is stable.
- There are clear checkpoints where your data is stable and no code is running.
Those are sometimes the case in some applications. Arguably, all of them are nice properties and therefore good targets to aim for.
But there are plenty of normal scenarios that don't meet those conditions:
- You have an expensive long-running task. Trying to change the code half-way through is a fool's errand. You need to re-run it from the start, and/or find a way to split it up into smaller steps. Unit tests and a good incremental build system are your friends here.
- You're working on the core data structures. Again, trying to update the code while all the data is still in memory is a bad idea. Just use a good build system and try to make the compilation and startup time as fast as possible.
- You're changing the navigation system, so the user's current position in the app won't make sense any more. You'll need to reset back to the front page (or whatever).
For that last one, "apply changes" in C# or Android Studio will generally do the right thing, even though those are mostly traditional build systems.
There isn't really a hard distinction between a build system and live coding, it's a spectrum. I'd argue further that a build system (in the traditional "make" fashion) is more fundamental, because you can achieve anything with a build system (even if it's tedious and inefficient) whereas some tasks are just intractable with live coding. A fast, reliable and correct build system (like Bazel, as several people have mentioned already) is the best foundation for everything else.
Re: The Build is Always Broken
#56Another point that seems related is the difficulty of call/cc semantics when dealing with changes in state that are outside the semantics of the virtual machine. It seems to me that there is no easy way to avoid cases like described in the article without forcing explicit accounting of a massive amount of implicit state. Depending on your use case the tradeoff might not be worth it. On the other hand, I would argue t…
Yes, that's a good way of expressing it!
I was just thinking that a running application is a little kernel of Code (the stuff that gets checked into source control) and big wrapper of State (what the user is currently doing with it).
The build system view is that the Code is sacrosanct, and what we need to focus on is checking the right stuff into source control and trying to ensure the latest commit is always correct and self-consistent. We should always be able to throw away all the state and rebuild everything from scratch.
The live coding view is that the user is more important than the Code, so we should focus on letting them get stuff done as effectively as possible. And programmers are users too! Therefore we should be able to modify the code without losing any of the user's state.
Both views are correct and what's actually needed is a good balance between the two.
Re: The Build is Always Broken
#57Is rebuilding everything from scratch on every build actually that common? If use use docker’s —cache-from flag on build, then assuming you’ve already pushed an image if you don’t make any changes on the part of the code that image is for you’re not going to actually rebuild that. That part of your build is just docker pull, build using cache, no changes so no work.
I'd say for compiled languages it's probably the norm. (i.e. where builds take one huge source repository and produce one set of compiled binaries e.g. a desktop app). Depending on how clever the build system is, I have been burned by "build hygiene" in the past. Some would go so far as even nuking the source repository and cloning it again rather than updating or running "git clean" etc, to be absolutely sure that n…
The norm is:
- Use an incremental build during development
- If you hit weird errors, try a clean build
- Always use a clean build for releases
Make is vulnerable to weird errors, as are newer systems like Gradle. Better build systems work hard to be 100% correct. Bazel is the best example I've seen.
I'd say working with an extremely reliable build system is just as eye-opening as working with a good live coding system. If you never have to do a clean build, and the incremental build is fast, that's most of the benefits of live coding right there.
Re: The Build is Always Broken
#58What? Sometimes live or pseudo-live is helpful when programming or debugging, and therefore the whole concept of building software from scratch is invalid?
"building software from scratch" as in `make clean && make` rather than F5 or Shift+F12, not as in "cat > a.out"
The solution to that being time-consuming is just to not make clean
Re: The Build is Always Broken
#59OP should take a look at bazel and reevaluate his opinions. Things like executing only tests when necessary, caching/remote executing everything and stellar cross language support make it easier than ever to rebuild the world every time.
(Can't tell if this is the joke) OP worked at Google as one of the creators of Dart so I assume he had a chance to encounter Bazel. Reading the post I got the feeling that these takes are partly informed by his encounters with die-hard Build System Believers.
Here's a question for die-hard live coding believers: what gets checked into source control?
I enjoy live coding, but the idea of checking in an "image" of the running system sounds horrible. You want to clearly distinguish between the key stuff and all the incidental state. The key stuff is what gets checked in, and that's your build system right there.
(I don't ask just for the sake of arguing, I'm interested in the answer! And I don't see it addressed anywhere in the article.)
Re: The Build is Always Broken
#60Earlier quoted context omitted.
I'd say for compiled languages it's probably the norm. (i.e. where builds take one huge source repository and produce one set of compiled binaries e.g. a desktop app). Depending on how clever the build system is, I have been burned by "build hygiene" in the past. Some would go so far as even nuking the source repository and cloning it again rather than updating or running "git clean" etc, to be absolutely sure that n…
The norm is: - Use an incremental build during development - If you hit weird errors, try a clean build - Always use a clean build for releases Make is vulnerable to weird errors, as are newer systems like Gradle. Better build systems work hard to be 100% correct. Bazel is the best example I've seen. I'd say working with an extremely reliable build system is just as eye-opening as working with a good live coding syst…
Incremental where you need to stop and restart running is still much slower than hot-edits (as in editing a dynamic/script language file) or hot-reloading statically compiled as C#). So clean build, incremental build+restart, incremental build while running I think is the 3 levels. I wonder if what the article is on about wrt. smalltalk is the last one.