Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Warp – self-contained, single binary applications

github.com

Post reply on HN
Print view View original

141–150 of 157 posts

Re: Warp – self-contained, single binary applications

#141

Post by rkeene2 »

dev_dull wrote: post #65
rkeene2 wrote: post #29

Earlier quoted context omitted.

Tcl is still the king of this kind of distribution with Starkits/Starpacks. Since you can virtualize the filesystem within Tcl, you can pack in all your resources as regular files and generally scripts will work without knowing that this has happened (and without hacky extracting of the archive to disk before running, or while running).

This is an awesome design concept and I'm surprised other languages aren't copying it.

It generally takes other languages a long time to become more like Tcl ;-)

Re: Warp – self-contained, single binary applications

#142

Post by wildmusings »

jordwalke wrote: post #108
wildmusings wrote: post #9

From the README, it looks like this unpacks the application and dependencies into a local cache in the user's home directory on first run. Is it possible to instead execute the compressed application code directly and present to the resulting process a virtualized file system that decompresses the dependencies on the fly too? Then you could run the single binary without giving it any filesystem access. One way to do…

I asked this question just yesterday. The challenge (the one I was interested in) was if this can be done in a general purpose way for arbitrary programming languages so long as they used libc to go through the file system. I actually think it might be theoretically possible using LD_PRELOAD and some equivalent on OSX. I don't know why I'm compelled to want the executable to leave no trace on disk, but it seems much…

That’s a good idea, doing it by shimming libc, but it feels kind of fragile. Go, for example, does not use libc and makes syscalls directly instead, so this would not work for any fs-accessing Go code either in the application or its dependencies. Even for things that do go through libc, wouldn’t you end up in a cat and mouse game to add shims for any new file operations added to the kernel? I suspect even shimming the existing interfaces is less straightforward than at first thought.

I don’t think you can have a truly robust solution without kernel support. Maybe the FUSE kernel-side code already supports, or could be modified to support, per-process ad-hoc virtual file systems? If done right, you might even get existing fs caching and other optimizations for (closed to) “free”.

I also can’t quite explain why I find this so desirable. It just feels intuitively cleaner and more robust to not have to spray files elsewhere into some cache.

Re: Warp – self-contained, single binary applications

#143

Post by hbt »

ealexhudson wrote: post #12
sagichmal wrote: post #5

Earlier quoted context omitted.

> This sounds like a much less general binctr. binctr is Linux-only, and appears to need Docker as a runtime requirement? This has neither constraint, apparently?

I wrote stowage.org to do something similar, having a Docker runtime (which stowage _does_ have) is actually pretty neat in a couple of ways because you also get a kind of basic package management for free. It's pretty nice for sharing dev tools amongst a team. Maybe these things don't have broad use cases, but I think there are a lot of solvable pains here.

check out whalebrew it is similar to your project

Re: Warp – self-contained, single binary applications

#144

Post by sebazzz »

meddlepal wrote: post #79
pjmlp wrote: post #39

Earlier quoted context omitted.

You can bundle everything together, or use a commercial JDK with AOT compilation support.

Neither of those options are "painless".

Is it? Often you would distribute an installer anyway, right?

Re: Warp – self-contained, single binary applications

#145

Post by icedchai »

virmundi wrote: post #100
icedchai wrote: post #55

Earlier quoted context omitted.

You can build "fat jars" that contain all dependencies. IMO, this is how most Java apps should be deployed.

You still have issues of needing a JVM. The JVM is not a small runtime considation. I'm in the process of converting to GO since I have more time than money. 1 JVM in Docker uses 512 MB to sit still. A Go version is around 16.

Mostly FUD. You can easily tweak your min and max heap sizes. You can have an app that takes less that 512 megs of RAM.

Re: Warp – self-contained, single binary applications

#146

Post by angst »

I hope it can use downloaded executable for warp-runner. Downloading executable from windows console doesn't work too well in my corporate environment. Currently placing windows-x64.warp-runner.exe to same directory as warp-packer doesn't help. I really like the idea though. Simplest 3rd party solution I've encountered so far for the purpose.

Re: Warp – self-contained, single binary applications

#147

Post by debiandev »

weberc2 wrote: post #137
wahern wrote: post #116

Earlier quoted context omitted.

The implication is that statically linked binaries make it easier to throw in dependencies because you neither need to worry about how to ship them nor bother with re-using the shared libraries on the system. If there's a bug in Go's TLS implementation, you need to identify, recompile, and redistribute every Go application you use. Whereas if it were using the system's OpenSSL library everything would magically be fi…

> The implication is that statically linked binaries make it easier to throw in dependencies because you neither need to worry about how to ship them nor bother with re-using the shared libraries on the system. I don't buy this argument; it's every bit as easy to `pip install` something as it is to `go get` something. > If there's a bug in Go's TLS implementation, you need to identify, recompile, and redistribute eve…

> I think the fix is fairly trivial

Not at all. Package maintainers and the Security Team in Debian do plenty of manual work to backport and test security fixes.

> As soon as a security issue pops up for one of their dependencies, their automation automatically compiles a new version

That pulls down new releases for the dependency instead of backporting a security fix. Now you have no guarantee that the new binary will behave like the old one (minus the vuln). On the contrary, it would be practically a new release.

A lot of companies have security policies allowing security updates on live production systems. A complete rebuild against new libraries is not that.

Re: Warp – self-contained, single binary applications

#148

Post by Izkata »

ASalazarMX wrote: post #64
w8rbt wrote: post #4

Very neat. Would like to see more languages targeting single binaries. This is one reason (I believe) that has made Go so popular.

I'm ambivalent about this. It is cool to distribute a single binary, but years ago there was serious vulnerability in a very common open source library library whose name escapes me (it wasn't libgcc, someone help a brother here). It was used in multiple programs in both Windows and Linux. On Windows there was a rush of individual updates as each application was fixed... on Linux, simply upgrading that library fixed…

> but years ago there was serious vulnerability in a very common open source library library whose name escapes me (it wasn't libgcc, someone help a brother here).

I believe you're referring to an OpenSSH vulnerability?

...though, I did a search just now to find when it was from (I remember there being a major one around the time you're thinking of), and instead found posts from only two months ago about another one that has existed for at least 18 years.

Re: Warp – self-contained, single binary applications

#149

Post by rkeene2 »

interfixus wrote: post #94
open-source-ux wrote: post #11

It's fascinating how self-contained binaries are now coveted as such a desirable feature. Dozens of languages (if not more) were capable of this decades ago. In fact, some of them could generate self-contained binaries that allowed you to bundle app assets like icons and images inside the executable - no additional add-ons needed. And they were native executables, not complicated wrappers with layer upon layer of dif…

Typing me some Nim and shaking my head. Passing -static to the linker, I get neat, robust, non-dynamic executables, typically in the low MB or even kB size-range. const varname = staticRead"filename" embeds a given resource at compile time - to make the project completely selfcontained if need be, and just for the hell of it. I know you can do this in several other languages, but the Nim way is - unsurprisingly - the…

Tcl beats it in terms of both transparency and ready-made-ness. In Tcl, all filesystem access goes through the Tcl Virtual Filesystem (VFS) layer. This is a layer of indirection that allows for the interpreter, including at the script-level, to intercept filesystem calls and replace them with their own data. The default (and catch-all) driver is called "native" that just passes things to the native OS, but you can write other drivers and over the decades people have done so.

The most popular one is Starkit/Starpack which uses the Metakit database (which is a heirarchial database) and appends that archive to a loader (either a script, as a stub, or to an executable which includes the stub with the ability to open itself). This database can be modified externally using the starkit tools (sdx), and when you pack all your resources into it, it appears from within the interpreter that all the files are just normal files even though only the database is being accessed.

Re: Warp – self-contained, single binary applications

#150

Post by rkeene2 »

AnIdiotOnTheNet wrote: post #139
shaan7 wrote: post #136

Earlier quoted context omitted.

> so who cares if you waste a GB or so I wonder if that sounds arrogant to just me. As a user I DO CARE. Its frustrating to find applications shipping their own copy of libraries wasting precious disk space. When I was using a Mac, I had at least 4 applications shipping their own copy of Qt5.4 ... aaaarrgh Back on the Linux desktop, package managers handle this beautifully. And for the cases where you need to skip th…

Package managers come with their own limitations. You can't have multiple versions of the same application, packages are often out of date, packages sometimes conflict, you can't choose where applications are installed, etc. And storage is ridiculously cheap. You can get an 8TB HDD for $150[0]. Or a 1TB SSD for the same price[1]. [0] https://www.amazon.com/Seagate-Expansion-Desktop-External-ST... [1] https://www.amaz…

> You can't have multiple versions of the same application

In many package managers you can have multiple versions of the same application without a big deal -- in most cases you don't want to SUPPORT multiple versions of the same application, so there's always a trade-off.

Post reply on HN