Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Cosmopolitan Libc: build-once run-anywhere C library

justine.lol

Post reply on HN
Print view View original

81–90 of 171 posts

Re: Cosmopolitan Libc: build-once run-anywhere C library

#81

Post by pjmlp »

jart wrote: post #47
moonchild wrote: post #42

Earlier quoted context omitted.

Ntdll is the kernel ABI on windows; it is stable, and cosmopolitan uses it. Not sure what you mean by ‘on Windows thing are more stable’; syscall numbers change all the time. See https://j00ru.vexillium.org/syscalls/nt/64/

Cosmopolitan uses the stable WIN32 API on Windows. The word ABI is used loosely in this context. It's somewhat accurate since Cosmopolitan still configures the assembler to generate the binary linkage by hand. NTDLL APIs are only used on rare occasions, such as sched_yield(). When it is used, it's used in accordance with Microsoft's recommendations: namely having fallback. It'd be great if we were authorized to use t…

They didn't had to copy anything, NeXTSTEP already did it by having a FreeBSD kernel copied into their hybrid kernel.

That doesn't mean it is going to stay like that forever, the UNIX network stack is already considered legacy for modern code.

https://developer.apple.com/videos/play/wwdc2017/707/

https://developer.apple.com/videos/play/wwdc2017/709/

Also I have some doubts about support in non-POSIX OSes like IBM i, z/OS, ClearPath MCP, or RTOS like INTEGRITY.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#83

Post by jart »

etamponi wrote: post #77

This project sounds amazing and very interesting! I have one question: how does it work on bare metal? I tried to look for an explanation in the documentation but I couldn't find anything. The specific questions I have are: 1) How does this interact with a boot loader like GRUB? Or does it come with its own boot loader? But in this case, shouldn't this go in a specific location on the boot disk? 2) How does it handle…

Author here. Bare metal so far has been a "we built it because we could" kind of feature, so please be warned it's not as polished. Here's a quick tutorial.

Basically, ape/ape.S is a traditional BIOS bootloader. It also embeds a GRUB stub, which basically just switches back to the real mode BIOS bootloader entrypoint. So it should work with or without GRUB. The boot sector loads the rest of your executable of disk and prints "actually portable executable" to your display. It then configures standard i/o to use serial uart, sets up the page tables, enters long mode, and calls _start(). That's all I've got working so far. I would love to have e1000 and virtio though!

To try it out, here's how you build the included PC BIOS system emulator and boot an x86-64 ape executable that prints a spinning deathstar:

    m=tiny; make -j12 MODE=$m o/$m/tool/build/blinkenlights.com o/$m/tool/viz/deathstar.com &&
      o/$m/tool/build/blinkenlights.com -rt o/$m/tool/viz/deathstar.com
Video screencast: https://justine.lol/cosmopolitan/deathstar.html

See also: https://justine.lol/blinkenlights/manual.html

Enjoy!

Re: Cosmopolitan Libc: build-once run-anywhere C library

#84

Post by GordonS »

jes wrote: post #16
jart wrote: post #11

Earlier quoted context omitted.

Author here. Here's the best explanation I've thought of so far: > If it's this easy, why has no one done this before? The best answer I can tell is it requires an minor ABI change, where C preprocessor macros relating to system interfaces need to be symbolic. This is barely an issue, except in cases like switch(errno){case EINVAL:...}. https://justine.lol/ape.html It also took an act of willpower. This is a novel ap…

Have you tried to code-sign executables produced by this method?

On Windows, Authenticode signatures rely on and use PE header - I'd also love if the author could answer this one. If it does work, I'd also like to know why it works :)

Re: Cosmopolitan Libc: build-once run-anywhere C library

#85

Post by roelschroeven »

danellis wrote: post #63

"Run anywhere" appears several times in the page, but it seems to be a wild exaggeration, especially given the comparison to Java.

Indeed: as far as I understand it, Cosmopolitan works across different OS's (or even without OS), but not across different CPU architectures. Comparing that to Java is a gross exaggeration indeed.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#86

Post by gary_0 »

smcameron wrote: post #48
jart wrote: post #44

Earlier quoted context omitted.

Author here. That's not an inherent limitation. See https://justine.lol/apelife/index.html for an example of a GUI + TUI that's built using Cosmopolitan. The reason why I said what you quoted, is I'm simply trying to calibrate expectations. I don't view desktop GUIs as a productive area of focus, because there's such a lack of consensus surrounding the APIs that requires, and web browsers do a great job. As for sizeo…

Thanks for the informative reply. > and web browsers do a great job. Suppose one had a native linux game that uses POSIX, OpenGL and SDL. Is there any hope?

Having given the apelife source a glance, it seems quite possible to me; it already shows you how to create a window on Win32.

And SDL2 has already been ported to Emscripten/WASM after all, and OpenGL is really no trouble, since it's usually accessed through a mostly platform-agnostic C API wrapper already (see for example, https://github.com/Dav1dde/glad ).

And maybe you could use MinGW to support POSIX on Win32...

Re: Cosmopolitan Libc: build-once run-anywhere C library

#87

Post by roca »

jart wrote: post #34
TimTheTinker wrote: post #32

This is an interesting, and potentially compelling argument that WebAssembly binaries and runtime actually aren’t the future of compile-once-run-anywhere systems software: these support syscalls, sockets, and stdin/stdout —- all of which requires platform-specific glue code to achieve using WebAssembly and standard libc (or JS).

Author here. WebAssembly has played an important role protecting CI on the Internet and it's cool that it enables us to run LLVM in the browser. I was very surprised when I learned that some people were using it offline outside of browsers, since doing so requires a JVM-like runtime such as wasmtime. Cosmopolitan proves that it's possible to just fix C instead, which might fix all the stuff that's built on top of C t…

"Only require JIT compilation if they're launched on a non-x86 machine" is a heck of a caveat.

The x86-64 instruction set is not a good portable IR. Some well-known problems:

* It's much more irregular to decode than other instruction sets/IRs

* There is a truly vast set of instructions, very many of which are almost never used. So in practice you need to define the subset of x86-64 you're using. E.g. does your subset contain AVX-512? AVX2? AVX? BMI? X87? MMX? XGETBV? etc etc etc. These decisions will impact the performance of your code on actual x86 CPUs as well as non-x86 CPUs.

* x86-64 assumes the TSO memory model which is horribly expensive to support on CPUs whose native model is weaker (e.g. ARM) (which is why Apple added a TSO mode for M1; no other ARM chips have this)

Honestly, declaring x86-64 your portable IR and then claiming that as a technological breakthrough sounds like a trick to me. I'd agree it's a breakthrough if you define your x86-64 subset, show your x86-to-ARM compiler, and show that it produces code competitive with your competitors (e.g. WASM compilers).

Re: Cosmopolitan Libc: build-once run-anywhere C library

#88

Post by thequux »

jart wrote: post #74
pjmlp wrote: post #70

Earlier quoted context omitted.

> We can in fact have first-class portable native binaries, which will only require JIT compilation if they're launched on a non-x86 machine. This is nothing new, IBM and Unisys mainframes have been doing it since the 60's.

IBM System/360 and NexGen i586 were big inspirations for the design of Cosmopolitan.

Are you familiar with IBM i (née AS/400)? There's a similar idea there, where programs are compiled to a virtual instruction set called TIMI then statically recompiled to whatever the machine's native instruction set is at the first load on a given machine. As a result, you can just drop a System/36 binary on a modern PowerPC system and not even notice that you've just crossed two changes of instruction set.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#89

Post by jart »

modeless wrote: post #59
jart wrote: post #19

Earlier quoted context omitted.

Author here. I intend to upstream a patch soon with zsh that restores backwards compatibility with the Thompson Shell. As for GUIs, while Cosmpolitan isn't intended to write complex GUI apps (since I normally build web apps for that) it still can be used to build WIN32 GUIs. See https://justine.lol/apelife/index.html as an example of Conway's Game of Life built with Cosmopolitan, where it manages to embed a UNIX TUI…

I think the right way to add GUI support to this is to use the web. Simply embed an HTTP server in your binary and open the user's default web browser on launch. I have a framework for this here: https://github.com/jdarpinian/web-ui-skeleton

Author here. I agree. Check out redbean which is a single-file distributable web server, built with cosmopolitan: https://justine.lol/redbean/index.html redbean is forking web server (works great on windows) that can serve 1 million+ gzip encoded responses per second (only linux and freebsd are capable of that kind of performance) because APE binaries are isomorphic to the zip format which enables kernelspace copies. You just bundle all your assets inside the .com executable by changing the extension to .zip and the web server will deliver them via http.

Re: Cosmopolitan Libc: build-once run-anywhere C library

#90

Post by anon321321323 »

roelschroeven wrote: post #85
danellis wrote: post #63

"Run anywhere" appears several times in the page, but it seems to be a wild exaggeration, especially given the comparison to Java.

Indeed: as far as I understand it, Cosmopolitan works across different OS's (or even without OS), but not across different CPU architectures. Comparing that to Java is a gross exaggeration indeed.

What if you include an emulator in the executable? eg x86-64 on ARM. Then you could slowly run the x86 code on the pi for various values of "slow".
Post reply on HN