Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

How do I inform Windows that I'm writing a binary file?

devblogs.microsoft.com

Post reply on HN
Print view View original

31–40 of 74 posts

Re: How do I inform Windows that I'm writing a binary file?

#31

Post by [deleted] »

p_l wrote: post #8
lmm wrote: post #4

The article seems to be taking the position that the C runtime library is not part of "Windows", which feels like a rather odd view to me. What is the stable API that Windows offers to application developers if not that?

Indeed, C runtime is not part of windows API, and it's normal to have a program include few different copies of C runtime library due to different modules compiled with different compilers/options. C runtime library being part of OS is accidental thing in Unix, 16bit and 32bit Windows API even does not use C-compatible ABI (instead, Pascal-compatible one is present)

[deleted]

Re: How do I inform Windows that I'm writing a binary file?

#32

Post by wvenable »

lmm wrote: post #14
ChrisSD wrote: post #13

Earlier quoted context omitted.

Calling it from C does not mean you need a full C standard library to exist. For example, much of the C standard library is itself written in C. But it's a "freestanding" C which assumes only a minimal set of library functions exist (e.g. functions for copying memory from one place to another, filling memory with zeroes, etc). And you can of course use non-C languages to call the Win32 API. Or even directly using ass…

> you can of course use non-C languages to call the Win32 API. Or even directly using assembly code. Is that a supported/official API though? On Linux you "can" put your arguments in registers and trigger the system call interrupt directly, and I think Go programs even do this, but it's not the official interface and they reserve the right to break your program in future updates, at least in theory.

> Is that a supported/official API though?

The Win32 API doesn't even use the "C" calling convention. C is just another language to Windows and the standard C library is a cross-platform library for C. You could also write C code on classic Mac OS and it had it's own API as well but more styled for Pascal.

The OS and C being closely related is not universal across all operating systems, it's just a Unix thing.

Re: How do I inform Windows that I'm writing a binary file?

#33

Post by mjg59 »

lmm wrote: post #14
ChrisSD wrote: post #13

Earlier quoted context omitted.

Calling it from C does not mean you need a full C standard library to exist. For example, much of the C standard library is itself written in C. But it's a "freestanding" C which assumes only a minimal set of library functions exist (e.g. functions for copying memory from one place to another, filling memory with zeroes, etc). And you can of course use non-C languages to call the Win32 API. Or even directly using ass…

> you can of course use non-C languages to call the Win32 API. Or even directly using assembly code. Is that a supported/official API though? On Linux you "can" put your arguments in registers and trigger the system call interrupt directly, and I think Go programs even do this, but it's not the official interface and they reserve the right to break your program in future updates, at least in theory.

This is incorrect. The syscall ABI is the supported stable ABI for Linux, not the libc API - there's no single supported C library for Linux, and libc often lags behind the kernel in terms of providing syscall wrappers, so punting it to that level wouldn't work. This is in contrast to the BSDs that have libc tightly coupled to the kernel.

Of course, the Linux solution results in some weirdness, especially because specs like POSIX cover the C API, not the syscall ABI. setuid() at the libc layer is specced as changing the UID for all threads in a process. The Linux setuid() syscall only changes the current thread[1], and it's up to the C library to do some absolute magic to then propagate that to all other threads. Which made things difficult for things not using the C library, like Go (https://github.com/golang/go/issues/1435). But that's still not an argument that the supported interface is the C library - the kernel advertises the interface it exposes via the syscall ABI, and will retain that functionality, and if you want POSIX compatibility then you get it from somewhere else.

[1] In Linux, a thread is just a very slightly special case of a process

Re: How do I inform Windows that I'm writing a binary file?

#34

Post by lmm »

Joker_vD wrote: post #27
lmm wrote: post #9

Earlier quoted context omitted.

> The Win32 API. E.g. using WriteFile to write files ( https://learn.microsoft.com/en-us/windows/win32/api/fileapi/ ...) Which is called from what, if not C? Does windows really offer no API for writing text (rather than bytes) to files? Or does it rely on the application developer to manage line endings in their own code? Neither of those sounds very developer-friendly.

From literally any language. The WriteFile function comes from kernel32.dll shared library, and follows the certain calling convention. You don't need to use this calling convention inside your own binary (and indeed, MinGW and MSYS use SysV ABI for everything except when calling Win32 API), or ask a random C runtime coming from God knows where to do this for you if you write something other than C. In the UNIX world…

> You don't need to use this calling convention inside your own binary (and indeed, MinGW and MSYS use SysV ABI for everything except when calling Win32 API), or ask a random C runtime coming from God knows where to do this for you if you write something other than C.

Well sure but you have to define it somewhere. At some point there's an interface where something that's part of the application asks something that's part of the OS to do something, and that interface had better be stable and well-specified. If you really want you can use a different interface from your C ABI, sure, but given that, like it or not, most of windows is written in C (or in C++ but using C linkage between component boundaries), what do you gain?

Re: How do I inform Windows that I'm writing a binary file?

#35

Post by lmm »

Joker_vD wrote: post #30
lmm wrote: post #16

Earlier quoted context omitted.

> The whole issue is specific to C and languages that copied C or use its runtime underneath in implementations (like Python) So it's "specific to" almost all programming languages in actual use. That's a rather esoteric point. > For reference, Unix has no API other than bytes either. Unix does offer an API for writing C-standard in-memory text strings to Unix-standard on-disk text files, it just happens to be the sa…

> Unix does offer an API for writing C-standard in-memory text strings Why on bloody Earth should a presumably generic-purpose OS provide a special API for dealing with internal representation of some data structure in a (particular) implementation of a (particular) programming language? Besides, it doesn't offer such an API anyhow; you need to take care to manually pass the result of a strlen() call instead of sizeo…

> Why on bloody Earth should a presumably generic-purpose OS provide a special API for dealing with internal representation of some data structure in a (particular) implementation of a (particular) programming language?

Because the purpose of the OS is to facilitate applications (and, on the other end, facilitate hardware), and those applications tend to have a need to process text in-memory and then store it on the filesystem?

Re: How do I inform Windows that I'm writing a binary file?

#36

Post by Joker_vD »

lmm wrote: post #34
Joker_vD wrote: post #27

Earlier quoted context omitted.

From literally any language. The WriteFile function comes from kernel32.dll shared library, and follows the certain calling convention. You don't need to use this calling convention inside your own binary (and indeed, MinGW and MSYS use SysV ABI for everything except when calling Win32 API), or ask a random C runtime coming from God knows where to do this for you if you write something other than C. In the UNIX world…

> You don't need to use this calling convention inside your own binary (and indeed, MinGW and MSYS use SysV ABI for everything except when calling Win32 API), or ask a random C runtime coming from God knows where to do this for you if you write something other than C. Well sure but you have to define it somewhere. At some point there's an interface where something that's part of the application asks something that's…

> At some point there's an interface where something that's part of the application asks something that's part of the OS to do something, and that interface had better be stable and well-specified.

It's defined, and well-specified.

> your C ABI

Which is a C ABI. Borland's Turbo C and C++Builder used different ABI than Microsoft C compiler did. GCC for Windows used to use a third, entirely different ABI as well. The ABI is not part of the language definition, you see.

> most of windows is written in C

And compiled with a very specific C compiler that used a particular ABI. That only means that you need to follow it when you call into the OS, sure, but not that you have to stick to it anywhere else — and indeed, most implementations of many programming languages on Windows didn't; they invented and used their own ABIs.

Re: How do I inform Windows that I'm writing a binary file?

#37

Post by Joker_vD »

lmm wrote: post #35
Joker_vD wrote: post #30

Earlier quoted context omitted.

> Unix does offer an API for writing C-standard in-memory text strings Why on bloody Earth should a presumably generic-purpose OS provide a special API for dealing with internal representation of some data structure in a (particular) implementation of a (particular) programming language? Besides, it doesn't offer such an API anyhow; you need to take care to manually pass the result of a strlen() call instead of sizeo…

> Why on bloody Earth should a presumably generic-purpose OS provide a special API for dealing with internal representation of some data structure in a (particular) implementation of a (particular) programming language? Because the purpose of the OS is to facilitate applications (and, on the other end, facilitate hardware), and those applications tend to have a need to process text in-memory and then store it on the…

All you need for that is the ability to read and write binary blobs to and from files, which Windows gives you, and to know what "text files" means for the other programs on that platform. Windows itself doesn't care for text much; but the other programs have a shared convention that ASCII text files have CRLF-separated variable-length lines of text, and Unicode text files store text in UTF16-LE, (including the CRLF pairs, so those look like "\x0D\x00\x0A\x00" as raw bytes).

All of this is left to the user space to sort out, just as it is on Linux, so I am not entirely sure why you demand Windows to do more for you than Linux does.

Re: How do I inform Windows that I'm writing a binary file?

#38

Post by manwe150 »

lmm wrote: post #34
Joker_vD wrote: post #27

Earlier quoted context omitted.

From literally any language. The WriteFile function comes from kernel32.dll shared library, and follows the certain calling convention. You don't need to use this calling convention inside your own binary (and indeed, MinGW and MSYS use SysV ABI for everything except when calling Win32 API), or ask a random C runtime coming from God knows where to do this for you if you write something other than C. In the UNIX world…

> You don't need to use this calling convention inside your own binary (and indeed, MinGW and MSYS use SysV ABI for everything except when calling Win32 API), or ask a random C runtime coming from God knows where to do this for you if you write something other than C. Well sure but you have to define it somewhere. At some point there's an interface where something that's part of the application asks something that's…

Even so, most of Windows historically did not use C ABI, but rather stdcall, so specifying a call from your C library to the Windows C library couldn’t be done in a purely standards-compliant C compiler (which doesn’t have calling convention modifiers), in a slightly pedantic quirk of the C spec design

Re: How do I inform Windows that I'm writing a binary file?

#39

Post by ChrisSD »

userbinator wrote: post #18
ChrisSD wrote: post #5

Earlier quoted context omitted.

The Win32 API. E.g. using WriteFile to write files ( https://learn.microsoft.com/en-us/windows/win32/api/fileapi/... ) It wasn't until fairly recently that the C runtime was stably shipped with Windows. Previously you had to install the correct version of the C library alongside your application.

It wasn't until fairly recently By "recently" you mean Win95? MSVCRT.DLL has been there for at least that long.

For some background on what I meant see:

https://devblogs.microsoft.com/oldnewthing/20140411-00/?p=12...

https://learn.microsoft.com/en-us/cpp/windows/universal-crt-...

Re: How do I inform Windows that I'm writing a binary file?

#40

Post by lmm »

Joker_vD wrote: post #36
lmm wrote: post #34

Earlier quoted context omitted.

> You don't need to use this calling convention inside your own binary (and indeed, MinGW and MSYS use SysV ABI for everything except when calling Win32 API), or ask a random C runtime coming from God knows where to do this for you if you write something other than C. Well sure but you have to define it somewhere. At some point there's an interface where something that's part of the application asks something that's…

> At some point there's an interface where something that's part of the application asks something that's part of the OS to do something, and that interface had better be stable and well-specified. It's defined, and well-specified. > your C ABI Which is a C ABI. Borland's Turbo C and C++Builder used different ABI than Microsoft C compiler did. GCC for Windows used to use a third, entirely different ABI as well. The A…

> Which is a C ABI. Borland's Turbo C and C++Builder used different ABI than Microsoft C compiler did. GCC for Windows used to use a third, entirely different ABI as well.

Sure, you can do that. Userspace code can use any ABI it wants, or none. But again, why, what do you gain?

And regardless of whether it's "the" ABI or merely "a" ABI, that ABI presumably has a representation for strings and allows passing them around - and while you certainly could use a different representation in your program (or in the OS internals) and transform strings back and forth when calling the OS (or when receiving calls from userspace), you probably don't want to. At which point we're back at needing a way to write strings in an in-memory format to OS-standard files in the filesystem.

Post reply on HN