Live data from Hacker News

Porting Python to GRUB: “Networking without an operating system”

lwn.net

11–20 of 49 posts

Re: Porting Python to GRUB: “Networking without an operating system”

#11

I'm one of the authors of BITS, and the presenter of the PyCon presentation this article discusses. Happy to answer any questions here.

Did you modify the CPython binary to run in the BIOS, or did you write a new interpreter from scratch? What was the hardest part of implementing this?

Re: Porting Python to GRUB: “Networking without an operating system”

#12
TempleOS is God's official temple. It will be burned into all Intel chips in the Intel factory in a ROM.

God says... still's clubbing temblors blenders personnel's upheaval boost ascend Medan CompuServe Tutankhamen log's mangrove weirdo coroner's homeliest transcending wefts dissent's inspector's Grammy meteorites purrs jargon's Billie's trapped lads coverall's tuckers shrilled Churchill auditions

Re: Porting Python to GRUB: “Networking without an operating system”

#13

why would they not use lua ? which is pretty popular (and pretty much designed) as embedded scripting languages. Not to mention much more lightweight than python.. and has a much smaller footprint. Nginx is a pretty good example of this.

Why would they not use C?

Or forth, or any number of alternatives.

You would get the same question with the names shuffled around whichever one you chose.

Re: Porting Python to GRUB: “Networking without an operating system”

#14

I'm one of the authors of BITS, and the presenter of the PyCon presentation this article discusses. Happy to answer any questions here.

Did you modify the CPython binary to run in the BIOS, or did you write a new interpreter from scratch? What was the hardest part of implementing this?

We build CPython's source into BITS, which runs as an EFI binary. (Or as a non-EFI image bootable on a BIOS, but that version doesn't include network support.) We have almost no changes to CPython itself, other than a couple of minor fixes to enable some additional calling conventions in the ctypes module.

The hardest part of porting Python in general was porting libffi to support the EFI calling convention, which required some very fiddly assembly. That libffi support allows us to call EFI protocols from Python without writing any C code.

The hardest part of implementing networking was handling the impedance mismatch between POSIX select and EFI asynchronous calls. POSIX select lets you ask "do any of these sockets have data to read", without actually reading the data; likewise for pending client connections to accept(). EFI expects you to provide a buffer to read the data and a callback for when the data gets read. So, the only way to ask for pending data is to buffer that data for the application to read later. This requires additional copies, limiting performance. (You might have heard of "zero-copy" networking; this is "several-copy" networking.)

Re: Porting Python to GRUB: “Networking without an operating system”

#15

why would they not use lua ? which is pretty popular (and pretty much designed) as embedded scripting languages. Not to mention much more lightweight than python.. and has a much smaller footprint. Nginx is a pretty good example of this.

What is it that makes the language suitable for being embedded? Actually, what does it mean to be embedded?

You can launch a lua vm with an empty environment table (in other words no standard library). This means only the functions you manually put into the environment table can be used which makes sandboxing extremely easy. I must admit I've only used luaj where interop to java boils down to something like http://stackoverflow.com/a/19629914 and you can optionally use reflection for even easier intertop if I recall correctly.

Re: Porting Python to GRUB: “Networking without an operating system”

#16

Earlier quoted context omitted.

> why would they not use lua ? The target audience for this is BIOS developers and firmware engineers. Several existing development tools in that area already support Python, including Simics and the Intel ITP, so the target audience is used to it. Lua also has several other impedance mismatches with the target audience, such as 1-based array indexing (remember, the target audience writes assembly and C). While peopl…

thank you for that. That explains a lot ! But Lua is already in Grub right ? wouldnt it have been more effective to just extend that anyway ?

Lua isn't part of upstream GRUB; it lives in a separate module, "grub-extras". As far as I can tell, it also wouldn't be easy to extend that with additional functionality or modules without forking it, due to the way GRUB builds out-of-tree modules.

Note that this port of Python is almost as old as grub's Lua; we've been working on BITS since ~2009, and it has had Python support since May 2011.

For reference, the Python port consists of less than 1000 lines of code, most of which provides compatibility implementations of standard C and POSIX functions that Python expects. (Plus another couple thousand lines implementing additional C Python modules like "_smp", "_acpi", and "_efi", but those are getting smaller as we move more code into Python using ctypes.)

Re: Porting Python to GRUB: “Networking without an operating system”

#17

I'm one of the authors of BITS, and the presenter of the PyCon presentation this article discusses. Happy to answer any questions here.

I am always interested to hear when a language is ported to a new plaform, in this case Python (or a subset of) on GRUB. The question would be why? Is this because you have lots of test cases already in Python and can easily deployed those test cases on using this?

Re: Porting Python to GRUB: “Networking without an operating system”

#18

I'm one of the authors of BITS, and the presenter of the PyCon presentation this article discusses. Happy to answer any questions here.

I am always interested to hear when a language is ported to a new plaform, in this case Python (or a subset of) on GRUB. The question would be why? Is this because you have lots of test cases already in Python and can easily deployed those test cases on using this?

Originally, it was because we had a lot of one-off test cases not written in Python. Before BITS, most BIOS test programs consisted of one-off DOS programs, or later one-off EFI programs. If you wanted a new test, or you wanted a customer to quickly gather some information for you, you sent them a new test program, or a bootable image. Now, many of those one-off tests have become Python one-liners, and you can use Python to explore the system with an interactive REPL.

Python also lets us provide additional APIs to make scripting easy. For instance, we include the complete ACPICA interpreter, so you can evaluate an arbitrary ACPI method with arguments and process the result. And we have an FFI with EFI support, so you can locate and call an arbitrary EFI protocol.

Re: Porting Python to GRUB: “Networking without an operating system”

#20

Was the talk recorded and can we expect it on the Pycon 2016 youtube channel[0] at some point? It's not there right now as far as I can tell. [0] https://www.youtube.com/c/pycon2016/videos

Yes: https://www.youtube.com/watch?v=AlkKvetGFSk

The first few minutes of the talk got cut off; the conference organizers have the original footage and plan to re-cut and re-upload it at some point.

The talk from PyCon 2015 on the Python port itself is available as well: https://www.youtube.com/watch?v=bYQ_lq5dcvM

Post reply on HN