Hacker News Board Hacker News, browsed like a classic message board
Live data from Hacker News

Ubus (OpenWrt micro bus architecture)

openwrt.org

Post reply on HN
Print view View original

41–50 of 56 posts

Re: Ubus (OpenWrt micro bus architecture)

#41

Post by bombela »

mkj wrote: post #26
fulafel wrote: post #2

Did they consider using a memory-safe language for this?

A decent question, I've wondered about Rust programs on openwrt but at the moment they're just too big! Linking the stdlib into every binary doesn't help, and the code is generally larger than the C equivalent. It doesn't seem unsolvable though, I'm hopeful. no_std rust binaries can be near competitive with C. Binary size is sometimes a big deal. Two decades ago I investigated using C++ for an "embedded" linux ssh se…

I used Rust on an attiny22 to decode standard 433MHz remote radio packets. It is little 8bit microcontroller with 256 bytes of RAM and enough flash to store about 1024 instructions.

The key was to turn off panic unwinding, turn on size optimization, and full LTO. I think I recall increasing the inlining threshold helped a bit.

Re: Ubus (OpenWrt micro bus architecture)

#42

Post by westurner »

PhilipRoman wrote: post #40

As someone having to debug issues on OpenWRT derivatives, I wish I had a time machine to tell the inventors of Unix to never add other forms of IPC than pipes. It's all a big pile of stateful daemons notifying other daemons with a billion race conditions and zero debugging capabilities, like a parody of how not to create reliable systems. When you have shell scripts parsing JSON messages, you know it's over.

But then how to add datatypes - schema - to pipes? How do we add a column to the messages on the pipe?

Newline-delimited != SOTA

https://en.wikipedia.org/wiki/Apache_Arrow :

> Arrow allows for zero-copy reads and fast data access and interchange without serialization overhead between these languages and systems

JSON lines formatted messages can be UTF-8 JSON-LD: https://jsonlines.org/

Linux networking is now all built on eBPF.

Re: Ubus (OpenWrt micro bus architecture)

#43

Post by hydroid7 »

PhilipRoman wrote: post #40

As someone having to debug issues on OpenWRT derivatives, I wish I had a time machine to tell the inventors of Unix to never add other forms of IPC than pipes. It's all a big pile of stateful daemons notifying other daemons with a billion race conditions and zero debugging capabilities, like a parody of how not to create reliable systems. When you have shell scripts parsing JSON messages, you know it's over.

Unfortunately pipes don't cover all kind of IPC we use today.

Re: Ubus (OpenWrt micro bus architecture)

#44

Post by packetlost »

westurner wrote: post #42
PhilipRoman wrote: post #40

As someone having to debug issues on OpenWRT derivatives, I wish I had a time machine to tell the inventors of Unix to never add other forms of IPC than pipes. It's all a big pile of stateful daemons notifying other daemons with a billion race conditions and zero debugging capabilities, like a parody of how not to create reliable systems. When you have shell scripts parsing JSON messages, you know it's over.

But then how to add datatypes - schema - to pipes? How do we add a column to the messages on the pipe? Newline-delimited != SOTA https://en.wikipedia.org/wiki/Apache_Arrow : > Arrow allows for zero-copy reads and fast data access and interchange without serialization overhead between these languages and systems JSON lines formatted messages can be UTF-8 JSON-LD: https://jsonlines.org/ Linux networking is now all buil…

The same way you add it to any other SOCK_STREAM? How do we add types and schemas to TCP connections?

Re: Ubus (OpenWrt micro bus architecture)

#45

Post by westurner »

packetlost wrote: post #44
westurner wrote: post #42

Earlier quoted context omitted.

But then how to add datatypes - schema - to pipes? How do we add a column to the messages on the pipe? Newline-delimited != SOTA https://en.wikipedia.org/wiki/Apache_Arrow : > Arrow allows for zero-copy reads and fast data access and interchange without serialization overhead between these languages and systems JSON lines formatted messages can be UTF-8 JSON-LD: https://jsonlines.org/ Linux networking is now all buil…

The same way you add it to any other SOCK_STREAM? How do we add types and schemas to TCP connections?

But then when I need to scale to more than one node with multiple cores, do pipes scale?

Re: Ubus (OpenWrt micro bus architecture)

#46

Post by the8472 »

PhilipRoman wrote: post #40

As someone having to debug issues on OpenWRT derivatives, I wish I had a time machine to tell the inventors of Unix to never add other forms of IPC than pipes. It's all a big pile of stateful daemons notifying other daemons with a billion race conditions and zero debugging capabilities, like a parody of how not to create reliable systems. When you have shell scripts parsing JSON messages, you know it's over.

Using SOCK_SEQPACKET would be saner than pipes or SOCK_STREAM.

Re: Ubus (OpenWrt micro bus architecture)

#47

Post by the8472 »

mkj wrote: post #26
fulafel wrote: post #2

Did they consider using a memory-safe language for this?

A decent question, I've wondered about Rust programs on openwrt but at the moment they're just too big! Linking the stdlib into every binary doesn't help, and the code is generally larger than the C equivalent. It doesn't seem unsolvable though, I'm hopeful. no_std rust binaries can be near competitive with C. Binary size is sometimes a big deal. Two decades ago I investigated using C++ for an "embedded" linux ssh se…

> Linking the stdlib into every binary doesn't help

It should be possible to use a shared library for std but it would come with the limitation that the bins must be compiled with exactly the same rustc as std.

Re: Ubus (OpenWrt micro bus architecture)

#48

Post by packetlost »

westurner wrote: post #45
packetlost wrote: post #44

Earlier quoted context omitted.

The same way you add it to any other SOCK_STREAM? How do we add types and schemas to TCP connections?

But then when I need to scale to more than one node with multiple cores, do pipes scale?

Yes, with some work: https://mazzo.li/posts/fast-pipes.html

Re: Ubus (OpenWrt micro bus architecture)

#49

Post by teunispeters »

fulafel wrote: post #2

Did they consider using a memory-safe language for this?

From working with OpenWRT for years on embedded systems where sometimes 128 bytes made a difference: this is not a useful question.

In the embedded space, you use small effective languages like assembly and C. You either design for explicit memory use entirely, or you systematically test for memory usage and waste - especially if you're delivering industrial applications. The so-called "memory safe" systems tend to be very expensive for memory usage and space, and not worth the investment. Usually, anyway. This may change in the future but it's a bad change if it comes with additional power usage requirements and the environment also requires minimal power use.

Re: Ubus (OpenWrt micro bus architecture)

#50

Post by codehero »

PhilipRoman wrote: post #40

As someone having to debug issues on OpenWRT derivatives, I wish I had a time machine to tell the inventors of Unix to never add other forms of IPC than pipes. It's all a big pile of stateful daemons notifying other daemons with a billion race conditions and zero debugging capabilities, like a parody of how not to create reliable systems. When you have shell scripts parsing JSON messages, you know it's over.

Pipes and named FIFOs are easy and great. I say this after implementing various IPC methods (unix domain sockets with fd passing, POSIX message queues, 0MQ, XML RPC, local TCP sockets, just to name a few). Use a simple line oriented protocol. If you are passing complex data through your IPC, you know it's time for files. Shared memory is another way to do IPC but hopefully you have robust method of detecting the liveliness of your local processes and you want to give up the Unix file paradigm.
Post reply on HN