Building Raspberry Pi Systems with Yocto

I have similar problem: we have consumer device on top of imx6. I used iMX6 SoloLite evaluation board kit (1GB of memory) with Fedora for arm as development prototype. For me, installing a package is not a problem at all: "dnf install package". I planed to use Raspbian or Fedberry on actual device, but our management blindly switched to Yocto, without comparing of alternatives. To install package on Yocto, we need to create our own recipe in our own layer,then rebuild Yocto, fix bugs and repeat, and then deliver update. We spent man-year to achieve same result as we already had with Fedora. We seriously lag behind schedule because two developers (and two consultants) are working on developing of our own custom OS instead of working on our application. Moreover, instead of single system for development/testing and production (Fedora or Debian on developer workstation, in Docker for CI, on actual device) we now have mix of two systems. I created configuration script with about fifty options to be able to compile for host and for Yocto at same time (including workaround for number of bugs in Yocto). Enormous development time was sink into ground because of this schizophrenia.

- Small enough that it fits into RAM.

- Fetched over the network on boot, PXE-style.

- Configured to not write any files to the greatest extent possible, and where writing files can not be avoided one would use tmpfs.

With the above three criteria satisfied, the following is achieved:

a) We never write to an SD card so we avoid SD card corruption issues.

b) Since the whole system image is sitting in RAM, we could do without NFS, Samba or similar either.

We do need to persist some data, but that data is coming from our application and is going to be stored in a database on a server on the local network.

I am currently working on developing the application that our image will run, but I have dug up some preliminary information relating to what I said above:

- Read-only Raspberry Pi [1]. Applies to Raspbian but certainly provides a starting point for doing the same thing with Yocto.

- Poky NFS Root [2]. Like I said, I don't need the NFS root part if the image can fit in RAM, but the portions that deal with gPXE might certainly be relevant.

- The State of Netbooting Raspberry Pis (December 2017) [3]. "Net-booting works well on the Raspberry Pi B Model 3 without an SD Card." So actually we won't be needing gPXE.

- Network Boot Your Raspberry Pi [4]. Referenced by [3] and provides the steps needed to configure Raspberry Pi 3 for network booting without an SD-card.

I am not completely decided upon whether or not to avoid NFS yet though. Because we never persist any files on the root file system anyway, serving said fs over NFS can be done in read-only mode, and we would put each revision of the corresponding root file system in a uniquely named top-level directory so that a client running the current version image has one place that it gets the files it expects from and the next version of the image which might expect other files or differing layout would get its files from another directory. Old root file system directories which are no longer in use would be removed so this scheme doesn't require much in terms of storage space.

It should be noted that NFS has some security issues [5], but our deployment is going to be running on an isolated network exclusive to the hardware that we are providing, so this concern is already resolved. Said isolated network will connect our embedded Raspberry Pis, a server (TFTP, database and NFS), and a tablet configured and provided by us. The tablet provides a graphical user interface that our customers will use to interact with the system.

[1]: https://learn.adafruit.com/read-only-raspberry-pi/

[2]: https://wiki.yoctoproject.org/wiki/Poky_NFS_Root

[3]: https://blog.alexellis.io/the-state-of-netbooting-raspberry-...

[4]: https://www.raspberrypi.org/documentation/hardware/raspberry...

[5]: https://www.tldp.org/HOWTO/NFS-HOWTO/security.html

I guess the fundamental big problem you are facing is SD card corruption. In our experience, we've found the SanDisk Extreme Pro cards to be most reliable. (coupled with a good power supply).

Read-only rootfs is almost a must have for any production environment.

resinOS does not satisfy all your requirements (nfs boot, state in memory etc).. But we do have stuff in place to reduce the problems you are facing.

- There is an initramfs in the kernel. When the kernel loads in memory, it runs fsck on the root/data partitions before mounting the rootfs. - The rootfs is read-only with only a few configuration files in the state partition. - Applications run inside a container so you can basically run rasbian on top of resinOS.

I wrote more about resinOS in another comment highlighting remote application container updates/vpn access etc.

https://news.ycombinator.com/item?id=18093336

Disclaimer: I work at resin in the OS team. I've noted down your feedback about running purely in memory.

The ability to modify existing recipes through append files is nice, but that seems like it could be handled in a normal distro just by forking a package and making changes.