tiny-initramfs - A minimalistic initramfs implementation

This is a very minimalistic initramfs
implementation for booting Linux systems. It has nearly no features,
but is very small and very fast. It is written purely in C, but uses
only parts of the standard library.

There are three primary use cases:

It is designed for systems where an initramfs is typically not
necessary (block device drivers + root file system compiled into the
kernel, no separate /usr file system), but where an initramfs is
required for microcode upgrades. Instead of having to use a full
initramfs, which is larger (more time spent in the boot loader
loading it) and slower (because it does more), tiny-initramfs will
add next to no overhead.
In cases where UUID-based boot is wanted not a full initramfs.
In systems with a split /usr file system, it is necessary to mount
that in the initramfs already, else subtle problems may occur. If
/usr resides on a simple block device already known to the kernel
(without user space helpers such as udev), tiny-initramfs provides
a mechanism with very little overhead to mount it before the system
is started.

Features

Simplicity: the implementation is really simple and very linear.
It's most likely easier to understand than other initramfs
implementations. The entire program is less about 2500 LoC, and that
includes the License headers in the files.
Size: the implementation is really small (see below).
Speed: there is no noticeable performance penalty, because very
little is done before execution is handed over to the operating
system proper.
Supports kernel-named devices, for example root=/dev/sda1.
Supports root=0xMAJMIN.
Supports root=UUID=... for ext2, ext3, ext4, xfs and btrfs.
Supports parsing /etc/fstab to determine if a separate /usr
partition exists and mounting that - as long as the entry there
follows the same rule as the root= parameter (kernel device name,
or UUID= entry for a select number of filesystems).
Supports the root=, rootflags=, rootfstype=, rootdelay=,
rootwait, ro, rw and init= parameters.
Default timeout of 180 seconds to wait for the root device to appear
(starts after the rootdelay= delay is over), after which a kernel
panic is caused; if rootwait is specified it will wait
indefinitely. (Recompilation is necessary for a larger timeout.)
Supports mounting NFSv4 file systems with sec=sys that do not
use the idampper, i.e. use raw UIDs/GIDs. For the /usr file system
the standard /etc/fstab entries are interpreted, for the root file
system one should use root=/dev/nfs and the nfsroot= parameter
(as documented in the kernel documentation). The network
configuration needs to be specified via the ip= kernel command
line parameter.
Very trivial module loading support (no automatic dependency
resolution).

When compiled on an x86_64 system with the default -Og compiler flags,
statically linked against dietlibc 0.33~cvs20120325-6, the binary
stripped and the resulting initramfs (without any modules added)
compressed with gzip -9, the images produced are between 9 kiB and
14 kiB, depending on the feature set selected.

Using musl instead of dietlibc adds between 1.8 and 2.4 kiB to the
resulting initrd.img size (depending on the feature set).

Using glibc instead of dietlibc adds around 310 kiB to the resulting
initrd image and is not recommended (although it will work).

Adding modules to the initramfs will increase the size, and many block
device and file system drivers are 100s of kiB in size. On the other
hand, the kernel would be larger if they were compiled in, so the
actual amount of space lost due to using modules is quite a bit
smaller.

Requirements

The kernel should have the necessary block device and file system
drivers built-in that are required to access the root and /usr
file systems. Warning: this is not true for most default kernels
of mainstream distributions, as they require a full initramfs to
load the modules required to mount the root file system.
If the necessary drivers are not built into the kernel, there is
limited support for loading modules from within the initramfs, see
below for details.
The kernel must have CONFIG_DEVTMPFS built-in, because this
implementation assumes the kernel will just create the devices by
itself. (This is true for most distribution kernels.)
NFSv4 requires at least kernel 3.5 on both server and client (in
order for raw UIDs/GIDs to work) and requires built-in kernel
support for network autoconfiguration (CONFIG_IP_PNP and for DHCP
support also CONFIG_IP_PNP_DHCP) as well as built-in kernel
support for NFSv4 (CONFIG_NFS_FS as well as CONFIG_NFS_V4).

When not to use

tiny-initramfs does not support PARTUUID= for mounting the root
or /usr file systems. It also doesn't support symlinks created by
udev (such as /dev/disk/by-label/...). Only the kernel names
themselves, such as /dev/sda1, as well as UUID= and hexadecimal
device numbers (0xMAJMIN, e.g. 0x801) are supported.
NFSv2/NFSv3 are currently not supported.
When booting from USB storage you should always use UUID=, because
device names are not necessarily stable.
/usr on a FUSE file system, as they require user space helpers
running to be able to mount. Generally speaking, any file system
that can't be mounted with just a trivial mount syscall, but
requires a userspace helper, will not work.
Any complex storage setup, such as LVM, encryption, iSCSI, etc.
Basically, only things that the kernel provides devices for out of
the box (potentially with additional kernel parameters) is
supported.

If your setup falls into one of these cases, please use a full
initramfs instead of tiny-initramfs. It is not meant to replace
those, but provide a light-weight solution in cases where the
complexities of a full initramfs are unnecessary.

Caveats

Since the initramfs is supposed to be small, fsck will not be
executed by tiny-initramfs. For the / file system this is
perfectly fine, as most distributions support checking the root file
system at boot outside of the initramfs. But this doesn't work for
/usr, because e.g. e2fsck will not check a mounted file system
other than the root file system; and e.g. systemd passes -M to
fsck by default for non-root file systems, so mounted file systems
are excluded anyway. It shouldn't be too difficult to special-case
/usr here as well, but that work needs to be done if a file system
check at boot is to be performed for /usr with tiny-initramfs.
(Note that e2fsck plus the required libraries are about 2.5 MiB in
size, so having fsck present in the initramfs image is not in the
scope of tiny-initramfs, because it would remove all the
advantages.)
You need to make sure that a split-/usr file system is remounted
read-write if the ro option is passed on the kernel command line
(because tiny-initramfs will also mount /usr read-only then);
otherwise /usr will remain read-only after boot. If you use
systemd as your init system, or e.g. the newest Debian initscripts
(2.88dsf-59.3 or higher) in conjunction with sysvinit, this should
work. tiny-initramfs itself doesn't care about which init system
is used, but the init system must be able to cope with the state
that tiny-initramfs leaves the /usr file system in. This may
require changes to some scripts.
If /usr is a bind mount in /etc/fstab, this will currently fail,
even though it should be supportable. (It's on the TODO list, as
long as that doesn't require yet another file system.)
Overlay-type file systems for /usr are untested, but they should
work if they are compiled into the kernel. What is not supported
are overlay-type file systems for / and/or if something has to be
done prior to mounting these file systems (such as creating a
directory, or mounting an additional tmpfs or similar).
Booting from kernel-assembled RAID arrays (via md=...) should
work, but is untested. Don't combine this with UUID=, though, as
tiny-initrd currently does not check if a block device has array
metadata, so it could falsely identify a member device (instead of
the entire array) when using UUID= in some cases. But for arrays
that are assembled by the kernel via md=... the device name is
known anyway (typically /dev/md0), so this shouldn't be an issue.
NFS support is not thoroughly tested.
Host names are unsupported for NFS mounts, only IP addresses work.
While this is supposed to be portable, this has only been tested on
x86_64 (amd64). Since low-level kernel syscalls are performed, there
may be some issues on other architectures. Please report those if
they are present.

HOWTO

Install an alternative libc implementation that's designed for embedded
use cases, such as musl or
dietlibc. This is strictly speaking not
required, as the default glibc will also work, but then the binary size
of the resulting binary will be far larger. If tiny-initramfs doesn't
work with your favorite libc implementation, please report this, so
that it may be fixed.

Find out the compile command required to use your C library. For
example, with dietlibc it's "diet gcc", with musl it's musl-gcc.

Use

./configure CC="diet gcc"
make

to compile the tiny_initramfs binary and

make initrd.img

to auto-create the initramfs image. Replace "diet gcc" with the
appropriate command for your libc implementation.

Note that if you specify CFLAGS (potentially via your build system)
you should take care to specify -Os and not to specify any debug
(-g) options, as those tend to increase the binary size quite a bit.
./configure will warn you about it, but it not abort in that case,
because the binary will work. Likewise, if you don't use an alternative
libc implementation but glibc, ./configure will warn you about it,
because that will increase the binary size by a factor of 10 to 20.

You may specify multiple options to enable/disable certain features in
the initramfs. Specifically, you can disable UUID mounting support
(enabled by default), and you can enable NFSv4 support (disabled by
default). A list of possible options is displayed when using

./configure --help

The initramfs creation is really simple, you may also do so manually:

create an empty directory, let's call it initramfs/
copy tiny_initramfs to the directory, call it init (you can
call it something else, but then you also need to pass the
rdinit=/newname option to the kernel command line)
strip the binary to reduce it's size (optional)
create the dev, proc and target subdirectories
cpio the directory and compress it

The following commands do just that:

mkdir initramfs
cp tiny_initramfs initramfs/init
strip initramfs/init
mkdir initramfs/dev initramfs/proc initramfs/target
cd initramfs ; find . | cpio -o --quiet -R 0:0 -H newc | gzip > ../initrd.img

With this there's now a (kernel-independent) initramfs image that may
be used to boot the system. Note that as of now there is no integration
with distributions, so configuring the boot loader etc. has to be done
manually.

Support for loading modules

There is limited support for loading modules if --enable-modules is
specified during the configure invocation. To use this feature, one
needs to create a file /modules in the initramfs image that is of the
following format:

/file.ko options

The modules should not be in a sub-directory, because the directory
containing them will not be cleaned-up by tiny-initramfs after mounting
the root file system. (Loading the modules will work though.)

For example, the virtio block device driver virtio_blk requires some
additional modules to work. Using modprobe one may find out which:

$ /sbin/modprobe --all --ignore-install --quiet --show-depends virtio_blk
insmod /lib/modules/[...]/kernel/drivers/virtio/virtio.ko
insmod /lib/modules/[...]/kernel/drivers/virtio/virtio_ring.ko
insmod /lib/modules/[...]/kernel/drivers/block/virtio_blk.ko

It turns out that this is not quite sufficient, because the
virtio_pci driver is also required for virtio_blk to work (the
driver loads without virtio_pci, but doesn't work), so one may use:

$ /sbin/modprobe --all --ignore-install --quiet --show-depends virtio_blk virtio_pci
insmod /lib/modules/3.16.0-4-amd64/kernel/drivers/virtio/virtio.ko
insmod /lib/modules/3.16.0-4-amd64/kernel/drivers/virtio/virtio_ring.ko
insmod /lib/modules/3.16.0-4-amd64/kernel/drivers/block/virtio_blk.ko
insmod /lib/modules/3.16.0-4-amd64/kernel/drivers/virtio/virtio.ko
insmod /lib/modules/3.16.0-4-amd64/kernel/drivers/virtio/virtio_ring.ko
insmod /lib/modules/3.16.0-4-amd64/kernel/drivers/virtio/virtio_pci.ko

(Note that in case soft dependencies are treated via install lines,
these have to be resolved manually. This is typically not the case for
drivers needed within initramfs, because other implementations also
suffer from the same issue. install is deprecated anyway according to
the manual page of modprobe.d.)

One may then copy these drivers to the initramfs image, and then add
a module file with the following contents:

/virtio.ko
/virtio_ring.ko
/virtio_blk.ko
/virtio.ko
/virtio_ring.ko
/virtio_pci.ko

The order is important, because dependency resolution is not
performed by tiny-initramfs, it has to be done while creating the
initramfs image. Duplicate entries are not a problem, because they will
silently be ignored (but you may remove duplicate entries if you don't
change the order otherwise).

Options may be specified when followed by a space (tab characters not
supported), for example:

/libata.ko noacpi

Debugging

If an error occurs, tiny-initramfs will print an error message
indicating the problem and then sleep for 10s before exiting. This is
because exiting will cause a kernel panic, but typical kernel traces
are so large that they replace the entire screen contents on a standard
terminal, so that the original message is not visible anymore. The 10s
delay allows the user to see what the problem is.

Additionally, one may use the --enable-debug flag of ./configure to
make initrd.img verbose (while increasing the size a bit). This makes
debugging easier, especially if the system hangs at a certain point.
When compiled with that option, tiny-initramfs will print the contents
of /proc/self/mountinfo and sleep for 5s after mounting the root (and
potentially /usr) file systems before executing init.

Design considerations

The design of tiny-initramfs is as minimalistic as possible. The
buffered I/O functions from the stdio.h standard library are
completely avoided, because they can increase the code size quite a
bit, depending on the libc implementation. At one point an own
minimalistic buffered I/O routine is implemented (much smaller than the
full standard library linked in).

Dynamic allocations are avoided and buffers on the stack are used. Code
that properly handles dynamic allocations tends to be longer, so this
reduces code size. The buffers are sized generously (there are not that
many buffers that the amount of RAM used is a concern just yet, even
for small systems), so that no real flexibility is sacrificed.

None of this is extremely performance-critical (it is going to be quite
fast regardless, because very little is done compared to even just
running a shell), so no algorithm is optimized for speed directly. For
example, the mount option parser table is somewhat compressed to reduce
the code size (negation and recursive variants of mount options are not
repeated), to the point where further reduction would likely sacrifice
the readability of the code. Execution speed is achieved by doing very
little, not by micro-optimizing algorithms.

Sometimes it is necessary to reimplement certain libc functions because
using the libc variants increase the image size too much. For example,
using inet_ntoa and inet_aton (to convert between ASCII to binary
representations of IP addresses) from the musl C library will cause
initramfs images (after compression) to be an additional 5 KiB larger
as compared to the own implementation.

Of course, changes that reduce the current code size even further (as
long as the code remains readable) are very welcome.

Future features

There is no goal of adding too many additional features here, because
any additional feature is going to increase the binary size, and this
is supposed to be minimalistic and not a replacement for a full
initramfs. If you need advanced features, please use an already
existing solution. That said, there are a couple of things that might
be interesting regardless:

Minimalistic NFSv2/3 mounting support (akin to the current NFSv4
code).
Maybe support host name lookups for NFS mounts? (Probably not going
to happen, as an own DNS resolver will likely increase the image
size by too much - and is likely going to be rather complicated.)
Support UUID= for more filesystems, as long as they are really
simple. Currently, the implementation checks the magic bytes of a
given file system on the each device, and then compares the UUID at
the right position in the file system metadata. (See devices.c for
details on how this is implemented for the currently supported file
systems.)
Support for excluding MD/RAID/... devices when probing for UUIDs of
file systems.

Note that the goal is to keep the initrd.img size smaller than 16 KiB
on all platforms, so a cutoff of 15 KiB is used on x86_64, to leave
room for different assembly code sizes etc., at least when used in a
minimal configuration. Therefore, some features (such as UUID= and
NFSv4 support) are compile-time optional.

Note: any features missing from tiny-initramfs that would be required
in a space-constrained environment (i.e. mainly embedded), where it was
designed for, stand an excellent chance of being included later, at
least compile-time optional. Please make your case if you are missing
something.

TODO

bind mounts for /usr
clean up the code a bit.
go through all messages printed and make sure they are uniform in
style

agamez/tiny-initramfs