Latest revision as of 11:31, 3 April 2021

Building a minimal RootFS with Busybox, GLIBC and DropBear

BusyBox is a collection of cut down versions of common UNIX utilities compiled into a single small executable. This makes BusyBox an ideal foundation for resource constrained systems.

Prerequisites

Install the following prerequisites (assuming an Ubuntu 14.04 build machine):

apt-get install gcc-arm-linux-gnueabi
apt-get install libncurses5-dev
apt-get install gawk

BusyBox

BusyBox can be built either as a single static binary requiring no external libraries, or built requiring shared libraries such as GLIBC (default). This setting can be found under BusyBox Settings -> Build Options -> Build BusyBox as a static binary (no shared libs).

I generally choose to build BusyBox to require GLIBC as it is highly likely you will want to run additional applications that will require GLIBC sometime in the future.

wget https://busybox.net/downloads/busybox-1.32.1.tar.bz2
tar -xjf busybox-1.32.1.tar.bz2
cd busybox-1.32.1/
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig

At the menu, you can configure BusyBox options. Once configured, you can build BusyBox:

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- 
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- install CONFIG_PREFIX=/home/export/rootfs

GLIBC

GLIBC is the GNU C Library and includes common system calls required by executables running on your system.

Download, build and install GLIBC:

wget http://ftp.gnu.org/gnu/libc/glibc-2.33.tar.xz
tar -xJf glibc-2.33.tar.xz
mkdir glibc-build
cd glibc-build/
../glibc-2.33/configure arm-linux-gnueabi --build=i686-pc-linux-gnu CXX=arm-linux-gnueabi-g++ --prefix= --enable-add-ons
make
make install install_root=/home/export/rootfs 

Some programs may require libgcc_s.so, otherwise you will receive an error:

error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory 

libgcc_s.so.1 can be copied over from your arm-linux-gnueabi installation:

cp /usr/lib/gcc-cross/arm-linux-gnueabi/4.7.3/libgcc_s.so.1 /home/export/rootfs/lib 

Preparing RootFS

Once BusyBox and GLIBC has been cross-compiled, you will want to create the remainder of the root file system. Start by creating the necessary directory structure:

mkdir proc sys dev etc/init.d usr/lib

Now we must mount the /proc & /sys filesystem and populate the /dev nodes. This can be done at runtime by creating a file called etc/init.d/rcS and adding:

#!bin/sh
mount -t proc none /proc
mount -t sysfs none /sys
echo /sbin/mdev > /proc/sys/kernel/hotplug
/sbin/mdev -s

and make executable:

chmod +x etc/init.d/rcS 

You should now have a basic, yet quite functional, BusyBox root file system.

DropBear (Optional)

DropBear is a small SSH server and client and is useful to allow remote shell access to your system.

Download, build and install DropBear:

wget https://matt.ucc.asn.au/dropbear/releases/dropbear-2020.81.tar.bz2
tar -xjf dropbear-2020.81.tar.bz2
cd dropbear-2020.81
./configure --host=arm-linux-gnueabi --prefix=/ --disable-zlib CC=arm-linux-gnueabi-gcc LD=arm-linux-gnueabi-ld
make
make install DESTDIR=/home/export/rootfs

DropBear requires RSA and DSS (Digital Signature Standard) encryption keys to be generated. I normally do this on the target, but you could generate the keys on the host if you have the dropbearkey executable installed.

To generate your keys:

mkdir /etc/dropbear
dropbearkey -t dss -f /etc/dropbear/dropbear_dss_host_key  
dropbearkey -t rsa -f /etc/dropbear/dropbear_rsa_host_key 

You will also require users and passwords to validate login credentials:

touch /etc/passwd
touch /etc/group
adduser root -u 0

Unless otherwise specified, root will be given a default home directory of /home/root. However as this doesn't exist, DropBear will close your connection immediately after successfully logging in. To address this, simply create a home directory for root:

mkdir /home /home/root

DropBear can now be started by running:

dropbear

and you should be able to remotely login to your system using the root user.

If you get an error after logging in, "Server refused to allocate pty" check you have Device Drivers > Character devices > Legacy (BSD) PTY support enabled in your kernel. (Especially applicable to Beaglebone kernels)

Alternatively, make sure you have mounted the /dev/pts filesystem (after /dev has been mounted)

mkdir /dev/pts
mount -t devpts none /dev/pts

ldconfig

ldconfig is used to configure dynamic linker run-time bindings. It creates symbolic links and a cache to the most recent shared libraries. As you build upon your root filesystem and add additional libraries, you may need to run ldconfig.

ldconfig will search for libraries in the trusted directory /lib. Additional search paths can be added to the ld.so.conf configuration file. ldconfig looks for a configuration file in /etc/ld.so.conf and generates a warning if this cannot be found. Suppress the warning and extend the search range to include /usr/lib by:

echo /usr/lib > etc/ld.so.conf

ldconfig will also generate a cache at /etc/ld.so.cache. If this file doesn't exist, it will be automatically generated.

Finally, to update the dynamic linker run-time bindings with verbose output, execute:

ldconfig -v

Read Only Filesystems

If you are using a root file system residing on flash memory, to improve longevity it may be desirable to either mount your rootfs as read only, or to move as many of the frequently written to files (for example /var) to a temporary volatile file system stored in RAM.

The following is an expanded etc/init.d/rcS file demonstrating this.

We mount /var and /dev as tmpfs. The device nodes are generated using mdev at boot and from hotplug, hence we move these to a temp file system otherwise they could not be generated on a read only root filesystem.

#!bin/sh
mount -t proc none /proc
mount -t sysfs none /sys
mount -t tmpfs none /var
mount -t tmpfs none /dev
mkdir /dev/pts
mount -t devpts none /dev/pts
echo /sbin/mdev > /proc/sys/kernel/hotplug
/sbin/mdev -s
mkdir /var/log
syslogd
dropbear

syslogd writes system messages to /var/log/messages and is something you probably don't want constantly writing to flash. After mounting /var as a temp file system, we create the /var/log directory for systemd.