(Updated to busybox 1.32.0)
 
(22 intermediate revisions by one user not shown)
Line 1: Line 1:
  
=== Building a minimal RootFS with Busybox ===
+
== Building a minimal RootFS with Busybox, GLIBC and DropBear ==
  
 
[http://www.busybox.net/ BusyBox] is a collection of cut down versions of common UNIX utilities compiled into a single small executable. This makes BusyBox an ideal foundation for resource constrained systems.
 
[http://www.busybox.net/ BusyBox] is a collection of cut down versions of common UNIX utilities compiled into a single small executable. This makes BusyBox an ideal foundation for resource constrained systems.
  
== BusyBox ==
+
=== Prerequisites ===
 +
 
 +
Install the following prerequisites (assuming an Ubuntu 14.04 build machine):
 +
 
 +
<PRE>
 +
apt-get install gcc-arm-linux-gnueabi
 +
apt-get install libncurses5-dev
 +
apt-get install gawk
 +
</PRE>
 +
 
 +
=== BusyBox ===
  
 
BusyBox can be built either as a single static binary requiring no external libraries, or built requiring shared libraries such as GLIBC (default). This setting can be found under BusyBox Settings -> Build Options -> Build BusyBox as a static binary (no shared libs).  
 
BusyBox can be built either as a single static binary requiring no external libraries, or built requiring shared libraries such as GLIBC (default). This setting can be found under BusyBox Settings -> Build Options -> Build BusyBox as a static binary (no shared libs).  
  
I generally choose to build BusyBox to require GLIBC as it is highly likely you will want to run additional applications that will require GLIBC.  
+
I generally choose to build BusyBox to require GLIBC as it is highly likely you will want to run additional applications that will require GLIBC sometime in the future.  
  
 
<PRE>
 
<PRE>
wget http://busybox.net/downloads/busybox-1.22.1.tar.bz2
+
wget https://busybox.net/downloads/busybox-1.32.0.tar.bz2
tar -xjf busybox-1.22.1.tar.bz2
+
tar -xjf busybox-1.32.0.tar.bz2
cd busybox-1.22.1/
+
cd busybox-1.32.0/
 
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- defconfig
 
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- defconfig
 
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig
 
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig
Line 25: Line 35:
 
</PRE>
 
</PRE>
  
== GLIBC ==
+
=== GLIBC ===
  
 
GLIBC is the [http://www.gnu.org/software/libc/libc.html GNU C Library] and includes common system calls required by executables running on your system.
 
GLIBC is the [http://www.gnu.org/software/libc/libc.html GNU C Library] and includes common system calls required by executables running on your system.
Line 32: Line 42:
  
 
<PRE>
 
<PRE>
wget http://ftp.gnu.org/gnu/libc/glibc-2.19.tar.gz
+
wget http://ftp.gnu.org/gnu/libc/glibc-2.22.tar.xz
tar -xzf glibc-2.19.tar.gz
+
tar -xJf glibc-2.22.tar.xz
 
mkdir glibc-build
 
mkdir glibc-build
 
cd glibc-build/
 
cd glibc-build/
../glibc-2.19/configure arm-linux-gnueabi --target=arm-linux-gnueabi --prefix= --enable-add-ons
+
../glibc-2.22/configure arm-linux-gnueabi --target=arm-linux-gnueabi --build=i686-pc-linux-gnu --prefix= --enable-add-ons
 
make
 
make
 
make install install_root=/home/export/rootfs  
 
make install install_root=/home/export/rootfs  
 
</PRE>
 
</PRE>
  
If you get an error similar to that below with cross-rpcgen, then it would appear cross-rpcgen was built for arm, but is trying to run on your x86 based build system. To alleviate the problem I pre-build a copy for x86 and place it in /glibc-build/sunrpc/cross-rpcgen and then restart the arm build.
+
Some programs may require libgcc_s.so, otherwise you will receive an error:
 +
<PRE>
 +
error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory
 +
</PRE>
 +
 
 +
libgcc_s.so.1 can be copied over from your arm-linux-gnueabi installation:
  
 
<PRE>
 
<PRE>
CPP='arm-linux-gnueabi-gcc -E -x c-header' /.../glibc-build/sunrpc/cross-rpcgen -Y ../scripts -c rpcsvc/bootparam_prot.x -o /.../glibc-build/sunrpc/xbootparam_prot.T
+
cp /usr/lib/gcc-cross/arm-linux-gnueabi/4.7.3/libgcc_s.so.1 /home/export/rootfs/lib
/lib/ld-linux.so.3: No such file or directory
+
make[2]: *** [/.../glibc-build/sunrpc/xbootparam_prot.stmp] Error 255
+
 
</PRE>
 
</PRE>
  
== Preparing RootFS ==
+
=== Preparing RootFS ===
  
 
Once BusyBox and GLIBC has been cross-compiled, you will want to create the remainder of the root file system. Start by creating the necessary directory structure:
 
Once BusyBox and GLIBC has been cross-compiled, you will want to create the remainder of the root file system. Start by creating the necessary directory structure:
  
 
<PRE>
 
<PRE>
mkdir proc sys dev etc etc/init.d
+
mkdir proc sys dev etc/init.d usr/lib
 
</PRE>
 
</PRE>
  
Now we must mount the /proc & /sys filesystem and populate the /dev nodes. This can be done at runtime by creating a file called /etc/init.d/rcS and adding:
+
Now we must mount the /proc & /sys filesystem and populate the /dev nodes. This can be done at runtime by creating a file called etc/init.d/rcS and adding:
  
 
<PRE>
 
<PRE>
Line 63: Line 76:
 
mount -t proc none /proc
 
mount -t proc none /proc
 
mount -t sysfs none /sys
 
mount -t sysfs none /sys
 +
echo /sbin/mdev > /proc/sys/kernel/hotplug
 
/sbin/mdev -s
 
/sbin/mdev -s
 
</PRE>
 
</PRE>
Line 69: Line 83:
  
 
<PRE>
 
<PRE>
chmod +x /etc/init.d/rcS  
+
chmod +x etc/init.d/rcS
 +
</PRE>
 +
 
 +
You should now have a basic, yet quite functional, BusyBox root file system.
 +
 
 +
=== DropBear (Optional) ===
 +
 
 +
[https://matt.ucc.asn.au/dropbear/dropbear.html DropBear] is a small SSH server and client and is useful to allow remote shell access to your system.
 +
 
 +
Download, build and install DropBear:
 +
 
 +
 
 +
<PRE>
 +
wget https://matt.ucc.asn.au/dropbear/releases/dropbear-2015.68.tar.bz2
 +
tar -xjf dropbear-2015.68.tar.bz2
 +
cd dropbear-2015.68
 +
./configure --host=arm-linux-gnueabi --prefix=/ --disable-zlib CC=arm-linux-gnueabi-gcc LD=arm-linux-gnueabi-ld
 +
make
 +
make install DESTDIR=/home/export/rootfs
 +
</PRE>
 +
 
 +
DropBear requires RSA and DSS (Digital Signature Standard) encryption keys to be generated. I normally do this on the target, but you could generate the keys on the host if you have the dropbearkey executable installed.
 +
 
 +
To generate your keys:
 +
<PRE>
 +
mkdir /etc/dropbear
 +
dropbearkey -t dss -f /etc/dropbear/dropbear_dss_host_key 
 +
dropbearkey -t rsa -f /etc/dropbear/dropbear_rsa_host_key
 +
</PRE>
 +
 
 +
You will also require users and passwords to validate login credentials:
 +
 
 +
<PRE>
 +
touch /etc/passwd
 +
touch /etc/group
 +
adduser root -u 0
 +
</PRE>
 +
 
 +
Unless otherwise specified, root will be given a default home directory of /home/root. However as this doesn't exist, DropBear will close your connection immediately after successfully logging in. To address this, simply create a home directory for root:
 +
 
 +
<PRE>
 +
mkdir /home /home/root
 +
</PRE>
 +
 
 +
DropBear can now be started by running:
 +
 
 +
<PRE>
 +
dropbear
 +
</PRE>
 +
 
 +
and you should be able to remotely login to your system using the root user.
 +
 
 +
If you get an error after logging in, "Server refused to allocate pty" check you have Device Drivers > Character devices > Legacy (BSD) PTY support enabled in your kernel. (Especially applicable to Beaglebone kernels)
 +
 
 +
=== ldconfig ===
 +
 
 +
ldconfig is used to configure dynamic linker run-time bindings. It creates symbolic links and a cache to the most recent shared libraries. As you build upon your root filesystem and add additional libraries, you may need to run ldconfig.
 +
 
 +
ldconfig will search for libraries in the trusted directory /lib. Additional search paths can be added to the ld.so.conf configuration file. ldconfig looks for a configuration file in /etc/ld.so.conf and generates a warning if this cannot be found. Suppress the warning and extend the search range to include /usr/lib by:
 +
 
 +
<PRE>
 +
echo /usr/lib > etc/ld.so.conf
 +
</PRE>
 +
 
 +
ldconfig will also generate a cache at /etc/ld.so.cache. If this file doesn't exist, it will be automatically generated.
 +
 
 +
Finally, to update the dynamic linker run-time bindings with verbose output, execute:
 +
 
 +
<PRE>
 +
ldconfig -v
 +
</PRE>
 +
 
 +
=== Read Only Filesystems ===
 +
 
 +
If you are using a root file system residing on flash memory, to improve longevity it may be desirable to either mount your rootfs as read only, or to move as many of the frequently written to files (for example /var) to a temporary volatile file system stored in RAM. 
 +
 
 +
The following is an expanded etc/init.d/rcS file demonstrating this.
 +
 
 +
We mount /var and /dev as tmpfs. The device nodes are generated using mdev at boot and from hotplug, hence we move these to a temp file system otherwise they could not be generated on a read only root filesystem.
 +
 
 +
<PRE>
 +
#!bin/sh
 +
mount -t proc none /proc
 +
mount -t sysfs none /sys
 +
mount -t tmpfs none /var
 +
mount -t tmpfs none /dev
 +
echo /sbin/mdev > /proc/sys/kernel/hotplug
 +
/sbin/mdev -s
 +
mkdir /var/log
 +
syslogd
 +
dropbear
 
</PRE>
 
</PRE>
  
You should now have a basic, yet functional, BusyBox root file system.
+
syslogd writes system messages to /var/log/messages and is something you probably don't want constantly writing to flash. After mounting /var as a temp file system, we create the /var/log directory for systemd.

Latest revision as of 06:10, 2 August 2020

Building a minimal RootFS with Busybox, GLIBC and DropBear

BusyBox is a collection of cut down versions of common UNIX utilities compiled into a single small executable. This makes BusyBox an ideal foundation for resource constrained systems.

Prerequisites

Install the following prerequisites (assuming an Ubuntu 14.04 build machine):

apt-get install gcc-arm-linux-gnueabi
apt-get install libncurses5-dev
apt-get install gawk

BusyBox

BusyBox can be built either as a single static binary requiring no external libraries, or built requiring shared libraries such as GLIBC (default). This setting can be found under BusyBox Settings -> Build Options -> Build BusyBox as a static binary (no shared libs).

I generally choose to build BusyBox to require GLIBC as it is highly likely you will want to run additional applications that will require GLIBC sometime in the future.

wget https://busybox.net/downloads/busybox-1.32.0.tar.bz2
tar -xjf busybox-1.32.0.tar.bz2
cd busybox-1.32.0/
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- defconfig
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- menuconfig

At the menu, you can configure BusyBox options. Once configured, you can build BusyBox:

make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- 
make ARCH=arm CROSS_COMPILE=arm-linux-gnueabi- install CONFIG_PREFIX=/home/export/rootfs

GLIBC

GLIBC is the GNU C Library and includes common system calls required by executables running on your system.

Download, build and install GLIBC:

wget http://ftp.gnu.org/gnu/libc/glibc-2.22.tar.xz
tar -xJf glibc-2.22.tar.xz
mkdir glibc-build
cd glibc-build/
../glibc-2.22/configure arm-linux-gnueabi --target=arm-linux-gnueabi --build=i686-pc-linux-gnu --prefix= --enable-add-ons
make
make install install_root=/home/export/rootfs 

Some programs may require libgcc_s.so, otherwise you will receive an error:

error while loading shared libraries: libgcc_s.so.1: cannot open shared object file: No such file or directory 

libgcc_s.so.1 can be copied over from your arm-linux-gnueabi installation:

cp /usr/lib/gcc-cross/arm-linux-gnueabi/4.7.3/libgcc_s.so.1 /home/export/rootfs/lib 

Preparing RootFS

Once BusyBox and GLIBC has been cross-compiled, you will want to create the remainder of the root file system. Start by creating the necessary directory structure:

mkdir proc sys dev etc/init.d usr/lib

Now we must mount the /proc & /sys filesystem and populate the /dev nodes. This can be done at runtime by creating a file called etc/init.d/rcS and adding:

#!bin/sh
mount -t proc none /proc
mount -t sysfs none /sys
echo /sbin/mdev > /proc/sys/kernel/hotplug
/sbin/mdev -s

and make executable:

chmod +x etc/init.d/rcS 

You should now have a basic, yet quite functional, BusyBox root file system.

DropBear (Optional)

DropBear is a small SSH server and client and is useful to allow remote shell access to your system.

Download, build and install DropBear:


wget https://matt.ucc.asn.au/dropbear/releases/dropbear-2015.68.tar.bz2
tar -xjf dropbear-2015.68.tar.bz2
cd dropbear-2015.68
./configure --host=arm-linux-gnueabi --prefix=/ --disable-zlib CC=arm-linux-gnueabi-gcc LD=arm-linux-gnueabi-ld
make
make install DESTDIR=/home/export/rootfs

DropBear requires RSA and DSS (Digital Signature Standard) encryption keys to be generated. I normally do this on the target, but you could generate the keys on the host if you have the dropbearkey executable installed.

To generate your keys:

mkdir /etc/dropbear
dropbearkey -t dss -f /etc/dropbear/dropbear_dss_host_key  
dropbearkey -t rsa -f /etc/dropbear/dropbear_rsa_host_key 

You will also require users and passwords to validate login credentials:

touch /etc/passwd
touch /etc/group
adduser root -u 0

Unless otherwise specified, root will be given a default home directory of /home/root. However as this doesn't exist, DropBear will close your connection immediately after successfully logging in. To address this, simply create a home directory for root:

mkdir /home /home/root

DropBear can now be started by running:

dropbear

and you should be able to remotely login to your system using the root user.

If you get an error after logging in, "Server refused to allocate pty" check you have Device Drivers > Character devices > Legacy (BSD) PTY support enabled in your kernel. (Especially applicable to Beaglebone kernels)

ldconfig

ldconfig is used to configure dynamic linker run-time bindings. It creates symbolic links and a cache to the most recent shared libraries. As you build upon your root filesystem and add additional libraries, you may need to run ldconfig.

ldconfig will search for libraries in the trusted directory /lib. Additional search paths can be added to the ld.so.conf configuration file. ldconfig looks for a configuration file in /etc/ld.so.conf and generates a warning if this cannot be found. Suppress the warning and extend the search range to include /usr/lib by:

echo /usr/lib > etc/ld.so.conf

ldconfig will also generate a cache at /etc/ld.so.cache. If this file doesn't exist, it will be automatically generated.

Finally, to update the dynamic linker run-time bindings with verbose output, execute:

ldconfig -v

Read Only Filesystems

If you are using a root file system residing on flash memory, to improve longevity it may be desirable to either mount your rootfs as read only, or to move as many of the frequently written to files (for example /var) to a temporary volatile file system stored in RAM.

The following is an expanded etc/init.d/rcS file demonstrating this.

We mount /var and /dev as tmpfs. The device nodes are generated using mdev at boot and from hotplug, hence we move these to a temp file system otherwise they could not be generated on a read only root filesystem.

#!bin/sh
mount -t proc none /proc
mount -t sysfs none /sys
mount -t tmpfs none /var
mount -t tmpfs none /dev
echo /sbin/mdev > /proc/sys/kernel/hotplug
/sbin/mdev -s
mkdir /var/log
syslogd
dropbear

syslogd writes system messages to /var/log/messages and is something you probably don't want constantly writing to flash. After mounting /var as a temp file system, we create the /var/log directory for systemd.